github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 09:48:58 -04:00
Code Issues Projects Releases Packages Wiki Activity

Force .kibana_task_manager reindex by specifying a new name (#185005)

Browse source 
## Summary

Addresses https://github.com/elastic/kibana/issues/163253 (see
[comment](https://github.com/elastic/kibana/issues/163253#issuecomment-2154576064))

TLDR: The PR renames the task manager SO index, forcing a reindex, and
preventing the `deleteByQuery`.

**Long story**

* The fact that `.kibana_task_manager` SO index is compatible since
7.14.0 triggers a compatible migration path for that migrator.
  * This includes a cleanup step for old and excluded SO types.
* This translates to a `deleteByQuery()`, which creates an async task on
ES.
  * ES stores this task internally in an index.
---
* The test at stakes is carefully limiting **shards_per_node**,
effectively limiting amount of indices that we can create.
* All migrators run in parallel, so whilst some of them are cloning (to
the expected failure), the `.kibana_task_manager` migrator is/was
running a compatible migration, which also requires an extra index.
* This creates a race condition, where either the _clone_ operations
(expected!) or the _deleteByQuery_ (flaky!) can fail arbitrarily,
depending on who attempts to create the index first, hence the flakiness
(well, this is my theory).
This commit is contained in:
Gerard Soldevila 2024-06-10 12:35:39 +02:00 • committed by GitHub
parent 8cd4858d2f
commit 67cd12e8ce
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 5 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

10
src/core/server/integration_tests/saved_objects/migrations/group3/split_failed_to_clone.test.ts
View file

@ -14,6 +14,8 @@ import {
MAIN_SAVED_OBJECT_INDEX,
} from '@kbn/core-saved-objects-server';
import { DEFAULT_INDEX_TYPES_MAP } from '@kbn/core-saved-objects-base-server-internal';
import type { CloneIndexParams } from '@kbn/core-saved-objects-migration-server-internal/src/actions';
import {
clearLog,
startElasticsearch,
@ -25,7 +27,6 @@ import {
} from '../kibana_migrator_test_kit';
import { delay } from '../test_utils';
import '../jest_matchers';
import { CloneIndexParams } from '@kbn/core-saved-objects-migration-server-internal/src/actions';
// mock clone_index from packages/core
jest.mock('@kbn/core-saved-objects-migration-server-internal/src/actions/clone_index', () => {
@ -51,7 +52,7 @@ const RELOCATE_TYPES: Record<string, string> = {
visualization: '.kibana_slow_clone_1',
'canvas-workpad': '.kibana_slow_clone_1',
search: '.kibana_slow_clone_2',
task: '.kibana_task_manager',
task: '.kibana_task_manager_new', // force reindex
'epm-packages-assets': '.kibana_slow_clone_1',
// the remaining types will be forced to go to '.kibana',
// overriding `indexPattern: foo` defined in the registry
@ -59,8 +60,7 @@ const RELOCATE_TYPES: Record<string, string> = {
export const logFilePath = Path.join(__dirname, 'split_failed_to_clone.test.log');
// Failing: See https://github.com/elastic/kibana/issues/163253
describe.skip('when splitting .kibana into multiple indices and one clone fails', () => {
describe('when splitting .kibana into multiple indices and one clone fails', () => {
let esServer: TestElasticsearchUtils['es'];
let typeRegistry: ISavedObjectTypeRegistry;
let migratorTestKitFactory: () => Promise<KibanaMigratorTestKit>;
@ -139,7 +139,7 @@ describe.skip('when splitting .kibana into multiple indices and one clone fails'
);
// remove the failure
await client.cluster.putSettings({ persistent: { 'cluster.max_shards_per_node': 20 } });
await client.cluster.putSettings({ persistent: { 'cluster.max_shards_per_node': 150 } });
const { runMigrations: runMigrations2ndTime } = await migratorTestKitFactory();
await runMigrations2ndTime();