Tim Sullivan 56383ccdde Use Data Stream for Reporting storage (#176022) ... ## Summary Closes https://github.com/elastic/kibana/issues/161608 * [X] Depends on https://github.com/elastic/elasticsearch/pull/97765 * [x] Depends on https://github.com/elastic/elasticsearch/pull/107581 * [x] Add create a new report job and check the details of the templated data stream. * [x] Run Discover tests in Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5999 ## Release Note Reporting internal storage has been changed from using regular indices to a data stream configuration for a more efficient sharding strategy. This change is not expected to have any impact to users. ## Screenshots ### Upgrade test (manual process) Using a report generated before this change, and a report generated after "upgrading":  Even though the two reports are in different types storage, they are still managed by the same policy:  Looking at the details of the policy shows how the different types of storage are used:  ### Log lines Initial startup in clean environment ``` [2024-05-13T13:22:49.138-07:00][INFO ][plugins.reporting.store] Creating ILM policy for reporting data stream: kibana-reporting [2024-05-13T13:22:53.337-07:00][INFO ][plugins.reporting.store] Linking ILM policy to reporting data stream: .kibana-reporting, component template: kibana-reporting@custom ``` Kibana restart with ES running continuously ``` [2024-05-13T13:24:32.733-07:00][DEBUG][plugins.reporting.store] Found ILM policy kibana-reporting; skipping creation. [2024-05-13T13:24:32.733-07:00][INFO ][plugins.reporting.store] Linking ILM policy to reporting data stream: .kibana-reporting, component template: kibana-reporting@custom ``` ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ~~See https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5302 (internal link)~~		2024-05-21 10:11:45 -07:00
..
images	[Reporting] Document steps to grant users reporting access under Basic (#127513)	2022-03-14 10:50:40 -07:00
automating-report-generation.asciidoc	Update automating-report-generation.asciidoc (#134410)	2022-06-15 13:01:14 -07:00
index.asciidoc	Use Data Stream for Reporting storage (#176022)	2024-05-21 10:11:45 -07:00
reporting-csv-limitations.asciidoc	[Docs] Clarifying CSV limitations (#175208)	2024-01-25 11:04:15 -05:00
reporting-troubleshooting.asciidoc	[DOCS] Update 'xpack.screenshotting.browser.chromium.disableSandbox` documentation (#148425)	2023-01-10 09:26:57 -05:00
response-codes.asciidoc	[DOCS] Changes Visualize to Visualize Library (#93055)	2021-03-08 13:44:30 -06:00
script-example.asciidoc	Use Data Stream for Reporting storage (#176022)	2024-05-21 10:11:45 -07:00
watch-example.asciidoc	[DOCS] Updates to the Reporting docs (#101326)	2021-06-30 12:10:04 -05:00