kibana/docs/user/reporting
Tim Sullivan 56383ccdde
Use Data Stream for Reporting storage (#176022)
## Summary

Closes https://github.com/elastic/kibana/issues/161608

* [X] Depends on https://github.com/elastic/elasticsearch/pull/97765
* [x] Depends on https://github.com/elastic/elasticsearch/pull/107581
* [x] Add create a new report job and check the details of the templated
data stream.
* [x] Run Discover tests in Flaky Test Runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5999

## Release Note

Reporting internal storage has been changed from using regular indices
to a data stream configuration for a more efficient sharding strategy.
This change is not expected to have any impact to users.

## Screenshots

### Upgrade test (manual process)
Using a report generated before this change, and a report generated
after "upgrading":

![image](f92193d8-70d6-4fa5-b1b7-8f6c1a0a5e9f)
Even though the two reports are in different types storage, they are
still managed by the same policy:

![image](9bd68d99-72ed-4cf0-bef9-55e644f039b7)
Looking at the details of the policy shows how the different types of
storage are used:

![image](6c0d1f80-97cb-4990-b2a8-45deab7528bc)

### Log lines

Initial startup in clean environment
```
[2024-05-13T13:22:49.138-07:00][INFO ][plugins.reporting.store] Creating ILM policy for reporting data stream: kibana-reporting
[2024-05-13T13:22:53.337-07:00][INFO ][plugins.reporting.store] Linking ILM policy to reporting data stream: .kibana-reporting, component template: kibana-reporting@custom
```

Kibana restart with ES running continuously
```
[2024-05-13T13:24:32.733-07:00][DEBUG][plugins.reporting.store] Found ILM policy kibana-reporting; skipping creation.
[2024-05-13T13:24:32.733-07:00][INFO ][plugins.reporting.store] Linking ILM policy to reporting data stream: .kibana-reporting, component template: kibana-reporting@custom
```

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
~~See
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5302
(internal link)~~
2024-05-21 10:11:45 -07:00
..
images [Reporting] Document steps to grant users reporting access under Basic (#127513) 2022-03-14 10:50:40 -07:00
automating-report-generation.asciidoc Update automating-report-generation.asciidoc (#134410) 2022-06-15 13:01:14 -07:00
index.asciidoc Use Data Stream for Reporting storage (#176022) 2024-05-21 10:11:45 -07:00
reporting-csv-limitations.asciidoc [Docs] Clarifying CSV limitations (#175208) 2024-01-25 11:04:15 -05:00
reporting-troubleshooting.asciidoc [DOCS] Update 'xpack.screenshotting.browser.chromium.disableSandbox` documentation (#148425) 2023-01-10 09:26:57 -05:00
response-codes.asciidoc [DOCS] Changes Visualize to Visualize Library (#93055) 2021-03-08 13:44:30 -06:00
script-example.asciidoc Use Data Stream for Reporting storage (#176022) 2024-05-21 10:11:45 -07:00
watch-example.asciidoc [DOCS] Updates to the Reporting docs (#101326) 2021-06-30 12:10:04 -05:00