## Summary
Part of #187684.
So far the popover to filter fields was only available when grouping was
enabled. This PR updates the behavior so it's available all the time and
can be used to exclude field candidates from the analysis. If we detect
the index to be based on an ECS schema, we auto-select a set of
predefined fields.
Changes in this PR:
- Creates a new route
`/internal/aiops/log_rate_analysis/field_candidates` to be able to fetch
field candidates independent of the main streaming API call.
- Fixes the code to consider "remaining" field candidates to also
consider text field candidates. This was originally developed to allow
to continue an analysis that errored for some reason. We use that option
to also pass on the custom field list from the field selection popover.
- Fetching the field candidates is done in a new redux slice
`logRateAnalysisFieldCandidatesSlice` using an async thunk.
- Filters the list of field candidates by a predefined field of allowed
fields when an ECS schema gets detected.
- Renames `fieldCandidates` to `keywordFieldCandidates` for clearer
distinction against `textFieldCandidates`.
- Refactors `getLogRateAnalysisTypeForCounts` args to a config object.
- Bump the API version for the full log rate analysis to version 3. We
missed bumping the version in
https://github.com/elastic/kibana/pull/188648. This update manages
proper versioning between v2 and v3, also the API integration tests
cover both versions.
[aiops-log-rate-analysis-fields-filter-0001.webm](https://github.com/user-attachments/assets/e3ed8d5b-f01c-42ef-8033-caa7135b8cc0)
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
This moves code from `plugins/aiops/common` to packages. The `aiops`
plugin will from now on have only a `server` and `public` directory.
This is in preparation for additional AIOps related public APIs and to
avoid cyclic dependency problems for other consuming plugins.
- Package `@kbn/aiops-utils` was renamed to `@kbn/aiops-common`.
- For each AIOps feature a package was created:
`@kbn/aiops-change-point-detection`, `@kbn/aiops-log-pattern-analysis`
and `@kbn/aiops-log-rate-analysis`.
### Checklist
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
This PR uses [conditional
types](https://www.typescriptlang.org/docs/handbook/2/conditional-types.html)
to allow the handling of both multiple API versions within one route handler. The more tricky bit turned out
to be not the updated request body, but the response since it is an
NDJSON stream where some messages were updated. In this case also the
functions that create these messages were updated with conditional types
to be able to create a message that fits the definition of the API
version.
The API integration tests originally had these message identifiers in
the `expected` section of their `testData`. I changed that to use helper
functions that retrieve the expected messages from the stream according
to the expected version. All API integration tests are run on both
versions. The functional tests are run only on the newer version since
the UI is expected to work with version `2` only.
## Summary
Makes version headers required for internal endpoints. We also require
version headers for public endpoints when in dev mode.
### Note to reviewers
This PR is a re-revert of the original
https://github.com/elastic/kibana/pull/158667 with some minor additions
(see comments).
The original was reverted due to failing Cypress tests blocking Kibana
promotion for 8.8.1 (CC @stephmilovic,
https://github.com/elastic/kibana/pull/158961)
Not sending headers to versioned, internal endpoints will return 400!
Due to the somewhat sensitive nature of this change, I went through all
of the existing `.versioned` endpoints and tried to ensure that for
_internal_ endpoints we send through a version as this is now
**required**.
I would greatly appreciate it if code owners could check their code,
think of any existing consumers of your versioned endpoints and ensure
they are sending a version.
Closes https://github.com/elastic/kibana/issues/158722
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Adds first functional tests for Explain Log Rate Spikes. The test clicks the menu item, selects an index, clicks the "Use full data" button and asserts the page's elements.
- Adds a check to aiops API endpoints to only allow requests with active platinum license.
- Adds integration tests for basic license where the endpoints should return permission denied.
- Improved error handling:
- Low level errors (like a non valid argument pushed to a stream) will now be logged to Kibana server's console, because the way HTTP streams work we cannot really emit a useful error to an already running stream to the client. So the stream will just abort but Kibana server will log an error.
- Higher level errors on the application level (like when we find out an index does not exist to run the analysis) will be pushed to the stream now as an error type action and we can update the UI accordingly. Note this PR only updates the API and corresponding tests to support this, the UI doesn't make use of it yet.
