## Summary
Background: https://github.com/elastic/kibana/pull/212173
Based off of feedback on the work in the PRs listed in that issue,
additional performance improvements can be made to the cells rendered in
the alert table. The changes made in this PR involve migrating out
shared context to a provider so certain hooks (some expensive... i.e.
browserFieldsByName) aren't made for every cell in the UI, but once and
passed down to each cell accordingly.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Closes#211783
Part of https://github.com/elastic/kibana/issues/195857
## Summary
This PR expands the logic to get the dashboard files based on the agent.
We have many different ways to ingest data so we want to add more
metrics dashboards to the APM metrics tab. The different ingest paths we
have:
Classic APM Agent --> APM Server --> ES
Vanilla OTel SDKs --> APM Server --> ES
EDOT OTel SDKs --> APM Server --> ES
Classic APM Agent --> EDOT Collector --> ES
Vanilla OTel SDKs. --> EDOT Collector --> ES
EDOT OTel SDKs --> EDOT Collector --> ES
We agreed on having a dashboard filename pattern to make showing the
correct dashboard easier described
[here](https://github.com/elastic/kibana/issues/195857#issue-2580733648)
First, we determine if the ingest path is through APM Server or EDOT
Collector by checking the `telemtry.sdk` fields.
## TODOs / Reviewer notes
- [ ] Currently, we have a fallback to metrics charts which is valid
only if we have APM agent so this PR adds an empty state message:
"Runtime metrics are not available for this Agent / SDK type." in case
there is no dashboard for the service language. To be improved in
https://github.com/elastic/kibana/issues/211774 and will be updated in
this PR when ready - I will still open it for review as the other logic
can be reviewed
- The dashboards are to be updated (by the agent team so not part of the
changes here)
## Testing:
- Using e2e PoC
- The available dashboard cases can be found in
[loadDashboardFile](91f169e19a/x-pack/solutions/observability/plugins/apm/public/components/app/metrics/static_dashboard/dashboards/dashboard_catalog.ts (L40))
- Cases to be checked:
- OTel native with Vanilla OTel SDKs with available dashboard (example
case file: `otel_native-otel_other-nodejs`, `...-java`, `...-dotnet`)
<img width="1903" alt="image"
src="https://github.com/user-attachments/assets/44d37b05-a8e7-4f14-a1de-2c631f1843bb"
/>
- APM server with Vanilla OTel SDKs service with available dashboard
(example case file: `classic_apm-otel_other-nodejs`, `...-java`,
`...-dotnet`)
- APM server with Classic APM Agent (example case file:
`classic_apm-apm-nodejs`, `...-java`)
<img width="962" alt="image"
src="https://github.com/user-attachments/assets/f9e96dce-55c8-467a-93f0-a09fa219597e"
/>
- OTel native with Vanilla OTel SDKs without available dashboard (empty
state case example: python service)
- APM server with Vanilla OTel SDKs service without available dashboard
(empty state)
<img width="1910" alt="image"
src="https://github.com/user-attachments/assets/5219cf94-5013-4874-aaea-e558cca69281"
/>
- APM server with Classic APM Agent without available dashboard (Current
metrics fallback)
<img width="1914" alt="image"
src="https://github.com/user-attachments/assets/66342f49-876c-4ad5-a4d1-1414c3abac75"
/>
- ⚠️ OTel native Dashboards are still not available (at the time of
adding the description)
---------
Co-authored-by: Sergi Romeu <sergi.romeu@elastic.co>
Co-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Backport
This will backport the following commits from `8.18` to `main`:
- [[SecuritySolution] Fix risk engine component template renaming
(#212853)](https://github.com/elastic/kibana/pull/212853)
<!--- Backport version: 9.6.6 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)
<!--BACKPORT [{"author":{"name":"Pablo
Machado","email":"pablo.nevesmachado@elastic.co"},"sourceCommit":{"committedDate":"2025-03-05T09:50:31Z","message":"[SecuritySolution]
Fix risk engine component template renaming (#212853)\n\n##
Summary\n\nThe previous implementation tried to rename the index
template during\n`init` and did not consider multiple spaces. to fix it,
I have:\n* Delete the previous code from `init`\n* Created a new
migration that created the new component templates and\nupdated the
index templates\n* Deleted the old component template after all spaces
migration ran\n* Add support for multiple spaces\n* I also renamed a
function inside `init` to make the code more clear\n* Added error
handling code that concatenates error messages and logs\nall of them at
the end\n\n\n### How to test it:\n\n**Scenario 1**\n\n* The usual way to
desk test this PR would be\n* Create a cluster with 8.17\n* Enable the
risk Engine in 8.17\n* Create a new space in 8.17\n* Upgrade the cluster
to 8.18 (this branch)\n* Enable the risk engine in the second
space.\n\n**Scenario 2**\n* Create a cluster with 8.17\n* Enable the
risk engine\n* Create a space\n* Enable another risk engine\n* Create
another space\n* Upgrade the cluster to 8.18 (this branch)\n* Check if
the migration ran in the logs\n* Check if all risk engines are installed
and the index templates and\nindex components are there.\n* Install a
new risk engine in the space where it isn't installed.\n* Restart Kibana
and make sure the migrations didn't run a second time\n\n###
Checklist\n\nReviewers should verify this PR satisfies this list as
well.\n\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [x] The PR
description includes the appropriate Release Notes section,\nand the
correct `release_note:*` label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n###
Identify risks\n\n[ ] This PR needs to be tested for upgrades between
different versions\nand a diverse number of spaces and risk engines
installed\n\n---------\n\nCo-authored-by: abhishekbhatia1710
<abhishek.bhatia@elastic.co>","sha":"b7908a4c6f91c79459f7b509bfd444ad169d6770","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","v9.0.0","Team:
SecuritySolution","Feature:Entity Analytics","Team:Entity
Analytics","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[SecuritySolution]
Fix risk engine component template
renaming","number":212853,"url":"https://github.com/elastic/kibana/pull/212853","mergeCommit":{"message":"[SecuritySolution]
Fix risk engine component template renaming (#212853)\n\n##
Summary\n\nThe previous implementation tried to rename the index
template during\n`init` and did not consider multiple spaces. to fix it,
I have:\n* Delete the previous code from `init`\n* Created a new
migration that created the new component templates and\nupdated the
index templates\n* Deleted the old component template after all spaces
migration ran\n* Add support for multiple spaces\n* I also renamed a
function inside `init` to make the code more clear\n* Added error
handling code that concatenates error messages and logs\nall of them at
the end\n\n\n### How to test it:\n\n**Scenario 1**\n\n* The usual way to
desk test this PR would be\n* Create a cluster with 8.17\n* Enable the
risk Engine in 8.17\n* Create a new space in 8.17\n* Upgrade the cluster
to 8.18 (this branch)\n* Enable the risk engine in the second
space.\n\n**Scenario 2**\n* Create a cluster with 8.17\n* Enable the
risk engine\n* Create a space\n* Enable another risk engine\n* Create
another space\n* Upgrade the cluster to 8.18 (this branch)\n* Check if
the migration ran in the logs\n* Check if all risk engines are installed
and the index templates and\nindex components are there.\n* Install a
new risk engine in the space where it isn't installed.\n* Restart Kibana
and make sure the migrations didn't run a second time\n\n###
Checklist\n\nReviewers should verify this PR satisfies this list as
well.\n\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [x] The PR
description includes the appropriate Release Notes section,\nand the
correct `release_note:*` label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n###
Identify risks\n\n[ ] This PR needs to be tested for upgrades between
different versions\nand a diverse number of spaces and risk engines
installed\n\n---------\n\nCo-authored-by: abhishekbhatia1710
<abhishek.bhatia@elastic.co>","sha":"b7908a4c6f91c79459f7b509bfd444ad169d6770"}},"sourceBranch":"8.18","suggestedTargetBranches":["9.0","main","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/212853","number":212853,"mergeCommit":{"message":"[SecuritySolution]
Fix risk engine component template renaming (#212853)\n\n##
Summary\n\nThe previous implementation tried to rename the index
template during\n`init` and did not consider multiple spaces. to fix it,
I have:\n* Delete the previous code from `init`\n* Created a new
migration that created the new component templates and\nupdated the
index templates\n* Deleted the old component template after all spaces
migration ran\n* Add support for multiple spaces\n* I also renamed a
function inside `init` to make the code more clear\n* Added error
handling code that concatenates error messages and logs\nall of them at
the end\n\n\n### How to test it:\n\n**Scenario 1**\n\n* The usual way to
desk test this PR would be\n* Create a cluster with 8.17\n* Enable the
risk Engine in 8.17\n* Create a new space in 8.17\n* Upgrade the cluster
to 8.18 (this branch)\n* Enable the risk engine in the second
space.\n\n**Scenario 2**\n* Create a cluster with 8.17\n* Enable the
risk engine\n* Create a space\n* Enable another risk engine\n* Create
another space\n* Upgrade the cluster to 8.18 (this branch)\n* Check if
the migration ran in the logs\n* Check if all risk engines are installed
and the index templates and\nindex components are there.\n* Install a
new risk engine in the space where it isn't installed.\n* Restart Kibana
and make sure the migrations didn't run a second time\n\n###
Checklist\n\nReviewers should verify this PR satisfies this list as
well.\n\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [x] The PR
description includes the appropriate Release Notes section,\nand the
correct `release_note:*` label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n###
Identify risks\n\n[ ] This PR needs to be tested for upgrades between
different versions\nand a diverse number of spaces and risk engines
installed\n\n---------\n\nCo-authored-by: abhishekbhatia1710
<abhishek.bhatia@elastic.co>","sha":"b7908a4c6f91c79459f7b509bfd444ad169d6770"}},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
## Summary
Fixes#212917
The root problem is belongs into the annotation layer logic to produce
the reference id for the persisted saved object.
In the previous logic a new `uuid` was generated all the time leading to
a continuous flow of `setState` calls to update the "runtime" state of
the Lens object when inline editing: the fix was to produce a stable id
in the `extractReferences` logic to avoid the re-renders.
The logic has been tweaked a bit now with some extra explanations inline
to make it more understandable.
New tests have been added to smoke test this scenario.
### Checklist
Check the PR satisfies following conditions.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Nick Partridge <nick.ryan.partridge@gmail.com>
The component was replace by an enablement dialog
## Summary
Delete the obsolete "enable risk score redirect" test.
The redirect button was replaced by an enablement dialog.
## Summary
###
* Fixes Bug: User with no cluster privileges should not be able to
enable the risk score
When users with no cluster privileges open the risk score page, they
don't see any errors and are able to click the install button.
This happened because we were only checking for index privileges in the
UI, but for the enablement flow we also need to check cluster
privileges. I also introduced a new parameter to the missing privileges
hook so pages that only need to check for `read` privileges can work as
before.
https://github.com/user-attachments/assets/fe162005-ee2b-497d-8744-6262e4511d2d
* Fixed Bug: The install button was enabled when all toggles were
disabled
There were too many booleans in the panel, which was confusing and led
me to introduce more bugs while trying to fix this one, so I refactored
the code to understand it before fixing it.
I also simplified the logic to display the modal. Now, it only shows
when one of the engines' status is "not_installed"
<img width="300"
src="https://github.com/user-attachments/assets/a2e8fbba-ac64-4c97-9ef0-ef6fe61e60cd"
/>
### To Reproduce
1. Create a user with security privileges and index privileges but no
cluster privileges
2. Go to the risk score page and enable the toggle
### Checklist
Check the PR satisfies following conditions.
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or
EmbeddableInput type is part of the legacy embeddable system. The legacy
embeddable system is being removed and as such, the EmbeddableInput type
is being removed.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
After upgrading the ES client to 9.0
(https://github.com/elastic/kibana/pull/208776), we noticed that the CI
fails to upload the results of the tests to the CI cluster:
```
ERROR ResponseError: media_type_header_exception
Caused by:
status_exception: Accept version must be either version 8 or 7, but found 9. Accept=application/vnd.elasticsearch+json; compatible-with=9
Root causes:
media_type_header_exception: Invalid media-type value on headers [Content-Type, Accept]
```
This PR makes sure that the CI client is still using v8.x until we
upgrade that cluster.
### Summary
This PR improves and fixes the TLS alert data handling in the Synthetics
plugin. Key updates include:
### Code changes
1. **Request Body Updates**:
- Added `agent.name`.
- Changed optional chaining to direct property access for
`ping.monitor.name`, `ping.monitor.id`, and `ping.observer.name`.
- Added `ping.agent.name`.
2. **Cert Type Adjustments**:
- Added fields: `monitorName`, `monitorId`, `monitorType`, `locationId`,
`locationName`, `@timestamp`, `hostName`.
3. **Observer Codec**:
- Ensured `name` and `geo.name` are required fields.
4. **Monitor Type and Ping Type**:
- Added `name` to `MonitorType`.
- Moved `@timestamp` to required fields in `PingType`.
7. **Message Utils**:
- Adjusted `getCertSummary` and `getTLSAlertDocument` so that we can
properly generate alert document
## Summary
Closes https://github.com/elastic/kibana/issues/191707
Summarize your PR. If it involves visual changes include a screenshot or
gif.
- Fixes flaky test`allows updating of an advanced setting` by making
sure to wait for page refresh
- Adds back test to check for toast on error
https://github.com/elastic/kibana/pull/191531
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com>
## Summary
This PR introduces validation rules for Case Observables, shared between
client and the server.
### Testing
- Create a case
- Add on observable to it, picking up the ipv4 as an observable type
(for instance)
- Verify that only the valid values are allowed.
- Try updating the observable after it is created, same validation rules
apply.
- Do the same thing using API routes.
## Summary
Partially addresses https://github.com/elastic/kibana/issues/210358
Adds all callouts and logic to incentivize users to upgrade their rules asap. These include:
- [x] Showing a callout on the Rule Management page
- [x] Showing a callout on the Rule Details page
- [x] Letting users open the Rule Upgrade flyout from the Rule Details page
- [x] Showing a callout on the Rule Editing page
- [x] Showing a callout in the Rule Upgrade flyout if rule has missing base version
This PR also adds related updates to the rule diff algorithms in order to facilitate an easier upgrade experience when rules have missing base versions. These include:
- [x] When the rule has a missing base version and is NOT marked as customized:
- [x] We should return all the target fields from the diff algorithm as NO_CONFLICT
- [x] When the rule has a missing base version and is marked as customized:
- [x] We should attempt to merge all non-functional mergeable fields (any field that doesn't have consequences with how the rule runs e.g. tags) and return them as `SOLVABLE_CONFLICT`.
- **NOTE**: When base versions are missing and the rule is customized, we attempt to merge all mergable, non-functional rule fields. These include all fields covered by the scalar diff array (`tags`, `references`, `new_terms_fields`, `threat_index`). We typically also consider multi-line string fields as mergeable but without three versions of the string, we are currently unable to merge the strings together, so we just return target version.
- [x] We should pick the target version for all functional mergeable fields (e.g. `index`) and non-mergeable fields and return them as `SOLVABLE_CONFLICT`.
### Screenshots
**Callout on Rule details page w/ flyout button**
---
**Upgrade flyout now accessible from rule details page**
---
**Callout on rule editing page**
---
**Dismissible callout on rule management page**
---
**Callout in rule upgrade flyout when rule has missing base version**
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
Follow-up to: https://github.com/elastic/kibana/pull/212077
This PR includes an API test that covers `get_alerts_dataset_info` and
would have caught the bug fixed in
https://github.com/elastic/kibana/pull/212077.
It also contains the following bug fixes:
- Fix system message in `select_relevant_fields`
- Change prompt in `select_relevant_fields` so that the LLM consistently
uses the right format when responding.
PR removes `dashboard_container` and adds `dashboard_renderer` folder. I
tried to find a best fit for files but have no strong opinions about
where I put things. Please comment if you would like files in different
locations.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
I noticed that a couple integration tests were not actually validating
whether the documents got routed the right way (the number and string
tests).
This PR fixes this by introducing a helper that can easily do the same
check we had a couple of times in there.
## Summary
Resolves https://github.com/elastic/kibana/issues/212972
This PR replaces the info and error log levels with debug since most of
these errors are for developers or users. Not operations.
## Summary
This is intended to be a "minimal" migration for Defend Insights to
langgraph + output chunking. Other than the increased events due to the
context increase from output chunking, the functionality is unchanged.
* migrates defend insights to langgraph
* adds output chunking / refinement
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
