This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [msw](https://mswjs.io)
([source](https://redirect.github.com/mswjs/msw)) | devDependencies |
patch | [`~2.7.0` ->
`~2.7.2`](https://renovatebot.com/diffs/npm/msw/2.7.3/2.7.2) | `2.7.3` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNsb3VkIFNlY3VyaXR5IiwiYmFja3BvcnQ6YWxsLW9wZW4iLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
## Summary
Removes the mSearch method from Dashboard content management.
The `mSearch` content management method was designed to be a temporary
implementation of search that allowed searching multiple saved object
types ([see more
[internal]](https://docs.google.com/document/d/1ssYmqSEUPrsuCR4iz8DohkEWekoYrm2yL4QR_fVxXLg/edit?tab=t.0#heading=h.6sj4n6bjcgp5)).
However, the mSearch implementation in the Dashboard Storage class lacks
extensibility as it requires a synchronous `toItemResult` function. As
we start migrating reference handling to the server, we will likely need
transforms that return Promises (ex. `savedObjectToItem`), such as
[retrieving tag saved objects from the SavedObjectTagging
client](https://github.com/elastic/kibana/issues/210619).
The Dashboard `mSearch` method was only used by the dashboard_picker and
this PR replaces its usage with the `search` method.
### Identify risks
There is a slight risk in serverless environments where a browser may
have already loaded the dashboard_picker module but lags behind the
server. In this case, the dashboard picker may fail to retrieve a list
of dashboards due to it calling the now non-existent `mSearch` method
provided by the server. In this case, the user simply needs to refresh
their browser to retrieve the latest UI modules.
**Resolves: https://github.com/elastic/kibana/issues/208361**
**Resolves: https://github.com/elastic/kibana/issues/210544**
## Summary
This PR introduces significant memory consumption improvements to the
prebuilt rule endpoints, ensuring users won't encounter OOM errors on
memory-limited Kibana instances.
Memory consumption testing results provided in
https://github.com/elastic/kibana/pull/211045#issuecomment-2689854328.
## Details
This PR implements a number of memory usage optimizations to the
prebuilt rule endpoints with the final goal reducing chances of getting
OOM errors. The changes are extensive and require thorough testing
before merging.
The changes are described by the following bullets
- The most significant change is the addition of pagination to the
`upgrade/_review` endpoint. This endpoint was known for causing OOM
errors due to its large and ever-growing response size. With pagination,
it now returns upgrade information for no more than 20-100 rules at a
time, significantly reducing its memory footprint.
- New backend methods, such as
`ruleObjectsClient.fetchInstalledRuleVersions`, have been introduced.
These methods return rule IDs with their corresponding installed
versions, allowing to build a map of outdated rules without loading all
available rules into memory. Previously, all installed rules, along with
their base and target versions, were fetched unconditionally before
filtering for updates.
- The `stats` data structure of the review endpoint has been deprecated
(it can be safely removed after one Serverless release cycle). Since the
endpoint now returns paginated results, building stats is no longer
feasible due to the limited rule set size fetched on the server side. As
the side effect it required removing related Cypress tests asserting
`Update All` disabled when rules can't be updated.
- All changes to the endpoints are backward-compatible. All previously
required returned structures still present in response. All newly added
structures are optional.
- Upgradeable rule tags are now returned from the prebuilt rule status
endpoint.
- The frontend logic has been updated to move sorting and filtering of
prebuilt rules from the client side to the server side.
- The `upgrade/_perform` endpoint has been rewritten to use lightweight
rule version information rather than full rules to determine upgradeable
rules. Additionally, upgrades are now performed in batches of up to 100
rules, further reducing memory usage.
- A dry run option has been added to the upgrade perform endpoint. This
is needed for the "Update all" rules scenario to determine if any rules
contain conflicts and display a confirmation modal to the user.
- An option to skip conflicting rules has been added to the upgrade
endpoint when called with the `ALL_RULES` mode.
- The `install/_review` endpoint's memory consumption has been optimized
by avoiding loading all rules into memory to determine available rules
for installation. Redundant fetching of all base versions has also been
removed, as they do not participate in the calculation.
---------
Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
## Summary
Updates form-data to ^4.0.2
Relates to https://github.com/elastic/kibana/pull/212183
form-data is used by data forge to load kibana assets. Ran data forge
with `--install-kibana-assets` to smoke test.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
In #211055 we enabled Elasticsearch Authc debug logs, but we use the
different entry function to start servers on CI. This PR moves the call
into `runElasticsearch` so that logs are enabled for every Scout script
(start-server or run-tests)
## Summary
Relates https://github.com/elastic/kibana/issues/206242
Implemented installing integrations from the doc in the follower index.
Can be tested locally by creating the ccr index locally or setting up 2
local clusters and set up ccr between them.
To test:
- Requires https://github.com/elastic/elasticsearch/pull/123493,
checkout and run es from source
```
yarn es source --license trial -E xpack.security.authc.api_key.enabled=true -E xpack.security.authc.token.enabled=true --source-path=/Users/juliabardi/elasticsearch -E path.data=/tmp/es-data -E xpack.ml.enabled=false
```
- Enable feature flag `xpack.fleet.enableExperimental:
['enableSyncIntegrationsOnRemote']`
- Create doc in ccr index, `hosts` should match local elasticsearch host
```
POST fleet-synced-integrations-ccr-remote1/_doc
{
"id": "fleet-synced-integrations",
"remote_es_hosts": [
{
"hosts": [
"http://192.168.64.1:9200"
],
"name": "remote1",
"sync_integrations": true
}
],
"integrations": [
{
"package_version": "1.25.0",
"package_name": "nginx",
"updated_at": "2025-02-24T09:03:51.936Z"
}
]
}
```
- Wait 1m until the task runs, verify that the integrations from the doc
are installed
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
To update the `global_telemetry_config` flag in Defend package policies,
we subscribe to the Telemetry plugin's `isOptedIn$` observable during
Kibana's `start()` phase, and receive the initial value immediately.
This feature is used for 'migrating' existing package policies: after
stack upgrade, when Kibana starts up, this subscription mechanism makes
sure that existing policies are backfilled with the new field.
But not on cloud and serverless instances.
It turned out, that while this works on local instances, on cloud and
serverless instances, at the very moment we receive the value during
`start()`, some mechanisms are not yet green, and this resulted in
`security_exception: missing authentication credentials for REST
request` when trying to read Saved Objects.
As subscribing to`core.status.core$`, and waiting until `ServiceStatus`
for `elasticsearch` and `savedObjects` is `available` didn't solve the
issue, I simply added a retry mechanism, which, at least, protects
against other temporary issues as well.
Some additional logging is added as well.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Remove extraneous dependencies:
* `canvas` was depending on 'webpack' purely for a type (dev-time).
* `@kbn/optimizer-webpack-helpers` (canvas depends on it 🤨) was
depending on 'webpack' solely for a function that could be defined in
`@kbn/optimizer` (devOnly).
## Summary
This PR upgrades React packages to version 18, while keeping Kibana
running in Legacy mode (`ReactDOM.render`). This is the first phase of
the React@18 upgrade; the second phase will gradually migrate Kibana to
Concurrent mode (`createRoot`) (exact plan is tbd).
Upgrade is intended to be non-breaking and behave just like React@17,
but it still requires thorough testing from all teams that own UI to
discover any potential critical UI issues. The testing was done in
https://github.com/elastic/kibana/issues/203114.
Most of the breaking work was completed in previous PRs and this PR only
includes minor breaking jest tests tweaks / snapshots updates
### Backports:
- This won't be backported to 9.0
- We will discuss if we should backport this to 8.x
### Risks
Kibana’s UI functional tests coverage and significant manual testing
that was done by a lot of teams in
https://github.com/elastic/kibana/issues/203114 gives us a lot of
confidence.
However, since this was a large internal change for React they still
could be issues hidden in "remote" parts of UIs, think of blank screens,
error splash screens, unresponsive pages, new errors in the console,
unpredictable UI behavior (like laggy text inputs that skip letters when
typing fast).
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Konrad Szwarc <konrad.szwarc@elastic.co>
Part of https://github.com/elastic/kibana/issues/200725
To track all relevant data:
> * User-facing deployment params, i.e. optimized for, VCPU level usage,
etc.
> * Resulting API params, e.g. number of allocations, etc.
This PR introduces
[EBT](https://docs.elastic.dev/telemetry/collection/event-based-telemetry)
event tracking, as ML is not currently using EBT events for data
collection, it includes the creation of `Telemetry Service`, accessible
via the `useMlKibana` hook.
I have already triggered some events for staging, which are visible
[here](https://telemetry-v2-staging.elastic.dev/s/ml/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:now-4M,to:now))&_a=(columns:!(timestamp),dataSource:(dataViewId:'74c6b7f0-3037-423f-a531-c736c70813b0',type:dataView),filters:!(),interval:auto,query:(language:kuery,query:'event_type:%20%22Trained%20Models%20Deployment%20Created%22%20'),sort:!(!(timestamp,desc)),viewMode:documents)).
Additionaly, I've created a test
[dashboard](b61d06af-4b3d-4550-8e5e-da6464652371/page/p_rssl6vh8ld/edit)
in Looker Studio to visualize the collected data.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The esql types are around many packages and plugins creating problems
either with the bundle size or cyclic dependencies.
This PR is moving some of them on the esql-types package. I am going to
move more there in follow up PRs.
(I decided to rename it as it makes more sense to have a generic one for
esql rather one only for the variables)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Adds the new docs CI checks. This will enable a preview to build our new
Markdown-based docs.
@florent-leborgne please tag reviewers, add the appropriate label(s),
and take this out of draft when you're ready.
---------
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Update search control example with buttons to programmatically interact
with controls
<img width="800" alt="Screenshot 2025-02-27 at 8 41 05 AM"
src="https://github.com/user-attachments/assets/e936cdeb-ce51-4fca-a8bc-ec5d983e3155"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR includes:
- Delete unused code
- Remove duplicate type definition and use the types defined in
`elastic` node module
- Refactor the code as needed for the type definition.
Tested in Serverless QA environment:
https://github.com/user-attachments/assets/43777b09-70c6-48ec-8ba2-988a3ab7e029
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Show error to the user when trying to setup Knowledge base on undersized
cluster
<img width="1847" alt="Zrzut ekranu 2025-02-26 o 19 03 43"
src="https://github.com/user-attachments/assets/a42d8560-aebb-410e-a364-7a27074f62fc"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
Co-authored-by: Garrett Spong <garrett.spong@elastic.co>
## Summary
Adds additional validations to Artifact APIs _(via `lists` plugin
server-side extension points)_ for the following conditions:
- If user has the global artifact management privilege, then they are
able to update/delete the artifact with no restriction (same as today)
- If user does NOT have the new global artifact management privilege,
then the update/delete action should fail:
- If it's a global artifact
- If it's a per policy artifact but it was created from a different
space than the active space the API is being called from
> [!NOTE]
> Functionality is currently behind the following feature flag:
`endpointManagementSpaceAwarenessEnabled`
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Closes https://github.com/elastic/security-team/issues/11683.
Adds an advanced setting to enable/disable the Asset Inventory page.
Replaces the old `assetInventoryUXEnabled` feature flag. The placement
of the setting is right below "Enable graph visualization", within the
Security Solution group.
### Screenshots
<details><summary>Setting off</summary>
<img width="943" alt="Screenshot 2025-02-21 at 09 38 43"
src="https://github.com/user-attachments/assets/c3b561cd-7dfa-4218-9004-cc89c5768551"
/>
</details>
<details><summary>Setting on</summary>
<img width="735" alt="Screenshot 2025-02-21 at 09 38 55"
src="https://github.com/user-attachments/assets/7a9ebf17-9339-49f2-820e-e26087f1c17c"
/>
</details>
<details><summary>Overriden setting - activated via
kibana.dev.yml</summary>
<img width="943" alt="Screenshot 2025-02-21 at 09 38 14"
src="https://github.com/user-attachments/assets/6ebb1e73-cffb-4bfd-ab21-631955574ce1"
/>
</details>
### How to test
Follow the instructions provided in the *README.md* file committed in
this PR.
### Definition of Done
- **Advanced Settings Integration**
- [x] Add a new setting under **Kibana Advanced Settings** for enabling
the Asset Inventory feature:
- **Setting Name**: `Enable Asset Inventory`
- **Setting Key**: `securitySolution:enableAssetInventory`
- **Description**: "Enable the Asset Inventory feature to view and
manage assets in the Security Solution plugin."
- **Type**: Toggle (On/Off).
- **Default Value**: Off.
- [x] Ensure the setting reflects the current status of the Asset
Inventory feature (On/Off).
- [x] Group the setting logically under the **Security Solution** in the
Kibana Advanced Settings page.
- [x] Ensure the toggle is discoverable and adheres to Kibana’s design
guidelines.
- **Implementation**
- [ ] ~~Update the `Asset Inventory` initialization logic to check the
new Kibana setting (`securitySolution:enableAssetInventory`) instead of
relying on the `assetInventoryUXEnabled` feature flag in
`kibana.dev.yml`.~~ For now we don't need to worry about initialization
- [ ] ~~Provide backward compatibility by allowing the `kibana.dev.yml`
flag (`xpack.securitySolution.assetInventoryUXEnabled`) to override the
setting in development environments.~~
- [x] The toggle should dynamically enable or disable the Asset
Inventory feature without requiring a Kibana restart.
- **Testing**
- [x] Add unit tests to verify:
- The toggle updates the setting value correctly.
- The Asset Inventory feature respects the toggle status
(enabled/disabled).
- [x] Add functional tests to validate the toggle’s behavior in the
Advanced Settings page.
- **Documentation**
- [x] Update the documentation to explain how to enable/disable the
Asset Inventory feature using Kibana Advanced Settings.
- [ ] ~~Provide details about the fallback behavior when using the
`kibana.dev.yml` flag.~~
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Feature will get exposed to end users if combination of setting and
feature flag is not set up correctly.
## For reviewers:
Only `*.schema.yml` files were edited (excluding
`*.bundled.schema.yml`). Rest of the changes comes from auto generation
and can be ignored.
## Description
Part of DW team effort - elastic/security-team#11804
This PR aligns the property/schema descriptions and examples in
AsciiDocs with OpenAPI schemas. The primary goal of this PR was not to
extend or enhance the documentation but to migrate from one system to
another.
Ascii docs -
https://www.elastic.co/guide/en/security/8.17/management-api-overview.html
OpenApi generated docs -
https://www.elastic.co/docs/api/doc/kibana/operation/operation-endpointgetactionslist
Changes:
Copied missing property descriptions from AsciiDoc to OpenApi properties
Copied existing AsciiDoc examples for both requests and responses
Fixed falsy query object in some GET requests - in OpenApi it was
defined as an object, not as path query params.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com>
Fixes#209695
## Summary
Adds a method from content management for exposing a server-side
Dashboard CRUD client.
Consumers who want to search, retrieve, or modify Dashboards from a
server plugin find themselves using the Saved Object client. This means
they need to handle JSON parse/stringify and reference handling
themselves. We could expose a CRUD functionality from content management
on the Dashboard server plugin contract to avoid re-creating all of this
boilerplate handling.
Commit c53f47d72a shows a crude
demonstration of how a plugin can use the methods available on the
Dashboard server plugin with a request to retrieve a list of dashboards.
You can test this in the Dev Tools:
```
GET kbn:/api/search_dashboards?spaces=*
```
This will use the Search method from content management to return a list
of dashboards across all spaces.
To allow the Search method to return all fields in the Dashboard, I
needed to remove the default fields. I updated all current uses of the
search method to specify the necessary fields. See
618e025210.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
Fixes https://github.com/elastic/kibana/issues/211911
The CSV processing is now a three-stage process:
1. Parse the samples with the temporary column names of the form
`column1`.
2. Test parsing with the actual pipeline that parses into
`package.dataStream.columnName`.
3. Convert the samples into JSON form `{"columnName": "value", ...}` for
further processing.
Now the pipeline works as expected:
```yaml
- csv:
tag: parse_csv
field: message
target_fields:
- ai_202502211453.logs._timestamp
- ai_202502211453.logs.message
description: Parse CSV input
- drop:
ignore_failure: true
if: >-
ctx.ai_202502211453?.logs?._timestamp == '@timestamp' &&
ctx.ai_202502211453?.logs?.message == 'message'
tag: remove_csv_header
description: Remove the CSV header line by comparing the values
```
There are unit tests tests for the CSV functionality that include a mock
CSV processing pipeline.
## Summary
This PR fixes a couple of things with regards to the parsing of the
OpenAPI spec for use in CEL generation:
1) fixes and greatly simplifies the parsing of the OpenAPI spec so that
we collect all the $ref tags in the response object
2) only collects the top level schemas from the response object (since
that's all we really need for the CEL program)
3) fixes it so that users cannot select 'Save configuration' if there is
a generation error
4) better error messaging if/when a spec parsing error occurs
Note re fix # 3, the 'Save configuration' button will still initially be
available upon an error occurring. Then when if the user tries to click
save after an error, it will then disable the save button and show the
message indicating they need a successful generation to save. This is
consistent with the UX philosophy in the rest of the flyout that all
buttons are enabled by default, and if the user does something 'wrong',
we then provide guidance for how to proceed.
Relates: https://github.com/elastic/kibana/issues/210271
## Screenshots
<details>
<summary>parsing fix</summary>
<img width="450" alt="Screenshot 2025-02-21 at 2 15 34 PM"
src="https://github.com/user-attachments/assets/80fe8e56-ffe3-4d5c-b6ac-5a57e025b70b"
/>
</details>
<details>
<summary>save disabled fix</summary>
<img width="450" alt="Screenshot 2025-02-21 at 2 13 45 PM"
src="https://github.com/user-attachments/assets/5220bad7-70b1-4ade-83f7-ce1f97d115d1"
/>
<img width="450" alt="Screenshot 2025-02-21 at 2 13 55 PM"
src="https://github.com/user-attachments/assets/427bb52c-6fa9-457f-ab28-f490be981094"
/>
</details>
