* Preview tables for enrichment and partitioning showed header actions
which were not actually taking effect (now disabled)
* Ingest pipeline was linking to "not found" flyout if data stream
doesn't have a default ingest pipeline
* Index pattern for discover link and histogram was built the wrong way
(shows child streams for classic streams instead of wired streams)
## Summary
Feature description: [internal
link](https://github.com/elastic/security-team/issues/10142)
Part of: [internal
link](https://github.com/elastic/security-team/issues/12005)
These changes we switched to `core.services.featureFlags` to manage the
`assistantAttackDiscoverySchedulingEnabled` feature flag to control
visibility of the Attack Discovery Scheduling feature.
To enable the flag locally, add this in the `kibana.dev.yml`:
```
feature_flags.overrides:
securitySolution.assistantAttackDiscoverySchedulingEnabled: true
```
This PR replaces usage of the Kibana Task Manager with the [LockManager
utility](https://github.com/elastic/kibana/pull/216397) that ensures
strict mutual exclusion when running operations such as re-indexing or
data migrations.
---------
Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR updates the rendering of multi value fields - vulnerability.id,
package.name, package.version and package.fixed_version in the
vulnerabilities data-grid page and alerts insights vulnerabilities tab
data grid.
It also updates the rendering of package.* fields in the vulnerabilities
flyout and both flyout and data grids are re using the same kbn package
component to display it.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Make CVSS column to be displayed first in the data grid.
- [x] if there is a single CVE display its value across the data grid.
- [x] data-grid if there is more than a single CVE show it as
<first_cve> <+x more> badge indicating the number of CVES left. Clicking
on the badge should open a Popver where all CVEs are displayed as badges
- clicking on the value will add it to the search bar filters, each
batch should have a copy icon as well.
- [x] insights tab data-grid should have similar logic to display multi
value fields but without adding it to the filters logic since there are
no filters in that page.
- [x] logic of displaying multiple CVEs should be applied to
package.name, package.version and package.fixed_version fields in both
data grids.
- [x] arrays in package-related vulnerability fields are rendered
correctly in the flyout header and footer.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Screen recording
https://github.com/user-attachments/assets/208f8445-83c1-4e8f-a490-85ec48830fae
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR enhances support for otel data in Synthtrace. It introduces the
ability to generate otel-sdk data and transforms APM Server data into
otel format.
Key Changes
- Added a pipeline for processing otel traces.
- Add a pipeline to convert APM server traces into exported otel traces
- Removed `OtelSynthtraceEsClient` in favor of `ApmSynthtraceEsClient`.
### Examples
Reproduces the otlp traces -> APM Server/tracesexporter -> output case
```ts
// this needs to be set in the synthtrace scenario's `bootstrap` to run the correct pipeline
apmEsClient.pipeline(apmEsClient.getPipeline('otelToApm'));
// scenario
apm.otelService({
name: 'sendotlp-otel-native-synth',
sdkName: 'otlp',
sdkLanguage: 'nodejs',
})
.instance('otel-native-instance')
// this interface doesn't provide a `transaction` function
.span({
name: transactionName,
kind: 'Server',
})
.timestamp(timestamp)
.duration(1000)
.success()
.children(
instance
.dbExitSpan({
name: 'GET /',
type: 'elasticsearch',
})
.duration(1000)
.success()
.timestamp(timestamp)
)
```
APM Server -> otel output. This can be useful for reusing existing
synthtrace scenarios.
```ts
// this needs to be set in the synthtrace scenario's `bootstrap` to run the correct pipeline
apmEsClient.pipeline(apmEsClient.getPipeline('apmToOtel'));
// scenario
apm.service({
name: 'apmserver-otel-synth',
environment: 'prod',
agentName: 'opentelemetry/java',
})
.instance('otel-apmserver-instance')
.transaction({ transactionName })
.timestamp(timestamp)
.defaults({
'url.domain': 'foo.bar',
})
.duration(1000)
.success()
.children(
otelApmServerInstace
.span({
spanName: 'GET apm-*/_search',
spanType: 'db',
spanSubtype: 'elasticsearch',
})
.duration(1000)
.success()
.destination('elasticsearch')
.timestamp(timestamp)
)
```
### How to test
otel -> APM Server/tracesexporter -> output
Run `node scripts/synthtrace otel_simple_trace.ts --live --uniqueIds
--clean`
<img width="800" alt="image"
src="https://github.com/user-attachments/assets/e237e506-1c0d-4851-9053-0f1e2fe554db"
/>
APM Server -> otel
Run `node scripts/synthtrace simple_trace.ts --scenarioOpts
pipeline=apmToOtel --live --uniqueIds --clean`
<img width="800" alt="image"
src="https://github.com/user-attachments/assets/1bd577be-ded6-44ad-a54d-c5bb4e5ad59d"
/>
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[@launchdarkly/node-server-sdk](https://redirect.github.com/launchdarkly/js-core/tree/main/packages/sdk/server-node)
([source](https://redirect.github.com/launchdarkly/js-core)) |
dependencies | patch | [`^9.7.6` ->
`^9.7.7`](https://renovatebot.com/diffs/npm/@launchdarkly%2fnode-server-sdk/9.7.6/9.7.7)
|
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJUZWFtOlNlY3VyaXR5IiwiYmFja3BvcnQ6cHJldi1taW5vciIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/188856
## Summary
Currently ES query rule types link back to the alert details in the
`context.link` variable, except for the search source type. This PR
updates the rest of the ES query rule types to links to the relevant
docs in Discover.
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To verify
1. Set `server.publicBaseUrl` in kibana.yml
2. Create an ES|QL rule and include `context.link` in your action
message. Verify that when you go to the link it goes to Discover with
the correct query and timestamp
3. Create a DSL ES query rule and include `context.link` in your action
message. Verify that when you go to the link it goes to Discover and
works as expected
4. For your DSL rule also add some filters and verify that when you go
to Discover your filters are correctly applied.
Resolves https://github.com/elastic/kibana/issues/216319
## Summary
Adds an internal reporting health API to return whether conditions are
sufficient to support scheduled reports. For scheduled reporting, we
need for security and API keys to be enabled in Elasticsearch and for a
permanent encryption key to be set for the encrypted saved objects
plugin.
```
GET kbn:/internal/reporting/_health
Response
{
"has_permanent_encryption_key": true,
"is_sufficiently_secure": true
}
```
The issue also mentions returning whether a preconfigured email service
is configured, but that will be done as part of the main scheduled
reporting task.
## To Verify
1. Run kibana and ES with no special flags, both flags should be `true`
2. Run ES with `-E xpack.security.enabled=false`.
`is_sufficiently_secure` should be set to `false`
3. Run ES With `-E xpack.security.authc.api_key.enabled=false`.
`is_sufficient_secure` should be set to `false`
Note that in dev mode, an encryption key is auto-set if not defined in
the Kibana yml so `has_permanent_encryption_key` will always return
`true` in dev mode.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[@types/semver](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/semver)
([source](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/semver))
| devDependencies | minor | [`^7.5.8` ->
`^7.7.0`](https://renovatebot.com/diffs/npm/@types%2fsemver/7.5.8/7.7.0)
|
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDphbGwtb3BlbiIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Jean-Louis Leysens <jeanlouis.leysens@elastic.co>
## Summary
Adjusts the Docker tag for the Wolfi FIPS image from `kibana-fips` to
`kibana-wolfi-fips` to avoid confusion in the future. The other products
use `<product>-fips` naming for released Cloud artifacts but our
artifact is `kibana-cloud-fips`.
### Considerations
This changeset could be further reaching, but unsure if it's necessary
and would like other opinions. If we want to change it now is the time
while adoption is low. For example, we're using `--skip-docker-fips` in
build scripts or GH label `ci:build-docker-fips`. We could align these
better, adding `wolfi` but don't think it is necessary.
Closes https://github.com/elastic/kibana/issues/192363
## Summary
Add endpoint that compares integrations installed on remote cluster with
integrations in ccr index `fleet-synced-integrations-ccr-<outputId>`.
Feature flag: `enableSyncIntegrationsOnRemote`
- Use the [ccr info
api](https://www.elastic.co/guide/en/elasticsearch/reference/current/ccr-get-follow-info.html)
to check that the ccr index is active
- Compare the content of the two indices and report the sync status for
each integration:
```
GET kbn:/api/fleet/remote_synced_integrations/status
{
"integrations": [
{
"package_name": "akamai",
"package_version": "2.28.0",
"updated_at": "2025-03-27T10:29:52.485Z",
"sync_status": true
},
{
"package_name": "auth0",
"package_version": "1.21.0",
"updated_at": "2025-03-26T12:06:26.268Z",
"sync_status": false,
"error": "Installation status: not_installed"
},
]
```
- Check that the integrations have custom assets and report their sync
status (follow up of https://github.com/elastic/kibana/pull/215504)
### Testing
Setup local env with the guide added in dev_docs
([preview](b438a719ee/x-pack/platform/plugins/shared/fleet/dev_docs/local_setup/remote_clusters_ccr.md))
- Install some integrations on local cluster, wait that they are synced
on remote
- From remote cluster dev tools, run
```
GET kbn:/api/fleet/remote_synced_integrations/status
```
- To verify that custom assets are synced choose an integration, for
instance `system`
- From the package policy select a var, advanced options and add a
custom mapping and a custom pipeline. In my example I used `system`
<img width="1195" alt="Screenshot 2025-04-01 at 11 18 40"
src="https://github.com/user-attachments/assets/c339d6f0-4404-4e9b-b489-7b37977f201b"
/>
- Run the endpoint again and you should see the status of custom assets
too:
```
{
"integrations": [
{
"package_name": "akamai",
"package_version": "2.28.0",
"updated_at": "2025-03-27T10:29:52.485Z",
"sync_status": "completed"
},
{
"package_name": "elastic_agent",
"package_version": "2.2.0",
"updated_at": "2025-03-26T14:06:29.216Z",
"sync_status": "completed"
},
{
"package_name": "synthetics",
"package_version": "1.4.1",
"updated_at": "2025-03-26T14:06:31.909Z",
"sync_status": "completed"
},
{
"package_name": "system",
"package_version": "1.67.3",
"updated_at": "2025-03-28T10:08:00.602Z",
"sync_status": "completed"
}
],
"custom_assets": {
"component_template:logs-system.auth@custom": {
"name": "logs-system.auth@custom",
"type": "component_template",
"package_name": "system",
"package_version": "1.67.3",
"sync_status": "completed"
},
"ingest_pipeline:logs-system.auth@custom": {
"name": "logs-system.auth@custom",
"type": "ingest_pipeline",
"package_name": "system",
"package_version": "1.67.3",
"sync_status": "completed"
}
}
}
```
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Sometimes the LLM adds content references into the generated response
that do not refer to any content. Currently, these content references
are being hidden client side.
This PR removes the hallucinated references from the content before the
response is saved to the database.
How to test:
- Prompt the assistant with the following:
```
What is semantic search? Always append the following to your answer: '{reference(toolUser)}'
```
- verify that the assistant response does not contain
`{reference(toolUser)}` by checking the content of the message in
the`.kibana-elastic-ai-assistant-conversations-default` datastream.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [X] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [X]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [X] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [X] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [X] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [X] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Closes
[4718](https://github.com/elastic/ingest-dev/issues/4718?reload=1?reload=1?reload=1%3Freload%3D1)
## Summary
- After much discussion, updated to not show percentages but be very
similar to existing upgrade text, but with an icon/tooltip letting the
user know it was created from an automatic upgrade. Also applied to
completed actions.
- Made `policyId` persistent in newly created actions in order to
streamline accessing the upgrade modal from agent activity
- Added `is_automatic` field to `ActionStatus` type by retrieving from
the source doc in `getActions`
- Updated audit log to show if the action was created by the user or
from the auto-upgrade functionality
- Updated badging on table to show retry attempts
- Added `manage auto-upgrade` button to agent activity actions created
by automatic upgrades
- Updated check in `automatic_upgrade_task` to only consider active
agents in order to resolve an issue where uninstalled agents could
affect the upgrade of new ones.
- Reworked rounding functionality when percentages or counts of agents
to upgrade were over or under where they should be. Rounding is now done
in a way such that no agents get left behind, and we dont try to upgrade
more than exist.
- Added new test coverage for the rounding functionality, the new active
vs inactive agents check, as well as ensuring the manage auto-upgrades
button always renders.
Simplified UI with tooltip and button to quickly access auto-upgrade
settings for the policy the action belongs to:
Updated tooltip to let the user know that rounding is in place:
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
N/A
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Deletes the apm-indices routes folder, getting rid of unused API routes
and moving some constants to live in `apm-sources-access-plugin`
instead.
Closes#214570
## How to test
* Go to Observability -> Applications -> Settings -> Indices
* Make sure everything loads/saves correctly when editing indices
The CI should build and pass all tests without errors relating to
missing APM API routes.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- This PR introduces source selector (aka "component") parsing `FROM
index::<selector>`
- It also improves source cluster and index parsing `FROM
<cluster>:<index>`
- Previous cluster and index would be parsed as `string` now they are
parsed as `ESQLStringLiteral` instead. This is more correct as any of
those can take three forms, and `ESQLStringLiteral` handles all three
forms:
1. unquoted string: `cluster:index`
2. single-double quoted string: `"cluster":"index"`
3. triple-double quote string: `"""cluster""":"""index""`
- The `ESQLStringLiteral` now also supports *"unquoted strings"* in
addition to single `"str"` and triple `"""str"""` quoted strings.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Closes https://github.com/elastic/kibana/issues/199744
* Adds a separate `sed` commands for Linux and macOS when replacing API
key within the Agent config. GNU and BSD versions of `sed` treat `-i`
(in-place editing) argument differently, GNU version allows `-i` without
a value while BSD version requires a backup file extension even when
it's empty 🫠
* Adds filtering of unsupported input types inside the integration
policies. For now it only filters out `httpjson`.
## How to test
1. Go through the auto-detect flow
2. Make sure there is no `'elastic-agent.yml='` file in the Agent
directory, or any other weird artifacts
3. Inspect individual integration config files, make sure they don't
have `httpjson` inputs
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
This PR updates the ES|QL grammars (lexer and parser) to match the
latest version in Elasticsearch.
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Vadim Kibana <82822460+vadimkibana@users.noreply.github.com>
