## Summary
Closes https://github.com/elastic/kibana/issues/214446
This PR introduces a new section in the span/transaction overview flyout
that will display the duration information along with a latency chart.
For now, only the duration data is included.
|Scenario||
|-|-|
|Span w/ `transaction.id`||
|Span w/o `transaction.id`||
|Transaction w/ `parent.id`||
|Transaction w/o `parent.id` (root transaction)||
Given that both spans and transactions require retrieving data from
their respective parents, a loader has been added to reflect the ongoing
data-fetching process:
## Tech details
The UI for displaying the duration and its percentage relative to the
parent is already in use in APM for spans and transactions, so the logic
is consistent with that.
To avoid duplicating components and formatters, as seen in previous PRs
for Traces in Discover, a new `Duration` component has been created in a
newly created `kbn-apm-ui-shared` package. This component will be used
in Discover and [later](https://github.com/elastic/kibana/issues/211781)
in APM as well.
## How to test
- Enable the discover profiles by adding this to the` kibana.yml `file:
```discover.experimental.enabledProfiles:
- observability-traces-data-source-profile
- observability-traces-transaction-document-profile
- observability-traces-span-document-profile
```
- Open Discover and select or create a data view that includes any APM
traces index (`traces-*`), or query them using ES|QL.
- Apply a filter for `data_stream.type:"traces"` to ensure only trace
documents are retrieved.
- Open the flyout.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Carlos Crespo <crespocarlos@users.noreply.github.com>
## Summary
this PR aims to reduce flakiness in metering.cy.ts test.
- fix on constant fail: a small change was needed for metering tests: to
not expect https connection by the dummy telemetry server
- improvement against flakiness: the test tends to fail if metering task
is performed while mock heartbeats are being indexed, therefore the task
interval has been increased from 1 minute to 3 minutes. on CI it takes
less than 2 minutes to start up Kibana and index the heartbeats, so with
3 minutes we should be okay for a while.
flaky runner: 100%
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8114
- all jobs were done in 8-9 minutes, which shows that they succeeded the
first time, without the internal retry mechanism.
> [!important]
> ~45c0a33d88 needs to be reverted before
merging~
closes: https://github.com/elastic/kibana/issues/187083
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
## Summary
Collection type was set with `agent.type` which conflicts on stack and
metricbeat monitorings (same `metricbeat` value). For the stack
monitoring, elastic-agent internally utilizes filebat or metricbeat.
This change hard codes the `collection_types` corresponding on each
monitoring type to categorize monitoring type correctly.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- ~~[ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)~~
- ~~[ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials~~
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- ~~[ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)~~
- ~~[ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.~~
- ~~[ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed~~
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
- [ ]
### Example data
- when stack monitoring
```
{
"clusterUuid": "WKhHGcv8Rt6nHwKRmKDdAw",
"stats": {
"collectionSource": "monitoring",
"cacheDetails": {
"updatedAt": "2025-03-31T23:07:32.154Z",
"fetchedAt": "2025-03-31T23:07:33.349Z"
},
"cluster_name": "logstash",
"timestamp": "2025-03-31T23:07:32Z",
"version": "9.1.0",
"cluster_uuid": "WKhHGcv8Rt6nHwKRmKDdAw",
"stack_stats": {
"logstash": {
"versions": [
{
"version": "9.1.0",
"count": 1
}
],
"count": 1,
"cluster_stats": {
"pipelines": {
"count": 1,
"batch_size_total": 125,
"batch_size_max": 125,
"batch_size_min": 125,
"batch_size_avg": 125,
"workers_total": 10,
"workers_max": 10,
"workers_min": 10,
"workers_avg": 10,
"sources": {
"file": true
}
},
"plugins": [
{
"name": "logstash-input-heartbeat",
"count": 1
},
{
"name": "logstash-filter-ruby",
"count": 1
},
{
"name": "logstash-filter-mutate",
"count": 1
},
{
"name": "logstash-output-stdout",
"count": 1
}
],
"monitoringClusterUuid": "WKhHGcv8Rt6nHwKRmKDdAw",
"collection_types": {
"agent": 1
},
"queues": {
"memory": 1
}
}
}
},
"cluster_stats": {}
}
}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves: https://github.com/elastic/kibana/issues/190661
This PR allows tasks to be ran scoped as the current user. It
accomplishes this by creating an API when the user schedules this task.
It will then persist the API key in the task instance which allows the
task handler to have access to the user's privileges.
To test:
1. Run `yarn start --run-examples`
2. Navigate to
`http://localhost:5601/app/triggersActionsUiExample/task_manager_with_api_key`
3. Click on `Schedule Task 1 and 2`, this will schedule a mock test that
is defined in
`x-pack/platform/plugins/shared/alerting/server/plugin.ts`, which prints
out the task instance and the scoped clients.
4. Click on `Remove` to cancel the task, which then cleans up and
invalidates the API keys.
5. Reschedule the 2 tasks and click on `Remove All Tasks` which will
bulk remove tasks and invalidate API keys.
Limitation of this approach:
- Because we depend on a request, this would mean every schedule that is
tied to a user needs to be triggered using a request. (Unless we have a
way of generating an API key without using the request).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/216886
8.17 and 8.16 resolve in branch specific patch
https://github.com/elastic/kibana/pull/216910
### Problem
1) Open previously saved empty dashboard. `panels$.value` is `{}`
2) add new markdown panel. This takes you to visualize editor. On return
to dashboard, embeddable state transfer service contains the markdown
embeddable runtime state. `panels$.value` is `{ markdownPanelId: {
explicitInput: {}}}`. Markdown panel is rendered with empty
serializedState and runtimeState from embeddable transfer service.
3) Click save dashboard. This is where the bug lies. `panels$.value` is
not updated to `{ markdownPanelId: { explicitInput: { // serializedState
}}}` but instead, remains unchanged.
4) Click duplicate panel. New panel is added to `panels$.value`. This
triggers unsaved changes to be stored in session storage. Unsaved
changes store `panels$.value` so `{ markdownPanelId: { explicitInput: {
// empty!!! }}, duplicatedMarkdownPanelId: { explicitInput: {
serializedState }}}` is put into session storage
5) Edit duplicated markdown panel. This takes you to visualize editor.
On return to dashboard, embeddable state transfer service contains the
new markdown embeddable runtime state for the duplicated panel. This is
where things start to fall apart. The dashboard state is loaded from
saved object state and `panels` contains the serialized state for both
`markdownPanelId` and `duplicatedMarkdownPanelId`. Then unsaved changes
state is loaded from dashboard session and replaces dashboard saved
state. This causes the first markdown panel to render with empty
serializedState.
### Solution
The problem is resolved by updating `panels$` on save. This causes step
4 to store the current serializedState in unsaved changes.
This updates the Knowledge base test utils and adds test for index asset
creation.
### `setupKnowledgeBase`
#### Before
```
await importTinyElserModel(ml);
await setupKnowledgeBase(observabilityAIAssistantAPIClient);
await waitForKnowledgeBaseReady({ observabilityAIAssistantAPIClient, log, retry });
```
#### After
```
await setupKnowledgeBase(getService);
```
### `deleteKnowledgeBaseModel`
#### Before
```
await deleteKnowledgeBaseModel(ml);
await deleteInferenceEndpoint({ es });
```
#### Before
```
await deleteKnowledgeBaseModel(getService);
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
## Summary
Main ticket ([Internal
link](https://github.com/elastic/security-team/issues/12004))
We are working on a new AI Assistant feature which will allow users to
schedule Attack Discovery generation. It will also allow users to setup
actions. The plan is to use alerting framework for that and create a new
rule type with executor handler. One thing which is missing right now is
the `actionsClient` (to get access and use AI connectors) within the
executor context.
With these changes, we expose `actionsClient` within the
`RuleExecutorOptions` for Attack Discovery Schedule rule type.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/streams-program/issues/38.
- Adds configuration driven support for processors
- Adds support for the following processors:
- Geoip
- KV
- Rename
- Set
- URL Decode
- User agent
## Notes
- `if` is now conditional as not all processors support a condition
- With `set` you can either have a `value` OR a `copy_from`. To avoid
types, form, and validation complexity for now I've just allowed the use
of `value`.
- I didn't add `extract_device` to the user agent configuration as it's
Beta.
## Future improvements
- We could add configuration for the grok, dissect, and date processors
too. This would align things so that each processor has things "in one
place". We could extend the field types available to provide a `custom`
type, this would allow those processors to return a React component for
their specialised fields / UI (still driven by `react-hook-form` under
the hood). Overall the code footprint would decrease.
- A couple of types can possibly be refined but I wanted to try and get
this open before the Milestone 2 deadline.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Closes https://github.com/elastic/kibana/issues/214805
## Summary
By default ES|QL records don't have `_id` unless it's requested via the
query `METADATA`.
This PR fixes the JSON view inside DocViewer for ES|QL records.
Previously it was relying on `textBasedHits` which gets updated when
query changes hence there is a possibility of loosing the reference to
the last viewed record.
## Testing
Example queries:
```
FROM kibana_sample_data_ecommerce METADATA _index
FROM kibana_sample_data_ecommerce METADATA _index, _id
```
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Resolves: https://github.com/elastic/kibana/issues/145266
* Allows SSL configuration to be disabled for Fleet agent logstash
output
* Adds an SSL toggle in the logstash output form.
* On is the default state of the form
* When off:
* Authentication form section is removed
* Logstash input config has SSL related fields removed
* Submitting update removes SSL fields and related SSL secrets in output
config
* Shows a call out to proceed with caution
## Summary
This PR replaces the Sourcerer component with the data view picker from
discover app, if the feature flag is enabled.
<img width="600" alt="Screenshot 2025-04-02 at 09 46 21"
src="https://github.com/user-attachments/assets/7ec35bf5-12dc-4e72-9292-4a734034806f"
/>
**Until we add relevant hooks on the target page that has the global
header, this will only be a cometic change.**
Don't expect data view changes made in the picker to be applied just
yet.
### Testing
Add the following feature flag to your configuration:
```xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled']```
and navigate to a page with global header, eg. alerts.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
### Identify risks
We should still show the original sourcerer if the flag is not enabled
## Summary
The error callouts were getting squished to the side, I have moved them
to be in their respective engine section.
**Before**
<img width="1728" alt="Screenshot 2025-03-27 at 11 05 38"
src="https://github.com/user-attachments/assets/c244acd1-a75e-4b43-a509-8bff9966b31e"
/>
**After**
<img width="1728" alt="Screenshot 2025-03-27 at 19 39 53"
src="https://github.com/user-attachments/assets/2b0dd9ad-568a-4e9c-aad7-6922b2085d69"
/>
Bonus refactor:
I have added type definitions for the error object keys on the engine
status object and moved the error callout to a common component as it's
now used in 3 places.
## Test Steps
the easies way to test this is to try and init the entity store in a
space that is lobger than 45 chars long.
- create a space with a very long name e.g
'this_is_a_very_long_space_name_indeed_hello_world_123456789'
- enable the entity store from the entity analytics dashboard (it should
error)
- navigate to the entity store management page to see the error
callouts.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR fixes an issue introduced by [this
PR](https://github.com/elastic/kibana/pull/184635) back in `8.16`. I
don't think anyone noticed the problem until now...
In the PR linked above, the name of the property responsible to render
custom components in the group stats section of the alerts table
grouping was changed from `renderer` to `component` but there was (at
least) one usage that had not been updated. Because that usage wasn't
correctly typed and there was no unit tests to verify the behavior, the
issue went unnoticed...
### Notes
This whole code should be refactored eventually. This is not the purpose
of this PR. This only focuses on fixing the issue, adding proper types
and unit tests.
| Before | After |
| ------------- | ------------- |
| 
| 
|
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
`lastLookbackDate` is a variable we used to use to set a field on the
SIEM rule status SOs, e.g.
[here](https://github.com/elastic/kibana/blob/main/x-pack/test/functional/es_archives/cases/migrations/7.11.1/mappings.json#L2068).
It contained the date from the last doc in the rule search results, if
there were any results. The last doc in the search results should be the
most recent one. Since we removed the SIEM rule status SOs, we have not
been storing this data anywhere so we don't need to compute it in the
rule executors anymore.
## Summary
It closes https://github.com/elastic/kibana/issues/210717
This PR adds the OnboardingSuccessCallout component to the All Assets
page, but the component is only visible to the user who initiated the
Onboarding and it no longer shows once dismissed by the user.
Also, this PR adds the `checkAndInitAssetCriticalityResources` to the
enablement as it's required by the transforms installed during the
Entity Store initialization.
## Recording
https://github.com/user-attachments/assets/31130195-c67c-4a55-aa37-555d527f38f0
closes https://github.com/elastic/kibana/issues/216032
## Summary
Create a fluent API for APM Otel. The interface is similar to the one
currently used to create scenarios for elastic APM data.
This will make it easier to create tests and more flexible synthetic
scenarios to cover Otel specificities.
### How to test
- run `node scripts/synthtrace otel_simple_trace.ts --live --uniqueIds
--clean`
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
- Closes https://github.com/elastic/kibana/issues/215893
## Summary
This PR extract the logic from Share > Export > Generate CSV into new
utils and uses it to fix CSV export for Dashboard panels.
<img width="903" alt="Screenshot 2025-04-01 at 12 05 58"
src="https://github.com/user-attachments/assets/20e611d7-b1da-4b50-a8fc-e18ac3db3a55"
/>
## Testing
Please test for both data view mode and ES|QL mode. The steps are the
following:
1. Save a discover session
2. Add it to a new dashboard
3. Save the Dashboard and switch to View mode
4. Via panel actions, press "Generate CSV report"
5. Compare the CSV results with what is shown in the grid
6. Add a custom time range to the panel via panel actions "Settings"
7. Generate a CSV report and compare results again
Also check that reporting still works when generated from Discover page.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
> [!CAUTION]
> Do not merge yet!
> This PR is mergeable based on the assumption that we do not allow
downgrading to `search_ai_lake` tier from `essentials`/`complete` or
`trial` `complete/essentals`.
## Summary
On serverless with following sets of product types/lines timelines and
notes are not available to the user.
```json5
[
{ product_line: 'ai_soc', product_tier: 'search_ai_lake' }
]
```
or
```json5
[
{ product_line: 'security', product_tier: 'search_ai_lake' }
]
```
or with
```json5
[
{ product_line: 'ai_soc', product_tier: 'search_ai_lake' },
{ product_line: 'security', product_tier: 'search_ai_lake' }
]
```
This applies to custom role creation flyout where timelines/notes
privileges are not available for RBAC.
### Screenshots (where timeline and notes are not available)
#### Custom role creation
Also on alerts page notes and timelines is not available action items.
Trying to access these via URL does not show them either.
#### Current alerts page (with timelines and notes not available)
#### Alert details via URL (without notes on the expanded flyout)
## How to Test
1. While on the Kibana root directory, run ES/Kibana on serverless mode
with:
```bash
yarn es serverless --kill --projectType security --kibanaUrl=http://0.0.0.0:5601
```
and on a new window
```bash
yarn serverless-security --no-base-path
```
Enable the AI for SOC tier, by adding the following to your
`serverless.security.dev.yaml` file:
```json5
xpack.securitySolutionServerless.productTypes:
[
{ product_line: 'ai_soc', product_tier: 'search_ai_lake' },
]
```
2. Once Kibana is up and running login in with the `admin` role using
the role dropdown.
3. Navigate to `app/management/roles/edit`
4. Click on `Assign to space` button and assign a space to that role on
the `Assign role to spaces` flyout.
5. Expand the `Security` privileges and verify that only 4
sub-privileges are seen and that Timelines and Notes are not among them.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
## Summary
Long ago, we did multiple searches on each "page" of results to search
for both docs with the timestamp override and default `@timestamp`. We
then merged the results together before attempting to bulk create
alerts. We no longer do this, instead we have a simpler process that
just does one query per page so there's no need to merge search results
together.
We also used to build the `tuple` inside `searchAfterBulkCreate`, so we
had logic to verify if the tuple was created correctly. The time range
tuple is now calculated in the shared security wrapper, which is
responsible for any error handling. The TS types tell us that `tuple`
and its subcomponents can't be null, we don't need to check in
`searchAfterBulkCreate`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
