## Summary
PR to add an experimental traces data source profile for the APM
data-view. This is the first part to unblock further Discover for Traces
work.
Closes#208690
Related #208689
### How to enable and test this new data source profile
Add the following to `kibana.dev.yml`:
```
discover.experimental.enabledProfiles:
- traces-data-source-profile
```
And select the APM data view:
## Summary
Adds filter to the entity definition schema so it can be used to filter
entity store data further.
### Testing Steps
1. Open Kibana and start the entity store.
2. Initialise an engine (e.g., user) with a filter:
```
POST kbn:/api/entity_store/engines/user/init
{
"filter": "@timestamp < now-6w"
}
```
3. Once the transform has run, verify in Dev Tools:
`GET _transform/entities-v1-latest-security_user_default`
You should see the applied filter in the results, alongside default
filters for user.name and timestamp from
[generate_latest_transform.ts](90d768be9b/x-pack/platform/plugins/shared/entity_manager/server/lib/entities/transform/generate_latest_transform.ts (L29))
Example output will include the added filter as part of a bool query.
```
"query": {
**"bool": {
"filter": [
{
"bool": {
"should": [
{
"range": {
"@timestamp": {
"lt": "now-6w"
}
}
}
],**
"minimum_should_match": 1
}
},
{
"exists": {
"field": "user.name"
}
},
{
"range": {
"@timestamp": {
"gte": "now-24h"
}
}
}
]
}
}
},
```
# Backport
This will backport the following commits from `8.x` to `main`:
- [[Upgrade Assistant] Data streams reindexing
(#208083)](https://github.com/elastic/kibana/pull/208083)
<!--- Backport version: 9.6.4 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)
<!--BACKPORT [{"author":{"name":"Ahmad
Bamieh","email":"ahmad.bamyeh@elastic.co"},"sourceCommit":{"committedDate":"2025-01-30T22:33:10Z","message":"[Upgrade
Assistant] Data streams reindexing (#208083)\n\n## Data streams
reindexing\r\n\r\nThis PR implements data stream reindexing corrective
action.\r\n\r\n\r\n## Screenshots\r\nFound
here\r\nhttps://docs.google.com/document/d/1QntGX5aTbjAv-VRZoKg43McZ_obddLkIPAQp_molMvw/edit?tab=t.0#heading=h.spoobki9vra3\r\n\r\n\r\n##
Testing\r\n#### Handling kibana restarts:\r\nSince we dont maintain any
state in Kibana for the data stream\r\nreindexing we dont worry about
this scenario\r\n\r\n#### Ensuring only one node is handling the task
sync with ES:\r\nSince we dont maintain any state in Kibana for the data
stream\r\nreindexing we dont worry about this scenario\r\n\r\n#### ES
task returns an exception:\r\nThis is a new state that was introduced in
persistent tasks. We show a\r\nhuge exception banner and ask user to
retry.\r\n\r\n#### ES task fails to reindex some indices but reindexing
is complete:\r\nShow an error in the in progress page. This is an
undesired state since\r\nthe user will still be asked to reindex the
data stream again for only\r\nthose failed indices.\r\n\r\n#### ES task
fails to reindex some indices but reindexing is still
in\r\nprogress\r\nWe should a count of how many failed while showing a
progress bar.\r\n\r\n#### Pause/resume/cancel functionality\r\ndata
transform reindexing only has ‘cancel’ from ES side however
it\r\nbehaves as “pause” since it will pick up where it last was
cancelled.\r\nSince we do not maintain a state in kibana we only show a
cancel button.\r\nOnce the reindex is cancelled the user can start it
again. All the\r\ndescriptions are updated from ES with the latest
incomplete countes.\r\n\r\n#### ES task stops responding\r\nWe mark the
Kibana task as failed\r\n\r\n#### ES task returns 404\r\nBoth cancelled
tasks and never started ones return 404 from ES side.\r\nThe only way to
tell the difference is to see if the Kibana task is “in\r\nprogress”
then this means it was cancelled from ES side (canelled via\r\nAPI by
the user directly for example) and we mark it as cancelled for\r\nthe
user.\r\n\r\n\r\n#### Other main flow tests:\r\n- Tested a small data
stream reindexing job\r\n- Tested a huge data stream reindexing job\r\n-
Tested pausing a job\r\n- Tested cancelling a job\r\n- Stopping ES
during reindexing and then restarting\r\n- Reindexing in a cluster with
2 Kibana
nodes","sha":"6925d129455ccb458e08cf33ce36c08e0d5313f5","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-major","v8.18.0"],"title":"[Upgrade
Assistant] Data streams
reindexing","number":208083,"url":"https://github.com/elastic/kibana/pull/208083","mergeCommit":{"message":"[Upgrade
Assistant] Data streams reindexing (#208083)\n\n## Data streams
reindexing\r\n\r\nThis PR implements data stream reindexing corrective
action.\r\n\r\n\r\n## Screenshots\r\nFound
here\r\nhttps://docs.google.com/document/d/1QntGX5aTbjAv-VRZoKg43McZ_obddLkIPAQp_molMvw/edit?tab=t.0#heading=h.spoobki9vra3\r\n\r\n\r\n##
Testing\r\n#### Handling kibana restarts:\r\nSince we dont maintain any
state in Kibana for the data stream\r\nreindexing we dont worry about
this scenario\r\n\r\n#### Ensuring only one node is handling the task
sync with ES:\r\nSince we dont maintain any state in Kibana for the data
stream\r\nreindexing we dont worry about this scenario\r\n\r\n#### ES
task returns an exception:\r\nThis is a new state that was introduced in
persistent tasks. We show a\r\nhuge exception banner and ask user to
retry.\r\n\r\n#### ES task fails to reindex some indices but reindexing
is complete:\r\nShow an error in the in progress page. This is an
undesired state since\r\nthe user will still be asked to reindex the
data stream again for only\r\nthose failed indices.\r\n\r\n#### ES task
fails to reindex some indices but reindexing is still
in\r\nprogress\r\nWe should a count of how many failed while showing a
progress bar.\r\n\r\n#### Pause/resume/cancel functionality\r\ndata
transform reindexing only has ‘cancel’ from ES side however
it\r\nbehaves as “pause” since it will pick up where it last was
cancelled.\r\nSince we do not maintain a state in kibana we only show a
cancel button.\r\nOnce the reindex is cancelled the user can start it
again. All the\r\ndescriptions are updated from ES with the latest
incomplete countes.\r\n\r\n#### ES task stops responding\r\nWe mark the
Kibana task as failed\r\n\r\n#### ES task returns 404\r\nBoth cancelled
tasks and never started ones return 404 from ES side.\r\nThe only way to
tell the difference is to see if the Kibana task is “in\r\nprogress”
then this means it was cancelled from ES side (canelled via\r\nAPI by
the user directly for example) and we mark it as cancelled for\r\nthe
user.\r\n\r\n\r\n#### Other main flow tests:\r\n- Tested a small data
stream reindexing job\r\n- Tested a huge data stream reindexing job\r\n-
Tested pausing a job\r\n- Tested cancelling a job\r\n- Stopping ES
during reindexing and then restarting\r\n- Reindexing in a cluster with
2 Kibana
nodes","sha":"6925d129455ccb458e08cf33ce36c08e0d5313f5"}},"sourceBranch":"8.x","suggestedTargetBranches":["8.18"],"targetPullRequestStates":[{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Some Osquery queries are expected to be long running. To accommodate
this, increase the maximum timeout in the query creation UI to 24 hours
(86400 seconds).
24 hours should allow most long-running queries, while still having a
limit that ensures misbehaving queries do not block others for an
extremely long time.
Relates to https://github.com/elastic/beats/issues/42352. Osquerybeat
will also increase its timeout limit to 24h, this change will allow the
higher timeout to be set by users in Kibana.
## Summary
This PR removes `Setup Guides` buttons.
Closes: #207026
> [!NOTE]
All the tests in
`src/platform/plugins/shared/guided_onboarding/public/components/guide_panel.test.tsx`
depend on those buttons so it was easier to remove those tests.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Users should use Lens to visualize data. Users should not use TSVB or
aggregation based panels. To this end, "easy button" UIs, such as
dashboard's "Add panel" flyout, should guide users to lens and not guide
them to legacy applications.
This PR removes `legacy` section from dashboard "Add panel" flyout.
Users can still add legacy panels to dashboards via "visualize"
application. The path is more steps, but this is as intended. Using old
stuff should be painful and require extra work as a caret to move users
to new stuff.
<img width="500" alt="Screenshot 2025-01-23 at 1 08 41 PM"
src="https://github.com/user-attachments/assets/e142cba9-a880-4ad6-a3f8-a981a03daaec"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR gets rid of the usage of the legacy "readdefinition" types in
the frontend except for the enrichment view, because @tonyghiani is
working on that in parallel.
I first looked into whether we can just use the existing
`IngestStreamGetResponse` type, but since it doesn't carry the name of
the stream, that is actually super annoying to use - passing the name as
a separate argument is pretty annoying and increases the risk of bugs.
Even though it's not fully "Elasticsearch"-y, I decided to add the
stream name to the `IngestStreamGetResponse` - this way the `stream`
property in the response is a regular `StreamDefinition`, so all the
functions that act on it can use it right away.
This simplifies the frontend code without hurting programmatic API
users.
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `d04ee51` ->
`8df3ca6` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
## Summary
BUG: https://github.com/elastic/kibana/issues/208907
This PR fixes the issue where user cannot select a different connector
type after mistakenly selecting a wrong one.
### Steps to reproduce without required endpoint installation:
1. Make sure there are no connectors
2. Open "AI Assistant" on one of the security solution pages
3. Press "(+) Add connector" button
4. Connector type selection modal is visible
5. Select "Amazon Bedrock" type (or any other connector type)
6. Cancel the modal
7. Press "(+) Add connector" button again
**ISSUE**: previously selected connector type is being displayed and
there is no way to switch between types
**EXPECTED**: we should show connector type selection modal once
previous one was closed
### Issue recording
https://github.com/user-attachments/assets/48052bf1-4e00-43b7-a63e-f8a7969b9dbf
### Fixed state recording
https://github.com/user-attachments/assets/48be1cc4-0326-43a1-bd57-bb82fc1f19eb
## Details
⚠️ This PR was created by an automated tool. Please review the changes
carefully. ⚠️
Add a workflow that will comment on PRs with AsciiDoc changes.
## Why
During the migration to Elastic Docs v3, the Docs team will focus
exclusively on migrating content.
To maintain consistency, prevent conflicts, and ensure a smoother
transition we will freeze all AsciiDoc changes.
This means you will get a warning when you create AsciiDoc changes in
your PRs.
See https://github.com/elastic/docs-builder/issues/281 for details
If there are any questions, please reach out to the
@elastic/docs-engineering
---------
Co-authored-by: Brandon Morelli <brandon.morelli@elastic.co>
## Summary
We anticipate generating a single document per API version for the
foreseeable future. This PR updates our OAS document merging logic to
avoid adding the `Elastic-Api-Version` header to our spec.
This will also remove this header from current documentation.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
