Closes https://github.com/elastic/kibana/issues/217115
## Summary
Elasticsearch has introduced a new field to component and index
templates called data_stream_options that enables failure stores on a
data stream. This field is not available for legacy templates. This PR
ensure that `data_stream_options` values are persisted in the
component/index template creation wizards, if previously set via the
API.
We will not surface the ability to change the configuration via the UI
until we have time to discuss the proper user experience.
Allow to select the data stream type when creating policies for input
packages.
It is only possible to select the data stream type when creating policies
and not when editing them.
The selector is added as an advanced option.
## Summary
Closes https://github.com/elastic/kibana/issues/204155
Adds support for the following behaviors:
1. Clicking "Open in Lens" opens the Lens editor in the same tab.
2. Using Command (or Ctrl if is not an apple device) + Click opens the
Lens editor in a new tab.
> [!NOTE]
> This is a temporary workaround until a more comprehensive solution,
which requires additional effort, is implemented.
### Details
The goal is that by clicking on the "Open in Lens" button, the Lens
editor is opened in the current tab, and by Command + Clicking, the Lens
editor is opened in a separate tab. Currently, the approach uses the
`navigateToPrefilledEditor` method, exposed by the Lens plugin, which is
called upon execution of the action.
Initially, the idea was to use the shortUrl service to generate a share
URL. This service generates an Elasticsearch object, which can be
excessive since the requirement doesn’t imply generating a URL that can
be shared.
With the current approach, the data is persisted because it is in the
session storage. That is why when we click on the button and it opens in
a new tab, you can see the data. However, if you copy the same URL to
another tab, nothing appears because there’s no data.
If we want to add an href by specifying the `getHref` method to the
action (as suggested
[here](https://github.com/elastic/kibana/issues/204155#issuecomment-2578218129),
it won’t work as you cannot persist the data when you click on a link.
To maintain the requested behavior, we update the openInNewTab parameter
that is passed to the Lens `navigateToPrefilledEditor` method to be true
when it is a modified event or false when it is not.
## Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Related to
[elastic/kibana#220620](https://github.com/elastic/kibana/issues/220620)
Following a review of the upcoming button visual refresh planned by the
EUI team for early June, we identified an issue with the background
styling on dropdown buttons within the Infrastructure Inventory view.
<img width="496" alt="image"
src="https://github.com/user-attachments/assets/5e293034-fd44-4d0e-824b-a66863c0ae8a"
/>
### Changes Implemented
- Removed the background from the dropdown button label.
- Added a right border to improve visual clarity.
These changes are safe to merge into `main` ahead of the EUI release, as
they do not impact the current UI behavior.
## Summary
fix#221241
## Bug and fix descriptions
The logic that applies a time filter by updating the time picker vs
adding it as a filter pill in the filter bar is described here:
e877bafa3e/src/platform/plugins/shared/unified_search/public/actions/apply_filter_action/apply_filter_action.tsx (L107-L114)
The `extractTimeFilter` function extracts the `timeRangeFilter` only if
the `timeFieldName === keys(filter.query.range)[0]` . The problem was
that the first key of `query.range` was different than the
`timeFieldName`. That `timeFieldName` in the ESQL world was wrongly
applied via the ` table.columns[xAxisColumnIndex].name` that doesn't
reflect the actual column name for ES|QL, in fact that name reflect the
visualization label name for the time dimension. In particular, before
the fix, that timeFieldName was set as the axis name (e.g.`@timestamp
every 5 minute` ) and cause the time range filter to fail to be
extracted from the filters list.
A combination of two PRs caused this to be now anymore the correct
behaviour:
- this PR https://github.com/elastic/kibana/pull/196049 introduced the
use of the `souceField` as field name for the creation of filter from a
range
- this PR https://github.com/elastic/kibana/pull/217719 instead
introduced the use of `sourceField` also for ESQL datasources. This
field points to the actual column name described in the ESQL query.
Both PR causes the `extractTimeFilter` to fail to extract the
timeFilter, pushing the filter up to the filter pills.
### Side notes
This could be probably fixed in other ways, like by avoiding using the
`sourceField` in ESQL, or by using only the column.name in the filter
creation, or by avoiding checking the `timeFieldName` against the
`query.range` key (not really sure why this is required). In general the
problem here is that there is a low confidence on what these
fields/params are supposed to be and which is supposted to be the
identifiers to use everywhere. For example the column ids reflects only
a link between the rows and the column descriptions, the name is the
associated label, but can we rely on that label for filtering? i believe
we need a stronger connection with the data and the actual original
source field or column identifier is a better choice.
I believe a valid subsequent task is
https://github.com/elastic/kibana/issues/189044
In the refactoring of the streams API state management, a bit of unused
code wasn't deleted. This PR removes the stuff we don't actually need
anymore.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Updates functional tests to ingest only one document, avoiding flaky failures caused by inconsistent ordering. With only one document, ordering issues are no longer possible.
Relaetd to https://github.com/elastic/kibana/issues/183248
Auto-increasing mapping limit PR:
https://github.com/elastic/kibana/pull/216719
## Summary
In this PR, we are saving dynamically mapped group by information for
the custom threshold rule. This consists of two parts:
1. Adding a dynamic field
```
// kibana.alert.grouping
[ALERT_GROUPING]: {
type: 'object',
dynamic: true,
array: false,
required: false,
},
```
2. Adding a dynamic template
```
dynamicTemplates: [
{
strings_as_keywords: {
path_match: 'kibana.alert.grouping.*',
match_mapping_type: 'string',
mapping: {
type: 'keyword',
ignore_above: 1024,
},
},
},
],
```
The result of adding these mappings can be seen below:
|Alert|Mapping|
|---|---|
||
If the number of mapping limit is exceeded, the fields that are not
mapped are going to be added to the `_ignored` field, but the value is
available in the doc.
<img
src="https://github.com/user-attachments/assets/b84bcf03-b757-4f37-a93f-2559aefa5bcf"
width=500 />
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base-fips | digest | `a9e07d9` ->
`68e0781` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwiY2k6YnVpbGQtZG9ja2VyLWZpcHMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/217379
## Summary
Updating task manager README docs to provide guidance on how to schedule
user-scoped tasks. Also made some other miscellaneous edits to try to
reflect the current status of the plugin.
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `55ee1dc` ->
`3d19648` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
## Summary
Our tests rely on static test data, and our cypress suite was running
rules with a large lookback (50000h) against them. Recently, tests have
begun failing because we've exceeded 50000h since the data's
`@timestamp`. This PR updates the cypress rules to use a fixed `from`
field wherever possible, in order to "fix" that query window's starting
point, ensuring that this issue does not recur. This is already how our
integration tests are creating similar rules, so as an added bonus we're
now more consistent across test suites, as well.
This unskips and thus resolves the following issues:
* https://github.com/elastic/kibana/issues/201334
* https://github.com/elastic/kibana/issues/220822
* https://github.com/elastic/kibana/issues/207913
* https://github.com/elastic/kibana/issues/199905
Note that there may be more instances of skipped tests that were failing
due to this issue (with e.g. data with an earlier date than the
`2019-09-02` here); the above tests were just the most obvious and most
recent (within the last week).
### Flaky Runner
* Detection Engine Cypress (x150):
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8291
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
So far failure store was enabled for wired streams on stateful (which is
not released yet). Until we know how the experience should look like,
this PR disables failure store. Once we have a decision, we can enable
it in a unified way for stateful and stateless, since both of these
support failure store now.
## Summary
This PR performs a very small renaming of variables, just to make sure
that `DataView` and `DataViewSpec` are not confused with each other.
We had a team discussion as we could extract some of the logic to
retrieve the old and experimental `DataViewSpec` done in a separate
hook, but we decided to go against that for now.
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base-fips | digest | `6e347a2` ->
`a9e07d9` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwiY2k6YnVpbGQtZG9ja2VyLWZpcHMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `67d82bc` ->
`55ee1dc` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/221321
### test instructions
1) install sample web logs
2) import canvas saved object
https://github.com/nreese/notes/blob/master/empty-canvas-workpad-saved-object-export.ndjson
3) refresh kibana
4) open canvas and add map embeddable
5) open browser network tab
6) click "Refresh data" button. Verify map requests new data
7) open "View" menu. Click "Refresh data". Verify map requests new data
8) set auto internal to "5s". Verify map requests new data on each
interval
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Similar to the following previous PRs([this
one](https://github.com/elastic/kibana/pull/216572) and [that
one](https://github.com/elastic/kibana/pull/219878)), this PR aims at
removing some logic built-in to the `GroupedAlertsTable` and pass the
information via props. It makes this reusable component a lot easier to
use in different scenarios.
In this current case, we're removing the retrieval of the dataView
within the `GroupedAlertsTable`. We now rely on the `DataViewSpec`
object passed via prop instead. This allows us to get rid of the
following props:
- `signalIndexName`, which we now retrieve from the DataViewSpec object
directly
- `runtimeMappings`, which we also now retrieve from the DataViewSpec
object directly
This solves one issue, which was related to the fact that the
`GroupedAlertsTable` was retrieving the `detections` dataView
internally, so there was a high chance that the `signalIndexName` and
`runtimeMappins` passed via props would not match the hardcoded
`detections` dataView retrieved internally... which is very confusing!
We are having this problem in the AI4DSOC Alert summary page, which
creates a adhoc dataView just for alerts...
**_No UI or behavior change are introduced in this PR!_**
Alerts page
https://github.com/user-attachments/assets/a4e0c1a6-fa91-4b1e-881c-56d95667e84b
Rule details page
https://github.com/user-attachments/assets/6d93e7d0-0446-4ecb-aa37-4ab266b62686
AI4DSOC Alert summary page
https://github.com/user-attachments/assets/7354bad2-64d1-4722-94c3-ae2d9b72fcab
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
This PR removes the `securitySolution:enableVisualizationsInFlyout`
advanced settings that allowed users to switch between Analyzer and
Session View components displayed in an overlay on top of tables, or
within the flyout expanded panel.
The setting was already defaulted to true, so this PR just removes the
ability to go back to the analyzer and session view components displayed
as table overlay.
This PR is the first of 4 that perform the clean up:
- this first PR that removes the advanced setting
- a follow up PR that will remove the code related to rendering session
view as overlay
- a follow up PR that will remove the code related to rendering analyzer
as overlay
- a final clean up that removes a lot of dead code related to timeline
full screen and such
The work is divided in multiple PRs because when I got everything
working locally it ended up touching nearly 120 files, making the review
difficult...
---------------------------
No more advanced settings
The alerts table goes to the flyout
https://github.com/user-attachments/assets/b8b934ba-ea9f-4bb2-aaf4-0f73f359eb3a
The table still works from the Cases page
https://github.com/user-attachments/assets/65ffe289-3a3c-4766-aa52-4f68a2f2dd87
And from timeline (the tabs are not showing up either)
https://github.com/user-attachments/assets/0bb17b2f-b6b7-4417-9212-2302b778cc23
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Tags user errors in all rule types. My hope is by placing the user error
function in the rule wrapper we can cover all the rule type errors
without having to put custom logic into each executor's catch block, and
other nested functions.
I added some unit tests but for manual testing the two new errors I
added to `checkErrorDetails` you can do the following:
1. install the prebuilt rule `Rapid7 Threat Command CVEs Correlation`
and start it. This is a threat match rule that will throw an
index_not_found exception because the threat index does not exist.
2. Install the ml rule `Unusual Process Spawned by a User` and enable
it. The rule will fail because the job is missing.
3. Add a console log statement within
`x-pack/solutions/security/plugins/security_solution/server/lib/detection_engine/rule_types/utils/check_error_details.ts`
to ensure `isUserError` is true for each given rule failure, or check
the event log in dev tools.
