## Summary
Closes https://github.com/elastic/kibana/issues/199188
Allow multiple SAML authc calls to succeed.
## Testing
Configure logging:
```yaml
logging.loggers:
- name: plugins.security
level: debug
```
### See the failure
Pull `main` and copy the code from the following files in this PR into
their respective files on that branch:
- `packages/kbn-mock-idp-plugin/public/login_page.tsx`
- `packages/kbn-mock-idp-plugin/server/plugin.ts`
- `packages/kbn-mock-idp-utils/src/index.ts`
- `packages/kbn-mock-idp-utils/src/utils.ts`
Start KB/ES in serverless from this modified main branch
Open 2 tabs to the local serverless login screen
As the same user, click login and change tabs and click login again
The you will get an error.
Shut down KB/ES
### See the success
Start KB/ES in serverless from this PR
Open 2 tabs to the local serverless login screen
As the same user, click login and change tabs and click login again
Both should succeed
## Release note
Refreshing multiple tabs where the user has logged out will
simultaneously login successfully
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base-fips | digest | `0135014` ->
`b6d3d24` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwiY2k6YnVpbGQtZG9ja2VyLWZpcHMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `c56628d` ->
`1c7f5aa` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Closes: #214335
**Description**
Dialog modal, flyout, field visible title should be announced for the
users, especially using assistive technology to know what dialog modal,
flyout opened, what field is active and what is needed to enter in it.
**Changes made:**
1. Added `aria-labelledby={flyoutTitleId}` for mentioned places
Fixes image paths to work with docs-assembler.
Notes for the reviewer:
* I was not able to get images in reference, extend, or release-notes to
work using the `:::{image}` syntax because it seems to resolve
differently than the Markdown `![]()` syntax. We should address this in
docs-builder, but in order to get images working as soon as possible,
I've used Markdown syntax and left us a `TO DO` in a code comment to add
back the `screenshot` class where applicable.
* Can you please add the appropriate labels needed for backporting?
## Summary
This PR fixes the issue with the error summary missing items using edot.
It includes e2e tests with synthtrace for both edot and otel services.
TODO
- [x] Test with serverless (waiting for the PR to be deployed)
Tested on serverless works as expected:
<img width="2560" alt="image"
src="https://github.com/user-attachments/assets/8dd7962e-7d66-482d-97fb-0b08882bd04f"
/>
Allow group by for ip fields !!
---------
Co-authored-by: Faisal Kanout <faisal.kanout@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/217923
Investigations in https://github.com/elastic/kibana/issues/217368 showed
that there was basically no performance impact to passing the AST across
a thread boundary. But we also didn't detect a pressing reason to remove
the worker.
Since then, however, we noticed another cost associated with the worker:
it's a hefty Javascript file, even in production builds. In addition, we
are doing parsing on the main thread _and_ the worker, so the
`kbn-esql-ast` package is actually being loaded and parsed twice by the
browser, once for the main thread and once for the worker.
This PR removes our worker. Our parsing associated with validation and
autocomplete will still be done asynchronously, but on the main thread.
I do not see any regression in perceived performance.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
This PR closes#215668.
The global parameters are synched in the endpoints where they are
created, edited or deleted.
---------
Co-authored-by: Shahzad <shahzad31comp@gmail.com>
Fixes https://github.com/elastic/kibana/issues/212902
## Summary
This PR fixes the logic for finding folding ranges by ignoring
opening/closing markers inside triple-quote strings.
**How to test:**
Verify that the `script` field in the following request is folded
correctly:
```
POST _ingest/pipeline/_simulate
{
"pipeline": {
"processors": [
{
"script": {
"source":
"""
for (field in params['fields']){
if (!$(field, '').isEmpty()){
def value = $(field, '');
def hash = value.sha256();
// Now we need to traverse as deep as needed
// and write to that field
// because we do not have a simple
// set operation available
"SCRIPT": parts = field.splitOnToken('.');
}
}
""",
"params": {
"fields": [
"user.name",
"geo.city",
"does.not.exist",
"this.is.quite.a.deep.field"
]
}
}
}
]
}
}
```
Note: The logic is for finding the ranges is best-effort without
compromising performance. We currently iterate through each line in the
text and use regex, but there are some cases which are not covered by
this logic; for example, opening parenthesis, followed by a string in
the same line would not be foldable. In order to cover all cases
correctly, we would need to iterate through every single character, but
that would make the logic much more complex and might affect performance
if we have a lot of text in the editor, as these folding ranges are
computed on every change in the editor.
OTel is expected to become more common over time. This PR adds a
synthtrace scenario to send sample otel logs to an otel data stream.
This helps testing whether Discover, Streams and so on function
correctly for this category of data.
Closes: #216147
**Description**
Dialog modal, flyout, field visible title should be announced for the
users, especially using assistive technology to know what dialog modal,
flyout opened, what field is active and what is needed to enter in it.
**Changes made:**
1. Added` aria-labelledby={flyoutTitleId} `for mentioned places
## Summary
Main ticket ([Internal
link](https://github.com/elastic/security-team/issues/12007))
These changes add the attack discovery schedules management table.
https://github.com/user-attachments/assets/619ad1d6-d919-4a8d-b743-6a73fbfbf318
## Key changes
* UI side API handlers
* Create schedule workflow
* Schedules table
* Enable schedule from the table
* Disable schedule from the table
* Delete schedule from the table
* Pagination and sorting in find schedules API
## NOTES
The feature is hidden behind the feature flag (in `kibana.dev.yml`):
```
feature_flags.overrides:
securitySolution.assistantAttackDiscoverySchedulingEnabled: true
```
This PR passes the current sample documents to the default form state
generation for new processors to pick a good default field.
The logic that's actually employed for `dissect` and `grok` is the
following:
* Go through all docs and order string fields occurring by how many
values they have
* Pick the top one from a list of "well known" fields that probably make
sense (in case of a tie, go by a the ordering of the well known fields)
* If no field is found this way, just leave it empty - this still shows
the full table and the user can pick the field they care about
Especially for otel this should be helpful.
## Summary
This PR better separates the request building logic in the detection
engine from query building logic, removes outdated error checking logic,
updates the `singleSearchAfter` `search` call to no longer use the
legacy `meta: true` param, and improves search response type inference.
Closes: https://github.com/elastic/kibana/issues/217511
**Description**
Dialog modal, flyout, field visible title should be announced for the
users, especially using assistive technology to know what dialog modal,
flyout opened, what field is active and what is needed to enter in it.
**Changes made:**
1. Set correct value for `aria-labelledby` attr.
## Summary
When the datatable comes with empty results the visualization fails with
bad way
<img width="396" alt="image"
src="https://github.com/user-attachments/assets/b4e266d7-edbd-452b-9192-84c957fe98db"
/>
With the fix
<img width="756" alt="image"
src="https://github.com/user-attachments/assets/d061d29e-9246-432a-944b-308b88d161e7"
/>
How to replicate:
- Create a field ES|QL control with 2 values (extension and geo.dest).
You can do it with multiple ways. I created with typing `FROM
kibana_sample_data_logs | STATS count = COUNT(*) BY` and then `Create
control`.
- Use the variable in another panel with query: `FROM
kibana_sample_data_logs | WHERE ??field == "css" | KEEP extension` (The
control value should be in the extension). This will work
- Select the second field (geo.dest). This will return an empty query
and will break the table viz.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
This PR makes sure a buggy `security_detection_engine` package doesn't affect a preview installation endpoint. Older security detection rules package versions contain saved object rule duplicates affecting the endpoint.
Having `security_detection_engine` v`8.17.1` package installed `/internal/detection_engine/prebuilt_rules/status` and `/internal/detection_engine/prebuilt_rules/installation/_review` endpoints return a different number of rules available to install.
## Details
Older `security_detection_engine` package versions contain rule saved objects duplicates representing the latest version. For example, `8.17.1` version has a rule `Microsoft 365 User Restricted from Sending Email` with `rule_id` = `0136b315-b566-482f-866c-1d8e2477ba16` and the latest version `206`. Since a package may contain multiple historical rule versions it sticks to the following format `<rule_id>_<version>` where `<rule_id>` is the unique rule's UUID and `<version>` it's version. Some older package versions omit `<version>` for the latest rule version. `Microsoft 365 User Restricted from Sending Email` rule mentioned above has two equal assets corresponding to the latest version with the only difference in the saved object id `0136b315-b566-482f-866c-1d8e2477ba16` and `0136b315-b566-482f-866c-1d8e2477ba16_206`.
Prebuilt rules preview endpoint was designed to handle `<rule_id>_<version>` format only. Consequently, it improperly handles older prebuilt rules package version.
This bug manifested in https://github.com/elastic/kibana/pull/217544 where `security_detection_engine` version has been bumped to `8.18.1`. It resulted in a failed integration test. Further investigation has shown that the test installs an older package version `8.17.1` to assert prebuilt rules upgrade workflow works correctly.
The fix is implemented in `PrebuiltRuleAssetsClient.fetchAssetsByVersion()` by using `Map` to deduplicate prebuilt rule assets.
## Summary
Follow-up to #217153
### Problem Description
In UI tests, there was no reliable way to determine when the Alerts
table content had fully loaded before interacting with it. This could
lead to flaky tests where interactions occurred before the data was
available (rows are not present yet), causing failures or inconsistent
results (checking for row with specific content to exist)
Quite often we see tests waiting for global indicator (spinner in the
top left corner) to be hidden as a condition for page loading is
complete. This is quite unreliable approach and testing tools have no
consistent built-in solution: FTR, Cypress or even Playwright - network
idle wait is officially marked as
[discouraged](https://playwright.dev/docs/api/class-page)).
We need to help testing tool to interact with UI components in ready
state only.
### Solution
To address this issue, I modified a `data-test-subj` property in the
`<EuiDataGrid>` component. The property dynamically switches between
`alertsTableIsLoading` when data is still loading and
`alertsTableIsLoaded `once the content is available. This allows UI
tests to wait for precisely `alertsTableIsLoaded` to be in in the DOM
before interacting with the table, ensuring more reliable and stable
test execution.
Passed 10/10
<img width="538" alt="image"
src="https://github.com/user-attachments/assets/e44bae5f-4094-4ed2-89f3-74a52cb2be53"
/>
Fixes https://github.com/elastic/kibana/issues/204963
## Summary
This PR fixes the SQL lexer rules which, previously, recognized a single
`/` as a start of comment, which is not a valid SQL syntax. According to
the [SQL
docs](https://docs.oracle.com/cd/B14117_01/server.101/b10759/sql_elements006.htm),
there are only two types of comments - the single-line one that starts
with `--` and the multi-line one that starts with `/*` and ends with
`*/`.
**What this fixes:**
Nested SQL script in Console that contains the `/` character was
incorrectly highlighted as a comment and was messing up the highlighting
of the rest of the request.
Test with the following request. Make sure that the SQL script in the
`"query"` object is highlighted correctly as well as the rest of the
request.
```
PUT _watcher/watch/ec_alerts_dependency_check
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"chain": {
"inputs": [
{
"initial_search": {
"search": {
"request": {
"indices": ["ec_alerts"],
"body": {
"query": {
"bool": {
"must_not": [
{
"exists": {
"field": "case_id"
}
}
]
}
},
"size": 1,
"_source": ["dependent.service.name"]
}
}
}
}
},
{
"dependent_search": {
"search": {
"request": {
"indices": ["ec_alerts"],
"body": {
//"_source": ["alert","service.name","dependend.service.name"],
"size": 999,
"query": {
"query_string": {
"query": """
{{#ctx.payload.initial_search.hits.hits.0._source.dependent}}
{{#service.name}}({{.}}) OR {{/service.name}}{{/ctx.payload.initial_search.hits.hits.0._source.dependent}} ({{ctx.payload.initial_search.hits.hits.0._source.dependent.0.service.name.0}})""",
"default_field": "service.name"
}
}
}
}
}
}
}
]
}
},
"condition": {
"compare": {
"ctx.payload.initial_search.hits.total": {
"gt": 0
}
}
},
"actions": {
"log_results": {
"logging": {
"text": "Found related alerts: {{ctx.payload.dependent_search.hits.total}}"
}
}
}
}
```
## Summary
- Adds ability to traverse `map` and `map-entry` AST nodes.
- In `Walker`
- And in `Visitor`
- Adds support for pretty-printing ES|QL maps
- In `BasicPrettyPrinter`, as well as, in the `WrappingPrettyPrinter`.
- Supports comments printing in both printers in any place: around maps,
as well as around any map key and map key value.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR fixes the missing allowHidden/allow_hidden usage for data views using the "allow hidden and system indices" functionality when creating a data view. There are 2 endpoints being requested in Discover: `/fields` & `/fields_for_wildcards`
Both use `allow_hidden=true`, but `/fields` ignored this parameter internally. This was fixed.
fixes [#189213](https://github.com/elastic/kibana/issues/189213)
## Summary
Checks whether the user has permission to ML before triggering requests
to fetch ML data
### How to test
- Create a user whose role doesn't have permission to ML, but has
permission to O11y apps
- Run `node scripts/synthtrace infra_hosts_with_apm_hosts --live
--clean`
- Navigate to Inventory Infrastructure / Hosts View
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Introduce a new Entity Engine Definition called Generic. The larger
context on why we are introducing a new entity definition is described
on this [private github
issue](https://github.com/elastic/security-team/issues/11857).
The tldr; is that we would like to have an entity store with all the
entities described by the [entity ecs
field](https://github.com/elastic/security-team/issues/11857). The
decision to call `generic` entity definition comes from the fact that
any entity can be described with the `entity` field - user, host,
service, database, queue, subscription and so on. Therefore it makes
sense to have the concept called `generic` entity, meanwhile the
existent entity definitions will be called concrete entities, because
they describe a very concrete type of entity (currently user, host,
service).
Other changes included on this PR:
- Don't override `entity.name` with `entity.id`, only set if no value is
found
- Migrate the usage of `entity.type` as the entity definition type to
`entity.EngineMetadata.Type`
- Changes touching Entity Analytics code around
`getRiskEngineEntityTypes` and `getAssetCriticalityEntityTypes`. There
was a somewhat unnecessary and duplicated logic in these functions which
essentially described the concrete entity definitions to be used by
entity analytics flows. A new function called
`getEntityAnalyticsEntityTypes` was introduced which unifies this logic
and returns the entity types that Entity Analytics care about.
Video of a scroll through the entities processed by the generic entity
store, source of the data is cloudbeat asset management integration.
https://github.com/user-attachments/assets/450afd05-dee0-4449-aaec-2cd69645d6ec
#### How to test:
- In Advanced Settings (`/app/management/kibana/settings`), enable
`securitySolution:enableAssetInventory`
<img width="883" alt="image"
src="https://github.com/user-attachments/assets/c342abb2-efb3-40a8-b945-d9558f085f34"
/>
- In Entity Store management (`/security/entity_analytics_entity_store`)
enable entity store
<img width="1251" alt="image"
src="https://github.com/user-attachments/assets/41f709e1-0aea-47dc-9c98-ffaebf18fdb1"
/>
- Verify Generic Engine Status
<img width="1203" alt="image"
src="https://github.com/user-attachments/assets/d26b764a-4695-436e-85f7-e3ed7df5a3be"
/>
- Ingest documents with `entity.id` and `entity.*` fields. Personally I
run `cloudbeat` asset discovery locally
- Verify ingested documents in
`.entities.v1.latest.security_generic_default`
<img width="1496" alt="image"
src="https://github.com/user-attachments/assets/88286cb9-38c1-4f9d-83a7-57ba33811c60"
/>
--
**OBS: Also test enabling the store without the uiSetting enabled, so
you can make sure that it doesn't enable**
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [x] I see risk on performance, given the amount of aggregations the
generated transform does
- tested, although we see a higher spike in CPU than before, it's behind
a feature flag and it's going to be used in controlled data sets (entity
centric logs that contain `entity.id` field) we decided it's good enough
to go.
- [ ] Enablement/disablement of entity store in a different uiSetting
configuration.
- [ ] Enable entity store with `securitySolution:enableAssetInventory`
disabled. Then enable `securitySolution:enableAssetInventory` ==> No
generic entity definition installed. You can manually install it in the
EntityStore status page
- [ ] Enable entity store with `securitySolution:enableAssetInventory`
enabled. Then disable `securitySolution:enableAssetInventory` definition
==> hanging assets of generic entity store that can be deleted manually
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/security-team/issues/10837
**WIP Draft**
Add first integration test for Security Solution's Entity Store. This
test checks basic API endpoint (`status`) and tests the first of three
transforms (`host`) on a single field (`host.ip`).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/217994
## Summary
The comment section containing a table that requires horizontal
scrolling was being visually pushed to the right due to the presence of
the user avatar. To ensure the comment section aligns properly with the
rest of the content and doesn't overlap the sidebar reserved for
connectors and custom fields, the `max-width` property was adjusted
accordingly.
The offset comes from:
- the width of the euiAvatar--m (which uses the `xl` size token)
- the left padding of the `euiTimelineItemEvent`, which corresponds to
`euiTheme.size.base` (16px)
## Summary
Closes https://github.com/elastic/kibana/issues/215157
This is fixing the wrong client side validation error that appears here.
```
FROM kibana_sample_data_ecommerce
| EVAL customer_id = TO_LONG(customer_id)
| LOOKUP JOIN customers ON customer_id
```
<img width="588" alt="image"
src="https://github.com/user-attachments/assets/50a9365f-91c6-45c8-ad04-48be5145eb77"
/>
We are using the new originalTypes that is being returned by ES to
identify if the field is a conflict and turn off the client side
validation for these fields.
This is not closing the aforementioned issue. It is the first step to
improve the join autocomplete experience. The fields retrieval for joins
is happening by running `from index1, lookup_index` but in case of
conflicts this is problematic. We could def improve it in the future but
identifying for now that this is a conflict, muting the client side
validation and let ES handling the error is better than what we have
now.
As a bonus the field list recognizes them too as conflict now
<img width="683" alt="image"
src="https://github.com/user-attachments/assets/7edf8cdf-156b-4933-b9dc-225e211bf2ec"
/>
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
