Commit graph

83349 commits

Author SHA1 Message Date
elastic-renovate-prod[bot]
f96c28faab
Update Mustache to v4 2025-01-30 18:13:34 +00:00
elastic-renovate-prod[bot]
aec58c14ef
Update docker.elastic.co/wolfi/chainguard-base:latest Docker digest to 8df3ca6 (main) (#208931)
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `d04ee51` ->
`8df3ca6` |

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
2025-01-30 11:56:06 -06:00
Shahzad
6dda88821d
[Synthetics] Clarify private location api docs !! (#208504)
## Summary

Clarify private location api docs !!

Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co>
2025-01-30 18:44:40 +01:00
Tiago Costa
bd0d6820af
skip failing es promotion (#208874) 2025-01-30 17:29:36 +00:00
Tiago Costa
28e431c7c8
skip failing es promotion (#208870) 2025-01-30 17:27:07 +00:00
Tiago Costa
48d2e4c0b1
skip failing es promotion (#208871) 2025-01-30 17:24:32 +00:00
Tiago Costa
3537b2d008
skip failing es promotion (#208872) 2025-01-30 17:22:10 +00:00
Tiago Costa
d3a115b326
skip failing es promotion (#208873) 2025-01-30 17:17:20 +00:00
Jen Huang
010ef6b4c5
[UII] Convert deployment mode selector to radio group (#208859) 2025-01-30 08:58:00 -08:00
Ievgen Sorokopud
b28036a2e0
[Security GenAI] "Select a Connector" popup does not show up after the user selects any connector and then cancels it from Endpoint Insights. (#208907) (#208969)
## Summary

BUG: https://github.com/elastic/kibana/issues/208907

This PR fixes the issue where user cannot select a different connector
type after mistakenly selecting a wrong one.

### Steps to reproduce without required endpoint installation:

1. Make sure there are no connectors
2. Open "AI Assistant" on one of the security solution pages
3. Press "(+) Add connector" button
4. Connector type selection modal is visible
5. Select "Amazon Bedrock" type (or any other connector type)
6. Cancel the modal
7. Press "(+) Add connector" button again

**ISSUE**: previously selected connector type is being displayed and
there is no way to switch between types
**EXPECTED**: we should show connector type selection modal once
previous one was closed

### Issue recording


https://github.com/user-attachments/assets/48052bf1-4e00-43b7-a63e-f8a7969b9dbf

### Fixed state recording


https://github.com/user-attachments/assets/48be1cc4-0326-43a1-bd57-bb82fc1f19eb
2025-01-30 17:53:36 +01:00
Jan Calanog
c7cef4c95b
github-action: Add AsciiDoc freeze warning (#208534)
## Details

⚠️ This PR was created by an automated tool. Please review the changes
carefully. ⚠️

Add a workflow that will comment on PRs with AsciiDoc changes.

## Why

During the migration to Elastic Docs v3, the Docs team will focus
exclusively on migrating content.
To maintain consistency, prevent conflicts, and ensure a smoother
transition we will freeze all AsciiDoc changes.

This means you will get a warning when you create AsciiDoc changes in
your PRs.

   See https://github.com/elastic/docs-builder/issues/281 for details

If there are any questions, please reach out to the
@elastic/docs-engineering

---------

Co-authored-by: Brandon Morelli <brandon.morelli@elastic.co>
2025-01-30 16:41:18 +00:00
Steph Milovic
b0a72e2b90
[Security AI] Move security-ai-prompt to package (#208787) 2025-01-30 17:34:30 +01:00
Kevin Qualters
0e7c608ed3
[Cases] Cases assignees sub feature (#201654)
## Summary

This pr implements a new cases assignee sub-feature, allowing users to
control a role's ability to change the assignee of a case. With the
permission enabled, they can assign any user to any case, with it
disabled, the assignees component is hidden.

Read only + enabled:

![image](https://github.com/user-attachments/assets/ba421784-d976-4ae9-a399-e404c26b3842)


All + assign disabled:

![image](https://github.com/user-attachments/assets/d835b6f9-5a14-4ae0-abed-b3c3252c2692)



### Checklist

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2025-01-30 17:04:38 +01:00
Jean-Louis Leysens
53cba303e5
[OAS] Remove Elastic-Api-Version (#207071)
## Summary

We anticipate generating a single document per API version for the
foreseeable future. This PR updates our OAS document merging logic to
avoid adding the `Elastic-Api-Version` header to our spec.

This will also remove this header from current documentation.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2025-01-30 16:59:45 +01:00
Tre
c1666be580
[SKIP ON MKI] .../open_in_lens/tsvb/*.ts (#208927)
see details: https://github.com/elastic/kibana/issues/208926
2025-01-30 16:49:37 +01:00
Shahzad
97e8bada1d
[Synthetics] Update pings query to not use last 30 days time range !! (#208366)
## Summary

Fixes https://github.com/elastic/kibana/issues/198443 !!

Instead of using last 30 days, it will now uses last 24 hours otherwise
it will fall back to last 7 days and then last 30 days.

Last run should still work as expected 

<img width="1728" alt="image"
src="https://github.com/user-attachments/assets/abad5d83-5456-440e-b5b5-147915315120"
/>

---------

Co-authored-by: Justin Kambic <jk@elastic.co>
2025-01-30 16:01:27 +01:00
Alex Prozorov
cd9096cf73
[Cloud Security] 10973 migrate flaky e2e tests to jest (#208345)
## Summary

This PR tries to fix the following issues - which are flaky FTR tests:

- https://github.com/elastic/kibana/issues/197143 - convert to unit test
- https://github.com/elastic/kibana/issues/190967 - fixed
- https://github.com/elastic/kibana/issues/187470 - fixed
- https://github.com/elastic/kibana/issues/186302 - convert to unit test
- https://github.com/elastic/kibana/issues/186438 - convert to unit test


There will be an RFC document which is going to be released to help us
better understand and decide which tests are more suitable to make as
E2E tests and which as unit tests.

### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed

### Closes
this PR closes the above mentioned issues in relation for this ticket -
https://github.com/elastic/security-team/issues/10973
2025-01-30 16:23:25 +02:00
Nicolas Chaulet
0a0a4d8b75
[Fleet] Improve reading package archive memory usage (#208869)
## Summary

Related to https://github.com/elastic/kibana/issues/208210 

As we know the package size from the content-length header we can
improve how read the archive stream to a buffer.

## Benchmark 

<img width="710" alt="Screenshot 2025-01-29 at 9 23 59 PM"
src="https://github.com/user-attachments/assets/79dc1f20-938b-402e-a823-1ab26a07b78e"
/>
2025-01-30 09:03:52 -05:00
Kris Gross
bf0b26d4de
Security periodic pipeline images non-preemptible (#208799)
Pre-GA, we previously modified the critical path, weekly Quality Gate
pipelines over to non-preemptible images in order to avoid the
inconsistent "agent lost" Buildkite issue that might cause unnecessary
churn.

Now that we're GA, the periodic pipelines are becoming more critical
path as well, and in addition, we're seeing a concerning increase in the
number of "agent lost" issues we're encountering, as described here:
https://github.com/elastic/security-team/issues/11700

As such, this PR is hoping to address this issue for the periodic
pipelines as well.
2025-01-30 08:58:31 -05:00
Joe Reuter
1b663384fe
🌊 Streams: Handle stale classic streams (#208221)
An invariant we have to handle somehow is if a user made additions to an
unwired data stream via the streams interface, then the underlying data
stream gets deleted.

This is "allowed", since the data stream is not managed by streams.

Currently, the UI breaks if this happens and shows error toasts when
trying to load the doc count or when trying to change processing.

This PR makes this a regular case the API can handle:
* The `GET /api/streams/<id>` endpoint does not throw, but still returns
the existing definition. A new key `data_stream_exists` indicates
whether we are in this situation
* The UI clearly communicates to the user and doesn't try to do
additional requests

<img width="838" alt="Screenshot 2025-01-24 at 16 42 23"
src="https://github.com/user-attachments/assets/92cc5a82-2bb2-4d66-b47a-057ddcff5888"
/>

Trying to update ingest via the API will still result in an error.

Another weird behavior related to that was that if a dashboard is linked
to a stale classic stream, no definition is ever saved and the stream
disappears from the list when deleted, making the dashboard link
inaccessible. This PR introduces `ensureStream` which is called by the
dashboard APIs and makes sure the definition is there if dashboard links
exist. As a side effect, this makes sure that a user can't add dashboard
links to a stream they don't have access to - IMHO we should have done
that from the start.

This does not touch wired streams - for those, the data stream getting
deleted is a breach of contract. We should still handle it gracefully,
but in this case I think we should go another route and offer a button
in the UI to use the "resync" API to reconcile the state of the streams
layer and Elasticsearch. I will look into this on a separate PR.
2025-01-30 14:56:22 +01:00
Shahzad
b655d78773
[OBS-UX-MG] Upgrade redux-saga !! (#208173)
## Summary

Upgrade redux-saga !!
2025-01-30 14:46:33 +01:00
Matthew Kime
7bc08f05b4
[search profiler] faster check for indices (#208825)
## Summary

Search profiler checks to see if there are indices before allowing the
execution of a search to prevent a "there are no indices" error. This PR
makes this check more performant.

In large clusters, requesting a full list of indices can be slow (up to
4 seconds) and results in excess data transfer. The indices exist api
simply returns a boolean if an index is found and remains performant on
large clusters.
2025-01-30 07:29:33 -06:00
Joe Reuter
15fcb182e3
🌊 Streams: Add stream.name field (#208514)
Fixes https://github.com/elastic/streams-program/issues/103

This PR adds a constant keyword `stream.name` field that functions
similar to the `data_stream.dataset` field.

It will allow us to clearly associate a document or a set of documents
with their streams and filter data per stream efficiently. It's also
used to validate that documents are sent properly to the root stream
instead of targeting specific child streams directly.

The `stream.name` field reports as `keyword` in the UI, but is actually
mapped per index template as a constant keyword set to the respective
value.
2025-01-30 14:25:19 +01:00
Tre
67719f2b57
[SKIP ON MKI] .../security/cloud_security_posture/graph.ts (#208904)
see details: https://github.com/elastic/kibana/issues/208903
2025-01-30 13:33:28 +01:00
Agustina Nahir Ruidiaz
1e63b5ae99
[Security Solution] [EUI Refresh] [Explore page] Migrate styled-components (#207318)
## Summary
Addresses [#206437](https://github.com/elastic/kibana/issues/206437)
This PR migrates `explore page` from `styled-components` to
`@emotion/styled`.
In the process I also convert the kbn/ui-theme json tokens to euiTheme
counterparts.



https://github.com/user-attachments/assets/f1702c0d-78f7-4e17-a8d6-c7c9ae19d3b7



### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2025-01-30 13:24:05 +01:00
Tre
ce1904533d
[Scout] add maps test (#204607)
## Summary

Add MapsPage to the scout core to be re-used by others.
Add rudimentary docs to show how to run these tests.
Added a `waitForRender` method.
Add test id.

---------

Co-authored-by: Robert Oskamp <traeluki@gmail.com>
Co-authored-by: Nick Partridge <nicholas.partridge@elastic.co>
Co-authored-by: Nick Partridge <nick.ryan.partridge@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
2025-01-30 11:55:35 +00:00
Stratoula Kalafateli
a63781a5ea
[ES|QL] Apply the timerange to the fields fetch in the editor (#208490)
## Summary

Provides the timerange in the fetch fields api. This:

- fixes the bug where the fields are not suggested when there are system
named params such as `_tstart` and `_tend`
- makes it more performant as now it checks for fields in the selected
timerange making it more performant (especially for data in the frozen
tiers)

<img width="834" alt="image"
src="https://github.com/user-attachments/assets/ec0e6f87-3149-4a3f-a620-d7eab6a852a2"
/>
2025-01-30 11:14:25 +01:00
Giorgos Bamparopoulos
17e3f82bd2
Mark xpack.infra.sources.default.fields.message as deprecated in the docs (#208683)
Mark `xpack.infra.sources.default.fields.message` as deprecated in the
docs

Closes https://github.com/elastic/kibana/issues/208679
2025-01-30 11:36:45 +02:00
Alexey Antonov
50272d651b
fix: [Security:Manage:EntityAnalytics:EntityStore] missing labels on Entity store page (#207400)
Closes: #205826
Closes: #205825

**Description**
Import entities input file upload box is missing label on entity store
page.

**Preconditions**
Security -> Manage ->Entity store page

**Steps to reproduce**
1. Open Entity store page
2. Run axe-core through the page

**Changes made**
1. added required `aria-label` attributes for mentioned places

**Screen**

<img width="1223" alt="image"
src="https://github.com/user-attachments/assets/29a76c2c-76f6-4bdc-b3f4-4feefc4009eb"
/>
2025-01-30 11:05:22 +02:00
Dzmitry Lemechko
cf7debdfa3
[scout] add script to discover playwright configs in repo (#208733)
## Summary

Adding script to discover Scout playwright tests in Kibana repo, will be
used to build CI pipeline step (running tests per plugin in a separate
worker for the start). We can also consider using it to decide if code
change should trigger only specific plugin tests to run.

Usage:
```
node scripts/scout.js discover-playwright-configs --searchPaths x-pack/platform/plugins/private/discover_enhanced
```

Output:
```
 info Searching for playwright config files in the following paths:
 info - x-pack/platform/plugins/private/discover_enhanced/**/ui_tests/{playwright.config.ts,parallel.playwright.config.ts}
 info
 info Discovered playwright config files in '1' plugins
 info [discover_enhanced] plugin:
 info - x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts
 info - x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts
```

More usage examples:
```
node scripts/scout.js discover-playwright-configs // by default will search in ['src/platform/plugins', 'x-pack/**/plugins'] and return all existing ones
node scripts/scout.js discover-playwright-configs --searchPaths x-pack/platform // platform ones under x-pack
node scripts/scout.js discover-playwright-configs --searchPaths x-pack/** // all under x-pack
```
2025-01-30 09:42:15 +01:00
Ania Kowalska
a376e9b169
refactor RowHeightSettings component to EUI layout (#203606) 2025-01-30 09:12:02 +01:00
Kibana Machine
6607934859
[api-docs] 2025-01-30 Daily api_docs build (#208875)
Generated by
https://buildkite.com/elastic/kibana-api-docs-daily/builds/968
2025-01-30 08:16:08 +01:00
Pierre Gayvallet
32175454c1
[inference] fix tool definition for rainbow-sprinkles (#208823)
## Summary

Port the tool definition hack that was done for bedrock to be used for
`rainbow-sprinkles` too.
2025-01-30 07:58:20 +01:00
Arturo Castillo Delgado
0c18b44621
Upgrade EUI to v99.1.0 (#207724)
`99.0.0-borealis.1`  `99.1.0-borealis.0`

[Questions? Please see our Kibana upgrade
FAQ.](https://github.com/elastic/eui/blob/main/wiki/eui-team-processes/upgrading-kibana.md#faq-for-kibana-teams)

---

# `@elastic/eui`

## [`v99.1.0`](https://github.com/elastic/eui/releases/v99.1.0)

- Updated `EuiColorPalettePicker` - adds `append` to
`EuiColorPalettePickerPaletteProps` to support appending custom content
to the title ([#8208](https://github.com/elastic/eui/pull/8208))
- Updated font-weight and font-size of `EuiBetaBadge`s to improve
legibility ([#8255](https://github.com/elastic/eui/pull/8255))
- Added suppport for `titleColor` variant `warning` on `EuiStat`
([#8278](https://github.com/elastic/eui/pull/8278))

# `@elastic/eui-theme-borealis`

## `v0.0.9`

- [Visual Refresh] Support new theme tokens on JSON exports
([#8277](https://github.com/elastic/eui/pull/8277))
- Remap few colors for EuiLoadingSpinner and EuiLoadingChart
([#8276](https://github.com/elastic/eui/pull/8276))
- [Visual Refresh] Update floating border on EuiPanel
([#8270](https://github.com/elastic/eui/pull/8270))

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: nickofthyme <nicholas.partridge@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2025-01-30 06:02:24 +01:00
Efe Gürkan YALAMAN
a172cb5691
[FTR][Synonyms UI] Add Synonyms overview FTRs (#208723)
## Summary

Adds FTR tests for synonyms

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.


- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2025-01-30 01:46:08 +01:00
Patryk Kopyciński
1935cedeaa
[Security Assistant] Switch to use default inference endpoint (#208668)
## Summary

In 8.17 we have introduced `semantic_text`
https://github.com/elastic/kibana/pull/197007 which required dedicated
inference endpoint.
As we now have default `.elser-2-elasticsearch` inference endpoint
available we want to migrate it out, but it's not possible to just
override `inference_id` mapping for the Knowledge Base data stream, so
instead we decided to first update the mapping by adding
`search_inference_id` pointing to the `.elser-2-elasticsearch` (to make
sure the data is queryable without the dedicated endpoint). Then we
update the Data Stream mapping to use the default endpoint and after
that we rollover the DS index to make sure new index is created and new
inference endpoint is used for new Knowledge Base data ingestion.

Will add testing steps soon
2025-01-30 01:00:33 +01:00
Julia Rechkunova
8ffb2ff628
[Discover] In-table search (#206454)
- Closes https://github.com/elastic/kibana/issues/192360

## Summary

The default browser Find-in-page does not work great with the grid
virtualization and our pagination and it can only find matches in rows
which are currently displayed.

This PR adds in-table search support to the grid so users can find
matches in all grid rows (up to `500` sample docs/rows by default) and
jump between them with "Previous"/"Next" buttons.

![Jan-24-2025
22-03-54](https://github.com/user-attachments/assets/95b31fb8-4740-4c5f-ba91-8e1c19066e02)

The implementation is extracted in a new package
`@kbn/data-grid-in-table-search`. This would allow to use in-table
search with `EuiDataGrid` on other pages of Kibana too.

`Cmd+F` shortcut is overridden when one of grid elements is in focus
otherwise we keep the browser default behaviour.

### Checklist

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
2025-01-29 19:52:07 -04:00
Abhishek Bhatia
a060baee83
[Entity Analytics] [Entity Store] [API] Changes to support event.ingested as a configurable timestamp field for init and enable endpoints (#208201)
## Summary

This PR introduces support for configuring the `timestamp` field for
entity store enablement.

By default, the `timestamp` field is set to `@timestamp`, but users can
opt to use `event.ingested` or another preferred value based on their
requirements.


### Entity Store API changes

#### Entity Store enable

```
POST kbn:/api/entity_store/enable
{
  "timestampField": "event.ingested"
}
```

#### Result 

```
{
  "engines": [
    {
      "status": "started",
      "type": "user",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    },
    {
      "status": "started",
      "type": "host",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    },
    {
      "status": "started",
      "type": "universal",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    }
  ],
  "status": "running"
}
```

```
POST kbn:/api/entity_store/enable
{
}
```
#### Result

```
{
  "engines": [
    {
      "status": "started",
      "type": "host",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    },
    {
      "status": "started",
      "type": "user",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    },
    {
      "status": "started",
      "type": "universal",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    }
  ],
  "status": "running"
}
```

Different entity types can have distinct `timestampField` values, as
described below:

For instance, the `host` entity uses `event.ingested` as its
`timestampField`, while other entities default to the `@timestamp`
field.

```
POST kbn:/api/entity_store/engines/host/init
{
  "timestampField": "event.ingested"
}

POST kbn:/api/entity_store/engines/user/init
{
}
```

#### Result

```
{
  "engines": [
    {
      "status": "started",
      "type": "universal",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    },
    {
      "status": "started",
      "type": "host",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "event.ingested"
    },
    {
      "status": "started",
      "type": "user",
      "indexPattern": "",
      "filter": "",
      "fieldHistoryLength": 10,
      "timestampField": "@timestamp"
    }
  ],
  "status": "running"
}
```


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)


### Testing steps

- Check out this PR branch.
- Start Kibana.
- Avoid enabling the Entity Store from the UI initially. Instead, enable
it via the API by following the steps outlined below.

```
DELETE kbn:/api/entity_store/engines/user
DELETE kbn:/api/entity_store/engines/host

GET kbn:/api/entity_store/status

 All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
  "timestampField": "event.ingested"
}

# All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
}

# Different entity with different timestamp Field
POST kbn:/api/entity_store/engines/host/init
{
  "timestampField": "event.ingested"
}

POST kbn:/api/entity_store/engines/user/init
{
}
```

4. Afterward, you can test the functionality through the UI. First,
clear the entity data from the UI, then start the Entity Store. Note
that the UI currently does not offer a way to configure this parameter.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Mark Hopkin <mark.hopkin@elastic.co>
2025-01-29 16:49:13 -07:00
James Gowdy
70cd3cee7c
[ML] Adds simple flyout based file upload to Search (#206864)
A minimal version of the file upload tool which can be triggered via a
uiAction.
The trigger takes a callback to enable subsequent actions after the
upload. This callback receives information about the upload, the index
and data view created and information about the files:
```
{
  "index": "test9",
  "dataView": {
    "id": "a870ef68-a624-4df1-9d5d-fa62b75dd297",
    "title": ""
  },
  "files": [
    {
      "fileName": "farequote-tiny.csv",
      "docCount": 20,
      "fileFormat": "delimited"
    },
    {
      "fileName": "farequote.csv",
      "docCount": 86275,
      "fileFormat": "delimited"
    }
  ]
}
```

If `autoAddInference` is set with the name of an inference endpoint
(`autoAddInference: '.elser-2-elasticsearch'`) the tool with
automatically add a `semantic_text` to the mappings for tika files (pdf,
txt docx)

Currently embedded in the search app's home page and playground. In
playgroubnd, after upload is complete and the flyout closed, the newly
created index will be selected.


https://github.com/user-attachments/assets/0589fa02-fb0e-400b-8e74-1eb9a993c6ba


Multiple files can be uploaded at once. They must be of the same file
format and the mappings cannot clash.


![image](https://github.com/user-attachments/assets/041824ce-3c26-4ce3-9b7a-7974fab3ea73)




![image](https://github.com/user-attachments/assets/a72408b0-7f33-4047-8351-90baa58d56a9)


If more than once file has the same field but they are of different
types, the files are considered incompatible.

<img width="612" alt="image"
src="https://github.com/user-attachments/assets/67307fd0-7d10-4eab-9e72-df133ebddcfe"
/>

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2025-01-30 00:35:12 +01:00
elastic-renovate-prod[bot]
6aea965ba7
Update docker.elastic.co/wolfi/chainguard-base:latest Docker digest to d04ee51 (main) (#208797)
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `bd40170` ->
`d04ee51` |

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->

Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
2025-01-29 16:54:03 -06:00
Rodney Norris
ea32413c08
[Search] Remove webcrawler endpoints (#208827)
## Summary

- Removed webcrawler endpoints
- Removes unused cloud_health endpoint
- Removes ent search node request handler

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2025-01-29 16:19:56 -06:00
Hannah Mudge
a149a1147b
[kbn-grid-layout] Allow panel to extend past viewport on resize (#208828)
Closes https://github.com/elastic/kibana/issues/208736

## Summary

This PR ensures that serverless / solution view Dashboards can be
resized to the full width.

The problem was, we had logic for "locking" the edge of the panel so
that it could not be resized past the width of the grid - however, we
did this by manually calculating the width of the grid via the
`runtimeSettings`, which did not account for the left-navigation added
on solution views (i.e. it relied on the left position of the grid being
zero in order to work). I **could** have fixed this while keeping the
original behaviour by locking it to the right edge of the grid -
however, this would require a call to `getBoundingRect()` on the panel's
parent component, which could be a slowdown point. See
bc96225a67 for this solution, if you're
curious.

Ultimately, we decided that the "max" width behaviour wasn't worth this
extra complexity, and so we are now exclusively using the mouse position
for calculating the width of the resized panel - therefore, this will
work regardless of whether the grid element has something to the left of
it (such as serverless navigation).

| Before | After |
|--------|--------|
| ![Jan-29-2025
12-45-28](https://github.com/user-attachments/assets/ec5388cc-b3aa-44b2-9b6c-6fb01d800bf5)
| ![Jan-29-2025
12-46-31](https://github.com/user-attachments/assets/efc0839b-e632-4160-bbff-97779971acc9)
|


**Before**


https://github.com/user-attachments/assets/019c7271-390a-4290-9151-f7ea9a5458b6


**After**


https://github.com/user-attachments/assets/08c38f36-66fe-4d53-9e43-87be99e94db6




### Checklist

- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2025-01-29 15:02:37 -07:00
Efe Gürkan YALAMAN
2cb7bea5f3
[Synonyms UI] Search synonyms rule flyout (#208564)
## Summary

Adds search synonym rule flyout.
Adds endpoints and hooks for synonym rule management.


https://github.com/user-attachments/assets/e43b4a40-6452-4cfd-921f-2bde1219f219



### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2025-01-29 22:55:52 +01:00
Bailey Cash
ee14e50fe1
[SLO]: Limit initial data backfill for SLO for serverless (#208790)
Resolves #188428 

## Summary

- Add callout in serverless to indicate that initial data backfill is
limited to 7 days.
- If prevent data backfill is checked, transform will still use the SLO
delay for the filter range.
- If serverless and prevent data backfill is not checked, data backfill
is hard coded to 7 days, the lowest rolling time window option. Time
window will not be rounded down.

<img width="980" alt="Screenshot 2025-01-29 at 11 18 01 AM"
src="https://github.com/user-attachments/assets/b6481ef9-cc0e-4403-8309-50b6d8f37e70"
/>
2025-01-29 22:37:34 +01:00
Davis McPhee
b53d3990a2
[Saved Queries] Rework saved query privileges (#202863)
## Summary

This PR reworks saved query privileges to rely solely on a single global
`savedQueryManagement` privilege, and eliminates app-specific overrides.
This change simplifies the security model for users, fixes bugginess in
the saved query management UI, and reduces code complexity associated
with maintaining two separate security mechanisms (app-specific
overrides and global saved query management privileges).

### Background

Saved queries allow users to store a combination of KQL or Lucene
queries, filters, and time filters to use across various applications in
Kibana. Access to saved query saved objects are currently granted by the
following feature privileges:
```json
[
  "feature_discover.all",
  "feature_dashboard.all",
  "feature_savedQueryManagement.all",
  "feature_maps.all",
  "feature_savedObjectsManagement.all",
  "feature_visualize.all"
]
```

There is also a saved query management UI within the Unified Search bar
shared by applications across Kibana:
<img
src="https://github.com/user-attachments/assets/e4a7539b-3dd4-4d47-9ff8-205281ef50e3"
width="500" />

The way access to this UI is managed in Kibana is currently confusing
and buggy:
- If a user has `feature_discover.all` and `feature_dashboard.all` they
will be able to load and save queries in Discover and Dashboard.
- If a user has `feature_discover.all` and `feature_dashboard.read` they
will be able to load queries in both Discover and Dashboard, but only
save queries in Discover (even though they have write access to the SO,
and API access). Instead they have to navigate to Discover to save a
query before navigating back to Dashboard to load it, making for a
confusing and frustrating UX.
- Access to the UI is even more confusing in apps not listed in the
above feature privileges (e.g. alerting, SLOs). Some of them chose to
check one of the above feature privileges, meaning users who otherwise
should have saved query access won't see the management UI if they don't
also have the exact feature privilege being checked. Other apps just
always show the management UI, leading to bugs and failures when users
without one of the above feature privileges attempt to save queries.

### Existing improvements

In v8.11.0, we introduced a new ["Saved Query
Management"](https://github.com/elastic/kibana/pull/166937) privilege,
allowing users to access saved queries across all of Kibana with a
single global privilege:
<img
src="https://github.com/user-attachments/assets/ccbe79a4-bd0b-4ed6-89c9-117cc1f99ee2"
width="600" />


When this privilege is added to a role, it solves the
`feature_discover.all` and `feature_dashboard.read` issue mentioned
above. However, it does not fix any of the mentioned issues for roles
without the new privilege. We have so far postponed further improvements
to avoid a breaking change.

### Approach

To fully resolve these issues and migrate to a single global privilege,
these changes have been made:
- Remove saved query SO access from all application feature privileges
and instead only allow access through the global saved query management
privilege.
- Stop relying on application feature privileges for toggling the saved
query management UI, and instead rely on the global privilege.

To implement this with minimal breaking changes, we've used the Kibana
privilege migration framework. This allows us to seamlessly migrate
existing roles containing feature privileges that currently provide
access to saved queries, ensuring they are assigned the global saved
query management privilege on upgrade.

As a result, we had to deprecate the following feature privileges,
replacing them with V2 privileges without saved query SO access:
```json
[
  "feature_discover.all",
  "feature_dashboard.all",
  "feature_maps.all",
  "feature_visualize.all"
]
```

Each area of code that currently relies on any of these feature
privileges had to be updated to instead access `feature_X_V2` instead
(as well as future code).

This PR still introduces a minor breaking change, since users who have
`feature_discover.all` and `feature_dashboard.read` are now able to save
queries in Dashboard after upgrade, but we believe this is a better UX
(and likely the expected one) and worth a small breaking change.

### Testing
- All existing privileges should continue to work as they do now,
including deprecated V1 feature privileges and customized serverless
privileges. There should be no changes for existing user roles apart
from the minor breaking change outlined above.
- Check that code changes in your area don't introduce breaking changes
to existing behaviour. Many of the changes are just updating client UI
capabilities code from `feature.privilege` to `feature_v2.privilege`,
which is backward compatible.
- The `savedQueryManagement` feature should now globally control access
to saved query management in Unified Search for all new user roles.
Regardless of privileges for Discover, Dashboard, Maps, or Visualize,
new user roles should follow this behaviour:
- If `savedQueryManagement` is `none`, the user cannot see or access the
saved query management UI or APIs.
- If `savedQueryManagement` is `read`, the user can load queries from
the UI and access read APIs, but cannot save queries from the UI or make
changes to queries through APIs.
- If `savedQueryManagement` is `all`, the user can both load and save
queries from the UI and through APIs.

### Checklist

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

### Identify risks

This PR risks introducing unintended breaking changes to user privileges
related to saved queries if the deprecated features have not been
properly migrated, and users could gain or lose access to saved query
management on upgrade. This would be bad if it happened, but not overly
severe since it wouldn't grant them access to any ES data they couldn't
previously access (only query saved objects). We have automated testing
in place to help ensure features have been migrated correctly, but the
scope of these changes are broad and touch many places in the codebase.

Additionally, the UI capabilities types are not very strict, and are
referenced with string paths in many places, which makes changing them
riskier than changing strictly typed code. A combination of regex
searches and temporarily modifying the `Capabilities` type to cause type
errors for deprecated privileges was used to identify references in
code. Reviewers should consider if there are any other ways that UI
capabilities can be referenced which were not addressed in this PR.

Our automated tests already help mitigate the risk, but it's important
that code owners thoroughly review the changes in their area and
consider if they could have unintended consequences. The Platform
Security team should also review this PR thoroughly, especially since
some changes were made to platform code around privilege handling. The
Data Discovery team will also manually test the behaviour when upgrading
existing user roles with deprecated feature privileges as part of 9.0
upgrade testing.

---------

Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: “jeramysoucy” <jeramy.soucy@elastic.co>
2025-01-29 17:34:58 -04:00
Tiago Costa
f3da71672c
skip flaky suite (#207423) 2025-01-29 21:19:55 +00:00
Tiago Costa
6a17202448
skip flaky suite (#207241) 2025-01-29 21:19:16 +00:00
Quynh Nguyen (Quinn)
201169a04a
[ML] Add new View job detail flyouts for Anomaly detection and Data Frame Analytics (#207141)
## Summary

This PR adds new View job detail flyout for Anomaly detection and Data
Frame Analytics

**For Anomaly detection jobs:** 

- New options are added when clicking on job's name (Remove from page,
View datafeed charts, Navigate to Single Metric Viewer/Anomaly Explorer)

<img width="553" alt="Screenshot 2025-01-24 at 15 02 10"
src="https://github.com/user-attachments/assets/207fa601-b04e-4ab6-b808-e0e420b40584"
/>

- If there's only one job, the remove from {page} is disabled
 
<img width="553" alt="Screenshot 2025-01-24 at 15 02 01"
src="https://github.com/user-attachments/assets/6b2e75a6-e508-4a7d-8e07-dec9b22fc67a"
/>


https://github.com/user-attachments/assets/1a4f0e8f-da15-4e8c-86bd-48045f9144f9




**For Anomaly detection groups:**

- Remove job option is not shown



https://github.com/user-attachments/assets/1976f7dc-8cfe-4f94-975e-233f0225e15b



https://github.com/user-attachments/assets/3381a4f2-ec99-4848-b2fe-9df456306523


**For Data frame analytics jobs:**


https://github.com/user-attachments/assets/7e067ac2-4eda-44b3-bc63-a5901912350f



### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2025-01-29 21:45:31 +01:00
Bharat Pasupula
3430ab8246
[Automatic Import] Remove Tech preview badge for GA (#208523) 2025-01-29 21:40:56 +01:00
Tiago Vila Verde
5b22aa9b66
[Entity Analytics][Entity Store] Add transform config options to the API (#208062)
## Summary


This PR adds the following parameters to the `INIT` engine API:

* `frequency`: the transform run frequency
* `timeout`: the timeout for the initial creation of the transform
* `docsPerSecond`: transform throttling option. See
[here](https://arc.net/l/quote/vxcmfnhh)
* `delay`: The transform delay duration. See
[here](https://arc.net/l/quote/mzvaexhv)
Coming soon


In addition, the PR adds these fields to the Saved Object with the
engine descriptor, as well as providing a migration with the appropriate
backfilling.

Finally, there are some utility function that were/are helpful in
working with objects.

## How to test

*NOTE*: Always make sure the security default data view exists. Easiest
way it to just navigate to some Security UI.

### Checking the new defaults

1. Initialize an engine via dev tools by calling: `POST
kbn:/api/entity_store/engines/<entity_type>/init {}`
2. Call `GET kbn:/api/entity_store/status`. This response should now
contain all the default optional values.

### Observing the parameters are being applied

1. Initialize an engine via the API. This time pass any of the `timeout,
frequency, delay and docsPerSecond` options in the request body.
2. Once the `status` changes to `started`, query the respective
transform: `GET
_transform/entities-v1-latest-security_<entity_type>_default`
3. Check that the parameters have been applied to the transform

### Checking Saved Object Migration

1. Check out `main`.
2. Initialize the store.
3. Query `GET kbn:/api/entity_store/status`. Note down the fields in the
engine object.
4. Check out this branch.
5. Restart kibana.
6. Query `GET kbn:/api/entity_store/status` again. Observe the new
fields have been added and backfilled

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2025-01-29 20:31:47 +00:00