## Summary
1. Show all packages owned by a specific team
```
node scripts/dependency_ownership -o <owner>
```
2. Identify owners of a specific dependency
```
node scripts/dependency_ownership -d <dependency>
```
3. List dependencies without an owner
```
node scripts/dependency_ownership --missing-owner
```
4. Generate a full dependency ownership report
```
node scripts/dependency_ownership
```
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
__Closes: https://github.com/elastic/kibana/issues/196767__
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [@redocly/cli](https://togithub.com/Redocly/redocly-cli) |
devDependencies | patch | [`^1.25.12` ->
`^1.25.13`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.25.14/1.25.13)
| `1.25.14` |
| [@redocly/cli](https://togithub.com/Redocly/redocly-cli) |
dependencies | patch | [`^1.25.12` ->
`^1.25.13`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.25.14/1.25.13)
| `1.25.14` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDphbGwtb3BlbiIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[@launchdarkly/node-server-sdk](https://togithub.com/launchdarkly/js-core/tree/main/packages/sdk/server-node)
([source](https://togithub.com/launchdarkly/js-core)) | dependencies |
patch | [`^9.7.1` ->
`^9.7.2`](https://renovatebot.com/diffs/npm/@launchdarkly%2fnode-server-sdk/9.7.1/9.7.2)
|
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJUZWFtOlNlY3VyaXR5IiwiYmFja3BvcnQ6cHJldi1taW5vciIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
## Summary
This PR introduces a new package, **kbn-scout**, designed to streamline
the setup and execution of Playwright tests for Kibana. The `kbn-scout`
package consolidates server management and testing capabilities by
wrapping both the Kibana/Elasticsearch server launcher and the
Playwright test runner. It includes:
- Test and worker-scoped fixtures for reliable setup across test suites
- Page objects combined into the fixture for Kibana UI interactions
- Configurations for seamless test execution in both local and CI
environments (`Cloud` execution is out of scope)
- This package aims to simplify test setup and enhance modularity,
making it easier to create, run, and maintain deployment-agnostic tests,
that are located in the plugin they actually test.
Tests example is available in `x-pack/plugins/discover_enhanced` plugin
under `ui_tests` folder
How to run:
1) As a single script (servers + tests):
```
node scripts/scout_test.js --config=x-pack/plugins/discover_enhanced/ui_tests/playwright.config.ts --serverless=es
```
2) Start servers first
```
node scripts/scout_start_servers.js --serverless=es
```
then run tests:
```
npx playwright test --config=x-pack/plugins/discover_enhanced/ui_tests/playwright.config.ts
```
if you have Playwright plugin in IDEA, just use it to run tests files
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR upgrades cypress to `13.15.1`. This is needed to fix an very
annoying issue where Cypress tests hangs when ran locally and when the
window is not focused
See Cypress issue here:
https://github.com/cypress-io/cypress/issues/28392
---------
Co-authored-by: PhilippeOberti <philippe.oberti@elastic.co>
This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [chromedriver](https://togithub.com/giggio/node-chromedriver) |
devDependencies | major | [`^130.0.4` ->
`^131.0.0`](https://renovatebot.com/diffs/npm/chromedriver/130.0.4/131.0.0)
| `131.0.1` |
---
### Release Notes
<details>
<summary>giggio/node-chromedriver (chromedriver)</summary>
###
[`v131.0.0`](https://togithub.com/giggio/node-chromedriver/compare/130.0.4...131.0.0)
[Compare
Source](https://togithub.com/giggio/node-chromedriver/compare/130.0.4...131.0.0)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Co-authored-by: Jon <jon@elastic.co>
Creates the Streams app plugin, which renders UI for managing streams
(see https://github.com/elastic/kibana/pull/198713).
Additional changes in this PR:
- The menus were updated to conditionally add a link to the Streams app.
The Streams plugin itself returns a status$ observable which signals if
Streams have been enabled. This value is used to conditionally render
the link in the various flavors of menus.
- There's a small change in the ES types to allow for ordered params in
ES|QL (vs named params)
- `@kbn/server-route-repository` was updated to be able to override
`access` (instead of only inferring it from the endpoint name).
Additionally, we now allow all route options by default.
- `@kbn/typed-react-router-config` now also exports a `useBreadcrumbs`.
This was copied over from the APM implementation.
- the signature of the `esql` method in
`ObservabilityElasticsearchClient` was updated to separate processing
options from options that are sent over to the _query endpoint.
---------
Co-authored-by: Chris Cowan <chris@elastic.co>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
[dependency-cruiser](https://github.com/sverweij/dependency-cruiser/tree/main)
is used for building dependency graph.
### Show all dependencies for a specific package/plugin or directory
#### Run for all plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins -o ./tmp/deps-result-all.json
```
#### Run for single plugin
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -o ./tmp/deps-result-single.json
```
#### Run for multiple plugins
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution x-pack/plugins/security -o ./tmp/deps-result-multiple.json
```
#### Run for `x-pack/packages`
```bash
bash scripts/dependency_usage.sh -p x-pack/packages -o ./tmp/deps-packages-1.json
```
#### Run for `packages`
```bash
bash scripts/dependency_usage.sh -p packages -o ./tmp/deps-packages-2.json
```
#### Benchmark
| Analysis | Real Time | User Time | Sys Time |
|-----------------------|-------------|-------------|------------|
| All plugins | 7m 21.126s | 7m 53.099s | 20.581s |
| Single plugin | 31.360s | 45.352s | 2.208s |
| Multiple plugins | 36.403s | 50.563s | 2.814s |
| x-pack/packages | 6.638s | 12.646s | 0.654s |
| packages | 25.744s | 39.073s | 2.191s |
#### Show all packages/plugins within a directory that use a specific
dependency
```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins/security_solution
```
---
#### Show all packages/plugins within a directory grouped by code owner
```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins -g owner
```
---
#### Group by code owner with adjustable collapse depth for fine-grained
grouping
**Fine-grained grouping**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 4
```
**Collapsed grouping**: groups the results under a higher-level owner
(e.g., `security_solution` as a single group).
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 1
```
---
#### Show all dependencies matching a pattern (e.g., `react-*`) within a
package
```bash
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -d 'react-*' -o ./tmp/result.json
```
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Related: https://github.com/elastic/kibana/issues/196767__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
- create `_search` endpoint to discover entities with esql queries. It
currently reads sources of the provided `type` from
`kibana_entity_definitions` index. Run this query to insert a
definition:
```
POST kibana_entity_definitions/_doc
{
"entity_type": "service",
"index_patterns": ["remote_cluster:logs-*"],
"metadata_fields": [],
"identity_fields": ["service.name"],
"filters": [],
"timestamp_field": "@timestamp"
}
```
By default `_search` will look at data in the last 5m. The lookup period
can be overriden by providing `start`/`end` parameters in ISO format. It
also accepts a `limit` to specify the number of entities returned which
defaults to 10
```
POST kbn:/internal/entities/v2/_search
{
"type": "service",
"start": "2024-11-19T20:40:00.000Z",
"end": "2024-11-19T20:50:00.000Z",
"limit": 20
}
```
- create `_search/preview` endpoint to preview output of entity sources
without persisting them
- create UI to preview results of an entity definition at
`/app/entity_manager`. The application is living in its own plugin at
`observability_solution/entity_manager_app`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Milton Hultgren <miltonhultgren@gmail.com>
## Summary
This PR is part of the Kibana Sustainable Architecture effort.
The goal is to start categorising Kibana packages into _generic
platform_ (`group: "platform"`) vs _solution-specific_.
```
group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'
```
Uncategorised modules are considered to be `group: 'common', visibility:
'shared'` by default.
We want to prevent code from solution A to depend on code from solution
B.
Thus, the rules are pretty simple:
* Modules can only depend on:
* Modules in the same group
* OR modules with 'shared' visibility
* Modules in `'observability', 'security', 'search'` groups are
mandatorily `visibility: "private"`.
Long term, the goal is to re-organise packages into dedicated folders,
e.g.:
```
x-pack/platform/plugins/private
x-pack/observability/packages
```
For this first wave, we have categorised packages that seem
"straightforward":
* Any packages that have:
* at least one dependant module
* all dependants belong to the same group
* Categorise all Core packages:
* `@kbn/core-...-internal` => _platform/private_
* everything else => _platform/shared_
* Categorise as _platform/shared_ those packages that:
* Have at least one dependant in the _platform_ group.
* Don't have any `devOnly: true` dependants.
### What we ask from you, as CODEOWNERS of the _package manifests_, is
that you confirm that the categorisation is correct:
* `group: "platform", visibility: "private"` if it's a package that
should only be used from platform code, not from any solution code. It
will be loaded systematically in all serverless flavors, but solution
plugins and packages won't be able to `import` from it.
* `group: "platform", visibility: "shared"` if it's a package that can
be consumed by both platform and solutions code. It will be loaded
systematically in all serverless flavors, and anybody can import / use
code from it.
* `group: "observability" | "security" | "search", visibility:
"private"` if it's a package that is intented to be used exclusively
from a given solution. It won't be accessible nor loaded from other
solutions nor platform code.
Please refer to
[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)
for any related questions.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [terser](https://terser.org)
([source](https://togithub.com/terser/terser)) | devDependencies | minor
| [`^5.34.0` ->
`^5.36.0`](https://renovatebot.com/diffs/npm/terser/5.34.1/5.36.0) |
---
### Release Notes
<details>
<summary>terser/terser (terser)</summary>
###
[`v5.36.0`](https://togithub.com/terser/terser/blob/HEAD/CHANGELOG.md#v5360)
[Compare
Source](https://togithub.com/terser/terser/compare/v5.35.0...v5.36.0)
- Support import attributes `with` syntax
###
[`v5.35.0`](https://togithub.com/terser/terser/blob/HEAD/CHANGELOG.md#v5350)
[Compare
Source](https://togithub.com/terser/terser/compare/v5.34.1...v5.35.0)
- Ensure parent directory exists when using --output on CLI
([#​1530](https://togithub.com/terser/terser/issues/1530))
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
---------
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Co-authored-by: Brad White <brad.white@elastic.co>
## Summary
This PR introduces the first internal version of the new theme
`Borealis` and ensures that:
- themes can be switched between "Amsterdam" and "Borealis"
- theme-specific Sass files are available and can be loaded with
`KBN_OPTIMIZER_THEMES=experimental`
- legacy JSON variable usage accounts for both themes
- static template styles account for both themes
## Running locally
```yml
// kibana.dev.yml or kibana.yml
uiSettings.experimental.themeSwitcherEnabled: true
```
Start kibana
```
KBN_OPTIMIZER_THEMES='v8light,v8dark,borealislight,borealisdark' yarn start
or
KBN_OPTIMIZER_THEMES=experimental yarn start
```
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
---------
Co-authored-by: Tomasz Kajtoch <tomasz.kajtoch@elastic.co>
## Summary
Close https://github.com/elastic/kibana/issues/193473
Close https://github.com/elastic/kibana/issues/193474
This PR utilize the documentation packages that are build via the tool
introduced by https://github.com/elastic/kibana/pull/193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.
Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).
For more information, please refer to the self-review.
## General architecture
<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">
## What this PR does
Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).
- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc
(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)
## What is NOT included in this PR:
- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
- installation
- utilization as tool
- FTR tests: this is somewhat blocked by the same things we need to
figure out for https://github.com/elastic/kibana-team/issues/1271
## Screenshots
### Installation from o11y assistant configuration page
<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">
### Example of output
#### Without product documentation installed
<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">
#### With product documentation installed
<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
close https://github.com/elastic/kibana/issues/194165
close https://github.com/elastic/kibana-team/issues/1245
### User-facing
<img width="1680" alt="image"
src="https://github.com/user-attachments/assets/6df4ee9f-1b4d-404c-a764-592998a1d430">
This PRs adds a new tab in the editor history component. You can star
your query from the history and then you will see it in the Starred
list. The started queries are scoped to a user and a space.
### Server
To allow starring ESQL query, this PR extends [favorites
service](https://github.com/elastic/kibana/pull/189285) with ability to
store metadata in addition to an id. To make metadata strict and in
future to support proper metadata migrations if needed, metadata needs
to be defined as schema:
```
plugins.contentManagement.favorites.registerFavoriteType('esql_query', {
typeMetadataSchema: schema.object({ query: schema.string(), timeRange:...., etc... }),
})
```
Notable changes:
- Add support for registering a favorite type and a schema for favorite
type metadata. Previosly the `dashboard` type was the only supported
type and was hardcoded
- Add `favoriteMetadata` property to a saved object mapping and make it
`enabled:false` we don't want to index it, but just want to store
metadata in addition to an id.
[code](https://github.com/elastic/kibana/pull/198362/files#diff-d1a39e36f1de11a1110520d7607e6aee7d506c76626993842cb58db012b760a2R74-R87)
- Add a 100 favorite items limit (per type per space per user). Just do
it for sanity to prevent too large objects due to metadata stored in
addtion to ids.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Closes https://github.com/elastic/kibana/issues/199361
While investigating, I found that fetching DOM element with id
`app-fixed-viewport` is a common pattern. I created the hook
`useAppFixedViewport` to consolidate this logic into a single location.
The hook only performs the DOM look-up on first render and then avoids
the DOM look-up on each additional render.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
- Exposes the interfaces that define the format of the response of the
`/api/status` endpoint.
- Moves them from `@kbn/core-status-common-internal` to
`@kbn/core-status-common`.
- Removes the former package, as it no longer contains anything.
- Fixes some of the illegal dependencies uncovered by
https://github.com/elastic/kibana/pull/199630.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Split up observability-utils package in browser, common, server. Also
made a small change to `withSpan` to automatically log operation times
when the debug level for the logger is enabled.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR introduces the new experimental "Streams" plugin into the Kibana
project. The Streams project aims to simplify workflows around dealing
with messy logs in Elasticsearch. Our current offering is either
extremely opinionated with integrations or leaves the user alone with
the high flexibility of Elasticsearch concepts like index templates,
component templates and so on, which make it challenging to configure
everything correctly for good performance and controlling search speed
and cost.
### Scope of PR
- Provides an API for the user to "enable" the streams framework which
creates the "root" entity `logs` with all the backing Elasticsearch
assets
- Provides an API for the user to "fork" a stream
- Provides an API for the user to "read" a stream and all of it's
Elasticsearch assets.
- Provides an API for the user to upsert a stream (and implicitly child
streams that are mentioned)
- Part of this API is placing grok and disscect processing steps as well
as fields to the mapping
- Implements the Stream Naming Schema (SNS) which uses dots to express
the index patterns and stream IDs. Example: `logs.nginx.errors`
- The APIs will fully manage the `index_template`, `component_template`,
and `ingest_pipelines`.
### Out of scope
- Integration tests (coming in a follow-up)
### Reviewer Notes
- I haven't implemented tests beyond a unit test for converting the
filter conditions to Painless. I wanted to get a PR up so we can start
iterating on the interface and functionality before we invest in
testing.
- You might need to add `server.versioned.versionResolution: oldest` to
your `config/kibana.dev.yaml` to play with the requests below in the
Kibana "Dev console".
### Example API Calls
Enable the root stream (and set the mapping for the internal `.streams`
index)
```
POST kbn:/api/streams/_enable
```
Read the root entity "logs"
```
GET kbn:/api/streams/logs
```
Fork the "root" entity "logs" and create "logs.nginx" based on a
condition
```
POST kbn:/api/streams/logs/_fork
{
"stream": {
"id": "logs.nginx",
"children": [],
"processing": [],
"fields": [],
},
"condition": {
"field": "log.logger",
"operator": "eq",
"value": "nginx_proxy"
}
}
```
Fork the entity "logs.nginx" and create "logs.nginx.errors" based on a
condition
```
POST kbn:/api/streams/logs.nginx/_fork
{
"stream": {
"id": "logs.nginx.error",
"children": [],
"processing": [],
"fields": [],
},
"condition": {
"or": [
{ "field": "log.level", "operator": "eq", "value": "error" },
{ "field": "log.level", "operator": "eq", "value": "ERROR" }
]
}
}
```
Set some processing on a stream and map the generated field
```
PUT kbn:/api/streams/logs.nginx
{
"children": [],
"processing": [
{ "config": { "type": "grok", "patterns": ["^%{IP:ip} – –"], "field": "message" } }
],
"fields": [
{ "name": "ip", "type": "ip" }
],
}
}
```
Field definitions are checked for both descendants and ancestors for
incompatibilities to ensure they stay additive.
If children are defined in the `PUT /api/streams/<name>` API,
sub-streams are created implicitly. If a stream is `PUT`, it's added to
the parent as well with a condition that is never true (can be edited
subsequently).
`POST /api/streams/_resync` can be used to re-sync all streams from
their meta data in case the Elasticsearch objects got messed up by some
external change - not sure whether we want to keep that.
Follow-ups
* API integration tests
* Check read permissions on data streams to determine whether a user is
allowed to read certain streams
---------
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Extracts `IndexAdapter` from `DataStreamAdapter` and
`IndexPatternAdapter` from `DataStreamSpaceAdapter`.
There are no breaking changes for the _data-stream-adapter_ package; the
behavior of both the `DataStreamAdapter` and `DataStreamSpaceAdapter`
remains unchanged.
The new _index-adapter_ package exports `IndexAdapter` and
`IndexPatternAdapter` to manage individual indices without using data
streams.
This is needed for SIEM rule migrations.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Create package for observability solution's dependencies from index
lifecycle management. Previously it relied on the plugin bundle but
thats best avoided with our sustainable architecture efforts.
Part of https://github.com/elastic/kibana-team/issues/1179
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR aims to fix Flaky tests related to agentless detected by
https://github.com/elastic/kibana/issues/189038 and
https://github.com/elastic/kibana/issues/192126 by adding proper
handling of the `waitFor` methods.
It was also detected with
https://github.com/elastic/security-team/issues/10979 that some other
methods were not proper handled by `waitFor`, leading to the assertions
inside those unhandled `waitFor` being skipped by Jest.
This PR also introduces ESLint to enforce proper handling of waitFor
methods in tests files for Fleet and Cloud Security plugins.
Additional note: These changes should also unblock the failing tests on
the [React18 use waitFor with assertion callbacks in place of
waitForNextUpdate](https://github.com/elastic/kibana/pull/195087) PR
**Fleet changes**
- ESLint rule added to enforce handling `waitFor` on React Testing
Library.
- `useSetupTechnology` hook tests reviewed and updated to handle the
waitFor. Fixed issue identified when reviewing the tests.
- step_define_package_policy.test.tsx: Added package policy vars to the
mock to proper handle the use cases
- step_select_hosts.test.tsx: Handled waitFor, identified outdated test
- step_edit_hosts.test.tsx: Handled waitFor, identified outdated test
With the introduction of the ESLint rule other tests were triggering
ESLint errors, I attempted to fix them while retaining the same
intention, let me know if more changes are needed.
**Cloud Security changes**
- ESLint rule added to enforce handling `waitFor` on React Testing
Library.
- Updated cloud security posture version to include agentless global
tags on End to End tests
**@elastic/kibana-operations changes**
- Added
[eslint-plugin-testing-library](https://testing-library.com/docs/ecosystem-eslint-plugin-testing-library/)
an ESLint plugin for Testing Library that helps users to follow best
practices and anticipate common mistakes when writing tests.
- The adoption and enablement of the rules are opt-in.
