This PR adds guidelines for designing Kibana HTTP APIs that are
terraform-provider developer friendly.
fix https://github.com/elastic/kibana/issues/224643
## Summary
Kibana doesn't have specific guidelines for designing HTTP APIs. With
increasing constraints, it's time to document what was previously tribal
knowledge.
Elasticsearch is far further along this road, and other teams have
compiled their own.
This document serves as guidelines to designing _public_ HTTP APIs that
are suitable for managing with Terraform.
## How to test this (recommended for easier reading)
- pull this PR
- setup
[`docs.elastic.dev`](https://docs.elastic.dev/docs/local-dev-docs-setup)
locally
- run `yarn dev` from `docs.elastic.dev`
- review the docs live!
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
---------
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
* Address https://github.com/elastic/kibana/issues/217145 - Put in place
a check to ensure we're upgrading from Kibana 8.18.0 or newer.
* Address https://github.com/elastic/kibana/issues/220521 - New attempt
at removing the `switchToModelVersionAt` property, inspired on
https://github.com/elastic/kibana/pull/219029.
The previous attempt caused a regression: index meta information started
storing _modelVersions_ that were older than the previously stored ones,
which were defaulting to 10.0.0 for SO types that did NOT define
`modelVersions`.
This was due to the removal of the applyTypeDefaults, which was ensuring
all SOs had the `switchToModelVersionAt` property set.
This flag was then used by
`src/core/packages/saved-objects/base-server-internal/src/model_version/version_map.ts`
to determine whether to use `modelVersions` or the legacy `migrations`
property in order to determine the latest model version for a given
type.
When removing the `switchToModelVersionAt` flag (and its default
backfill), the logic started defaulting to the latest `migrations`
version for those SO types that were not defining any `modelVersion`,
resulting in older versions that those stored in the SO indices. This
caused incident https://elasticco.atlassian.net/browse/INC-3818.
This regression has been shipped in 9.0.0 (the PR was
[backported](https://github.com/elastic/kibana/pull/219329)), so in top
of the cleanup, we now need to address
https://github.com/elastic/kibana/issues/220521 to ensure a smooth
transition _OnPrem => Serverless_.
---------
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR applies **lossless compression** to all SVG and JPG/PNG assets
across Kibana using:
- [`svgo`](https://github.com/svg/svgo) — for optimizing SVGs
- [`image-optimize`](https://www.npmjs.com/package/image-optimize) — for
JPG/PNG compression
‼️**Please scroll to ''Unknown metric groups" accordion to see what's
the gain for your code.**
<img width="542" alt="Screenshot 2025-06-18 at 13 24 20"
src="https://github.com/user-attachments/assets/191afb28-44fc-4551-9026-756a8385c66a"
/>
The goal is to reduce asset size and improve load performance without
compromising visual quality.
This PR achieves a **23 MB** reduction in asset size across all images
bundled in Kibana’s running code—meaning these compressed images
directly impact what ships in Kibana.
Some assets get bundled into chunks due to our bundling strategy but
might not actually be requested at runtime.
Additionally, I ran the same optimization script on the docs assets as a
harmless extra step, but those savings aren’t included in the 23 MB
total.
---
## Why
While working on Emotion rewrites, I noticed some SVGs seemed
unnecessarily heavy. That led to a broader investigation into our image
assets, and it turns out we’re not consistently optimizing them during
development or build.
---
## Notes
- Visual fidelity of optimized assets has been manually verified — no
visible differences
- The optimization is **lossless**, meaning no quality degradation
- Some assets (like large background images) could benefit further from
**lossy compression**
---
## Follow-ups / Ideas
1. **Automate compression in the dev/build pipeline**
- e.g. add `svgo` as a pre-commit or CI step for SVGs
2. **Improve CI reporting**
- Currently, bundle size diffs for images are hidden under "Unknown
metric groups" in the GitHub CI comment. We may want to make these more
visible.
-
3. **Audit large assets manually** — apply lossy compression where
appropriate
4. **Avoid redundant image loading**
- e.g. background images on the login page are loaded again on the space
selector page since they’re bundled twice. I’m working on a separate PR
to address that.
## Snippets I used to apply the compression
```
# Find SVG files
find . -type f -iname "*.svg" \
-not -path "*/node_modules/*" \
-not -path "*/functional/*" > svg-files.txt
# Compress SVGs
while IFS= read -r file; do
svgo "$file"
done < svg-files.txt
```
This snippet has been used for png and jpg, but the example below is for
png:
```
# Find PNG files
find . -type f -iname "*.png \
-not -path "*/node_modules/*" \
-not -path "*/functional/*" > png-files.txt
# Compress PNGs
while IFS= read -r file; do
image-optimize -f jpg "$file"
done < png-files.txt
```
Enhances the performance metrics documentation by explaining how
to use the `meta.description` field for contextualizing render time
events and how to track subsequent page or section loads using
`onPageRefreshStart`.
- Added a section describing the default meaning of render time and how
to provide a custom `description` in the `meta` field of `onPageReady`
for more precise event context.
- Documented the use of `onPageRefreshStart` to distinguish between
initial loads and subsequent refreshes, clarifying that
`meta.isInitialLoad` is set to `false` for refreshes and `true` by
default.
- Included code examples and sample indexed event structures for both
features.
## Summary
Set `KibanaRenderContextProvider` as deprecated and comment that
consumers should use `core.rendering.addContext` instead.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
## Summary
Part of https://github.com/elastic/kibana/issues/201807
**Strategy**:
This PR assumes that `switchToModelVersionAt` has done it’s job and all
SO type owners now use `modelVersions` instead of `migrations`.
It takes the approach of trusting that this is fine™
**Changes in this PR:**
- `switchToModelVersionAt` removed from all registered saved objects, so
their mapping hash had to be updated.
- Implements `globalSwitchToModelVersionAt` directly in `version_map`
- Updates logic to prioritize `modelVersions` over `migrations` in
`getLatestMappingsModelVersion`, that previously relied on
`switchToModelVersionAt` being set
- Updates unit tests to match the refactored logic
- Updates snapshots
- Adapts SO types registered in the **cases**, **SLO** and **encrypted
saved objects** plugins.
- Updates docs
**Risks**:
- Plugin developers disregard the deprecated status of `migrations` and
introduce new versions, which will not be BWC. Saved object type hash
changes will notify the core team for a code owner review. The core team
needs to investigate the related changes and provide feedback.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Introduces a new `security_solution/gen_ai_evals.yml` BuildKite pipeline
for automatically running our Assistant and Attack Discovery evaluation
suites weekly.
### To Run Locally:
Ensure you are authenticated with vault for LLM + LangSmith creds:
> See [internal
docs](https://github.com/elastic/infra/blob/master/docs/vault/README.md#login-with-your-okta)
for setup/login instructions.
Fetch Connectors and LangSmith creds:
> [!NOTE]
> In discussion with @elastic/kibana-operations it was preferred to use
the ci-prod secrets vault, so we cannot self-manage the secrets. To test
this locally though, you can grab the secrets and follow the
instructions in this [paste
bin](https://p.elstc.co/paste/q7k+zYOc#PN0kasw11u2J0XWC2Ls5PMNWreKzKTpgWA1wtsPzeH+).
```
cd x-pack/test/security_solution_api_integration
node scripts/genai/vault/retrieve_secrets.js
```
Navigate to api integration directory, load the env vars, and start
server:
```
cd x-pack/test/security_solution_api_integration
export KIBANA_SECURITY_TESTING_AI_CONNECTORS=$(base64 -w 0 < scripts/genai/vault/connector_config.json) && export KIBANA_SECURITY_TESTING_LANGSMITH_KEY=$(base64 -w 0 < scripts/genai/vault/langsmith_key.txt)
yarn genai_evals:server:ess
```
Then in another terminal, load vars and run the tests:
```
cd x-pack/test/security_solution_api_integration
export KIBANA_SECURITY_TESTING_AI_CONNECTORS=$(base64 -w 0 < scripts/genai/vault/connector_config.json) && export KIBANA_SECURITY_TESTING_LANGSMITH_KEY=$(base64 -w 0 < scripts/genai/vault/langsmith_key.txt)
yarn genai_evals🏃ess
```
### To manually run on BuildKite:
Navigate to
[BuildKite](https://buildkite.com/elastic?filter=ftr-security-solution-gen-ai-evaluations)
and run `ftr-security-solution-gen-ai-evaluations` pipeline.
### To manually run on BuildKite for specific PR:
In `.buildkite/ftr_security_stateful_configs.yml`, temporarily move the
`genai/evaluations/trial_license_complete_tier/configs/ess.config.ts`
line down to the `enabled` section. Will see if we can do this without
requiring a commit. @elastic/kibana-operations is it possible to set a
buildkite env var that can be read in FTR tests when a specific GitHub
label is added to the PR? I.e. can I create a `SecurityGenAI:Run Evals`
label that when added will run this suite as part of the build?
> [!NOTE]
> Currently the connectors secrets only include `gpt-4o` and
`gpt-4o-mini`. Waiting on finalized list w/ credentials from @jamesspi
and @peluja1012 and then we can have ops update using the scripts
included in this PR.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <patryk.kopycinski@elastic.co>
## Summary
Currently, our `requiredPrivileges` structure supports `allRequired` and
`anyRequired` for defining authorization logic. However, there is [a
need to
support](https://github.com/elastic/kibana/pull/205335#issuecomment-2569275302)
more complex scenarios as `(privilege1 AND privilege2) OR (privilege3
AND privilege4)`
To achieve `anyRequired` has been extended to allow defining multiple
AND conditions evaluated with OR logic:
```ts
security: {
authz: {
requiredPrivileges: [{
anyRequired: [
{ allOf: ['privilege1', 'privilege2'] },
{ allOf: ['privilege3', 'privilege4'] }
]
}
]
}
}
```
`allRequired` now also supports scenarios `(privilege1 OR privilege2)
AND (privilege3 OR privilege4)`
```ts
security: {
authz: {
requiredPrivileges: [{
allRequired: [
{ anyOf: ['privilege1', 'privilege2'] },
{ anyOf: ['privilege3', 'privilege4'] }
]
}
]
}
}
```
> [!IMPORTANT]
> We expect to have unique privileges in `anyOf` or `allOf` conditions,
assuming that most complex conditions can be simplified by boolean
algebra laws (OR/AND distributive etc).
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
__Closes: https://github.com/elastic/kibana/issues/210977__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
- Closeselastic/kibana-operations#245
- Convert FIPS base image from UBI to `chainguard-base-fips`
- Add FIPS base image updates to Renovate
- Adjust naming scheme for FIPS image from `kibana-ubi-fips` to
`kibana-fips`
- Adds new image flavor `kibana-cloud-fips`
- Adds support for `ci:build-cloud-fips-image` label
- Move Cloud image building to its own step instead of being part of
`Build Kibana Distribution` step so it will be triggered when the build
is reused and the `build` step is skipped.
Currently Kibana forwards `query_range_secs` and `query_offset_secs` to
mark the selected time range when reporting TTFMP event. This format
caused some challenges to identify `from`, `to` date offsets in
visualizations.
To simplify, the PR renames and sends the three fields explicitly:
- `query_from_offset_secs` offset to `0` (now), with -ve for past and
+ve for future dates
- `query_to_offset_secs` offset to `0` (now), with -ve for past and +ve
for future dates
- `query_range_secs` same as previously sent
_This approach is followed after a discussion, and based on the
[gist](https://gist.github.com/andrewvc/1f04a57a336d768e4ec5ff2eff06ba54)
excerpt:_
```
Earliest date -> QueryFrom
Newest date -> QueryTo
Duration -> QueryRange
```
### Indexing
These fields then should be mapped in the EBT indexer to ingest in the
top level of the document, eventually removing the need to create
runtime fields in data views for visualizations.
Also, runtime fields in data views should be updated to reflect this
change. For backward compatibility, the runtime fields can cater both
the old and new field names conditionally.
### Testing
- Ensure that the TTFMP events are correctly reporting the date ranges.
### Example
## Summary
The intent is to have a centralised place to store the list of Kibana
solutions and serverless project types.
To that end, this PR creates a `@kbn/projects-solutions-groups` package.
It also adds the new solution type `'chat'`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The `/packages` folder at the root of the Kibana repository used to
contain a lot of packages.
In the context of SKA, they have been gradually moved to various
locations:
* `src/platform/packages`
* `x-pack/platform/packages`
* `src/core/packages`
Currently, only `devOnly: true` packages are left in this folder. This
comprises libraries for CLI scripts as well as testing utilities.
With this PR, we are moving ~half of these packages under
`src/platform/packages/(private|shared)/`.
In particular, we are moving those packages that are being used from
platform and/or solutions.
Since they are `"devOnly": true`, this means they are ONLY used from
tests, cypress tests, storybook configs, ./scripts/ folders inside some
modules, or other non-prod-time logic. Nonetheless, they are effectively
referenced from platform and/or solutions code, hence I decided they
should be placed under `platform` folders.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
As part of our effort to harden API action definitions and enforce
standards this PR adds an utility `ApiPrivileges` class.
It is supposed to be used for both feature registration and API route
definition to construct the privilege name.
```ts
plugins.features.registerKibanaFeature({
privileges: {
all: {
app: [...],
catalogue: [...],
api: [ApiPrivileges.manage('subject_name')],
...
},
read: {
...
api: [ApiPrivileges.read('subject_name')],
...
},
},
})
....
// route definition
router.get(
{
path: 'api_path',
security: {
authz: {
requiredPrivileges: [ApiPrivileges.manage('subject_name')],
},
},
},
async (ctx, req, res) => {}
);
```
`require_kibana_feature_privileges_naming` eslint rule has been added to
show warning if the API privilege name doesn't satisfy the naming
convention.
### Naming convention
- API privilege should start with valid `ApiOperation`: `manage`,
`read`, `update`, `delete`, `create`
- API privilege should use `_` as separator
❌ `read-entity-a`
❌ `delete_entity-a`
❌ `entity_manage`
✅ `read_entity_a`
✅ `delete_entity_a`
✅ `manage_entity`
> [!IMPORTANT]
> Serverless ZDT update scenario:
>
> - version N has an endpoint protected with the `old_privilege_read`.
> - version N+1 has the same endpoint protected with a new
`read_privilege`.
>
> There might be a short period between the time the UI pod N+1 passes
SO migrations and updates privileges and the time it's marked as
ready-to-handle-requests by k8s, and when UI pod N is terminated.
>
> After discussion with @legrego and @azasypkin we decided to ignore it
due to the perceived risk-to-cost ratio:
> 1. The time window users might be affected is very narrow because we
register privileges late in the Kibana startup flow (e.g., after SO
migrations).
> 2. The transient 403 errors users might get won't result in session
termination and shouldn't lead to data loss.
> 3. The roll-out will be performed in batches over the course of
multiple weeks and implemented by different teams. This means the impact
per release shouldn't be significant.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Relates: https://github.com/elastic/kibana/issues/198716__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
- Updates `scripts/dependency_ownership` to use the
`@kbn/dev-cli-runner` for consistency with other CI-related CLIs.
- Adds a new `failIfUnowned` flag to exit with an error code if any
dependencies are unowned.
- Adds a new dependency ownership check to `quick_checks` and `renovate`
CI steps.
From a CI run, the additional quick check executes successfully in 3
seconds:
```sh
info [quick-checks] Passed check: /opt/buildkite-agent/builds/bk-agent-prod-gcp-abc123/elastic/kibana-pull-request/kibana/.buildkite/scripts/steps/checks/dependencies_missing_owner.sh in 3s
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adding documentation to help teams diagnose FIPS pipeline errors in
test.
Linked to FIPS compliant OpenSSL/Node setup docs
---------
Co-authored-by: Jeramy Soucy <jeramy.soucy@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 3 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/code-editor` |
`src/platform/packages/shared/shared-ux/code_editor/impl` |
| `@kbn/code-editor-mock` |
`src/platform/packages/shared/shared-ux/code_editor/mocks` |
| `@kbn/monaco` | `src/platform/packages/shared/kbn-monaco` |
<details >
<summary>Updated relative paths</summary>
```
src/platform/packages/shared/kbn-monaco/jest.config.js:12
src/platform/packages/shared/kbn-monaco/tsconfig.json:2
src/platform/packages/shared/kbn-monaco/tsconfig.type_check.json:2
src/platform/packages/shared/shared-ux/code_editor/impl/jest.config.js:12
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.json:16
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.json:2
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:18
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:2
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:25
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:28
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:31
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:34
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:37
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:40
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.json:16
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.json:2
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.type_check.json:18
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.type_check.json:2
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.type_check.json:25
```
</details>
## Summary
This PR adds support for explicit indication whether endpoint is
restricted to operator only users.
### Context
1. If user has [all operator
privileges](https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/operator/DefaultOperatorOnlyRegistry.java#L35-#L53)
granted, but is not listed as operator in `operator_users.yml`, ES would
throw an unauthorized error.
2. If user is listed as operator in `operator_users.yml`, but doesn't
have necessary privileges granted, ES would throw an unauthorized error.
3. It’s not possible to determine if a user is operator via any ES API,
i.e. `_has_privileges`.
4. If operator privileges are disabled we skip the the check for it,
that's why we require to explicitly specify additional privileges to
ensure that the route is protected even when operator privileges are
disabled.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Relates: https://github.com/elastic/kibana/issues/196271__
### How to test
1. Add your user to the operators list
1bd8144924/packages/kbn-es/src/serverless_resources/operator_users.yml (L4)
or use existing user from the list to log in.
2. Run ES and Kibana serverless
3. Change any endpoint or create a new one with the following security
config
```
security: {
authz: {
requiredPrivileges: [ReservedPrivilegesSet.operator],
},
},
```
4. Check with enabled and disabled operator privileges (set
`xpack.security.operator_privileges.enabled`)
## Release Note
Added support for explicit indication whether endpoint is restricted to
operator only users at the route definition level.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Relates to https://github.com/elastic/kibana/issues/181995
This PR updates the examples to include availability information and
another description.
Co-authored-by: Jean-Louis Leysens <jeanlouis.leysens@elastic.co>
