# Backport
This will backport the following commits from `main` to `8.x`:
- [[Search] Inference Endpoints - Align rendering to plugin
(#203313)](https://github.com/elastic/kibana/pull/203313)
<!--- Backport version: 8.9.8 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Rodney
Norris","email":"rodney.norris@elastic.co"},"sourceCommit":{"committedDate":"2024-12-09T18:36:16Z","message":"[Search]
Inference Endpoints - Align rendering to plugin (#203313)\n\n##
Summary\r\n\r\nThis PR removes rendering of the Inference Endpoints UI
from\r\n`enterprise_search` for stack and instead utilizes
the\r\n`search_inference_endpoints` plugin to render the UI for both
serverless\r\nand stack. This can be done by utilizing the
`search_navigation` plugin\r\nfor rendering the classic navigation for
stack.\r\n\r\nTo support this change the
`xpack.searchInferenceEndpoints.ui.enabled`\r\nwas updated to default to
`true` instead of only being set for\r\nserverless search. To account
for this change I have added\r\n`xpack.searchInferenceEndpoints.enabled:
false` to the serverless\r\nconfigs for both security and observability
to ensure the\r\n`search_inference_endpoints` plugin is disabled in both
of those\r\nprojects.\r\n\r\n### Checklist\r\n\r\n- [x] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] This was
checked for breaking HTTP API changes, and any breaking\r\nchanges have
been approved by the breaking-change committee.
The\r\n`release_note:breaking` label should be applied in these
situations.\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"ebb4f503a5c9c0a3d883a706566920ae27fedfad","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Search","backport:prev-minor","v8.18.0"],"number":203313,"url":"https://github.com/elastic/kibana/pull/203313","mergeCommit":{"message":"[Search]
Inference Endpoints - Align rendering to plugin (#203313)\n\n##
Summary\r\n\r\nThis PR removes rendering of the Inference Endpoints UI
from\r\n`enterprise_search` for stack and instead utilizes
the\r\n`search_inference_endpoints` plugin to render the UI for both
serverless\r\nand stack. This can be done by utilizing the
`search_navigation` plugin\r\nfor rendering the classic navigation for
stack.\r\n\r\nTo support this change the
`xpack.searchInferenceEndpoints.ui.enabled`\r\nwas updated to default to
`true` instead of only being set for\r\nserverless search. To account
for this change I have added\r\n`xpack.searchInferenceEndpoints.enabled:
false` to the serverless\r\nconfigs for both security and observability
to ensure the\r\n`search_inference_endpoints` plugin is disabled in both
of those\r\nprojects.\r\n\r\n### Checklist\r\n\r\n- [x] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] This was
checked for breaking HTTP API changes, and any breaking\r\nchanges have
been approved by the breaking-change committee.
The\r\n`release_note:breaking` label should be applied in these
situations.\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"ebb4f503a5c9c0a3d883a706566920ae27fedfad"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/203313","number":203313,"mergeCommit":{"message":"[Search]
Inference Endpoints - Align rendering to plugin (#203313)\n\n##
Summary\r\n\r\nThis PR removes rendering of the Inference Endpoints UI
from\r\n`enterprise_search` for stack and instead utilizes
the\r\n`search_inference_endpoints` plugin to render the UI for both
serverless\r\nand stack. This can be done by utilizing the
`search_navigation` plugin\r\nfor rendering the classic navigation for
stack.\r\n\r\nTo support this change the
`xpack.searchInferenceEndpoints.ui.enabled`\r\nwas updated to default to
`true` instead of only being set for\r\nserverless search. To account
for this change I have added\r\n`xpack.searchInferenceEndpoints.enabled:
false` to the serverless\r\nconfigs for both security and observability
to ensure the\r\n`search_inference_endpoints` plugin is disabled in both
of those\r\nprojects.\r\n\r\n### Checklist\r\n\r\n- [x] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] This was
checked for breaking HTTP API changes, and any breaking\r\nchanges have
been approved by the breaking-change committee.
The\r\n`release_note:breaking` label should be applied in these
situations.\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"ebb4f503a5c9c0a3d883a706566920ae27fedfad"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `8.x`:
- [AI Assistant] Add assistant to Serverless Search (#196832) (3bc5e2db)
<!--- Backport version: 8.9.8 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Sander
Philipse","email":"94373878+sphilipse@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-25T10:03:04Z","message":"[AI
Assistant] Add assistant to Serverless Search (#196832)\n\n##
Summary\r\n\r\nThis adds the AI assistant to Serverless Elasticsearch.
It also disables\r\nthe knowledge base, and disables a few config values
we don't want users\r\nto be able to set in that
context.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elena Shostak
<165678770+elena-shostak@users.noreply.github.com>","sha":"3bc5e2db73799dc9c7831b6f9da4a52063cf112f"},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[]}]
BACKPORT-->
# Backport
This will backport the following commits from `main` to `8.x`:
- [Update max supported package version
(#196551)](https://github.com/elastic/kibana/pull/196551)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Giorgos
Bamparopoulos","email":"georgios.bamparopoulos@elastic.co"},"sourceCommit":{"committedDate":"2024-10-16T16:40:00Z","message":"Update
max supported package version (#196551)\n\nUpdate the max supported
package version to 3.3.0\r\n\r\nRelated to
https://github.com/elastic/package-spec/pull/818","sha":"2f76b60b0e2646b71cbc95b0de559154dd947dca","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","Team:Fleet","v9.0.0","backport:prev-minor"],"title":"Update
max supported package
version","number":196551,"url":"https://github.com/elastic/kibana/pull/196551","mergeCommit":{"message":"Update
max supported package version (#196551)\n\nUpdate the max supported
package version to 3.3.0\r\n\r\nRelated to
https://github.com/elastic/package-spec/pull/818","sha":"2f76b60b0e2646b71cbc95b0de559154dd947dca"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196551","number":196551,"mergeCommit":{"message":"Update
max supported package version (#196551)\n\nUpdate the max supported
package version to 3.3.0\r\n\r\nRelated to
https://github.com/elastic/package-spec/pull/818","sha":"2f76b60b0e2646b71cbc95b0de559154dd947dca"}}]}]
BACKPORT-->
Co-authored-by: Giorgos Bamparopoulos <georgios.bamparopoulos@elastic.co>
# Backport
This will backport the following commits from `main` to `8.x`:
- [[ML] Update vCPUs ranges for start model deployment
(#195617)](https://github.com/elastic/kibana/pull/195617)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Dima
Arnautov","email":"dmitrii.arnautov@elastic.co"},"sourceCommit":{"committedDate":"2024-10-14T14:38:26Z","message":"[ML]
Update vCPUs ranges for start model deployment (#195617)\n\n##
Summary\r\n\r\n#### Different vCPUs ranges and enabling support for
static allocations\r\nbased on the serverless project type\r\n\r\n- Each
serverless config yml,
e.g.\r\n[search.es.yml](84b3b79a15/config/serverless.es.yml (L61))\r\nnow
contains parameters required for start model
deployment:\r\n\r\n```yml\r\nxpack.ml.nlp:\r\n enabled: true\r\n
modelDeployment:\r\n allowStaticAllocations: true\r\n vCPURange:\r\n
low:\r\n min: 0\r\n max: 2\r\n static: 2\r\n medium:\r\n min: 1\r\n max:
32\r\n static: 32\r\n high:\r\n min: 1\r\n max: 512\r\n static:
512\r\n```\r\n\r\nNote: _There will be no static allocations option for
serverless O11y\r\nand serverless Security._\r\n\r\n#### The minimum
values of vCPUs\r\n\r\n- 0 for the Low usage level on both serverless
and ESS.\r\n- 1 for the Medium and High usage levels on both serverless
and ESS.\r\n \r\n#### The default vCPUs usage levels\r\n- Low in
serverless.\r\n- Medium in ESS and on-prem\r\n\r\n### Checklist\r\n\r\n-
[x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"13897083dc8c465a16ddb3856d4e4904b9629610","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement",":ml","v9.0.0","Feature:3rd
Party
Models","Team:ML","ci:project-deploy-elasticsearch","ci:project-deploy-observability","v8.16.0","backport:version"],"title":"[ML]
Update vCPUs ranges for start model deployment
","number":195617,"url":"https://github.com/elastic/kibana/pull/195617","mergeCommit":{"message":"[ML]
Update vCPUs ranges for start model deployment (#195617)\n\n##
Summary\r\n\r\n#### Different vCPUs ranges and enabling support for
static allocations\r\nbased on the serverless project type\r\n\r\n- Each
serverless config yml,
e.g.\r\n[search.es.yml](84b3b79a15/config/serverless.es.yml (L61))\r\nnow
contains parameters required for start model
deployment:\r\n\r\n```yml\r\nxpack.ml.nlp:\r\n enabled: true\r\n
modelDeployment:\r\n allowStaticAllocations: true\r\n vCPURange:\r\n
low:\r\n min: 0\r\n max: 2\r\n static: 2\r\n medium:\r\n min: 1\r\n max:
32\r\n static: 32\r\n high:\r\n min: 1\r\n max: 512\r\n static:
512\r\n```\r\n\r\nNote: _There will be no static allocations option for
serverless O11y\r\nand serverless Security._\r\n\r\n#### The minimum
values of vCPUs\r\n\r\n- 0 for the Low usage level on both serverless
and ESS.\r\n- 1 for the Medium and High usage levels on both serverless
and ESS.\r\n \r\n#### The default vCPUs usage levels\r\n- Low in
serverless.\r\n- Medium in ESS and on-prem\r\n\r\n### Checklist\r\n\r\n-
[x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"13897083dc8c465a16ddb3856d4e4904b9629610"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195617","number":195617,"mergeCommit":{"message":"[ML]
Update vCPUs ranges for start model deployment (#195617)\n\n##
Summary\r\n\r\n#### Different vCPUs ranges and enabling support for
static allocations\r\nbased on the serverless project type\r\n\r\n- Each
serverless config yml,
e.g.\r\n[search.es.yml](84b3b79a15/config/serverless.es.yml (L61))\r\nnow
contains parameters required for start model
deployment:\r\n\r\n```yml\r\nxpack.ml.nlp:\r\n enabled: true\r\n
modelDeployment:\r\n allowStaticAllocations: true\r\n vCPURange:\r\n
low:\r\n min: 0\r\n max: 2\r\n static: 2\r\n medium:\r\n min: 1\r\n max:
32\r\n static: 32\r\n high:\r\n min: 1\r\n max: 512\r\n static:
512\r\n```\r\n\r\nNote: _There will be no static allocations option for
serverless O11y\r\nand serverless Security._\r\n\r\n#### The minimum
values of vCPUs\r\n\r\n- 0 for the Low usage level on both serverless
and ESS.\r\n- 1 for the Medium and High usage levels on both serverless
and ESS.\r\n \r\n#### The default vCPUs usage levels\r\n- Low in
serverless.\r\n- Medium in ESS and on-prem\r\n\r\n### Checklist\r\n\r\n-
[x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"13897083dc8c465a16ddb3856d4e4904b9629610"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Co-authored-by: Dima Arnautov <dmitrii.arnautov@elastic.co>
Kibana should already support packages with spec version 3.2, update the
max version in serverless.
Supersedes part of https://github.com/elastic/kibana/pull/184792.
Co-authored-by: Jen Huang <its.jenetic@gmail.com>
## Summary
Resolves https://github.com/elastic/kibana/issues/182827
As the title says :)
This also corrects `spec.max` to `3.1`, which it should have been all
along.
Integrations available for Observability projects after change:
[Click
here](b9592e60-0bda-4597-b7fc-0e7ee5a673da)
Integrations available for Security projects after change:
[Click
here](f74142ae-88d8-4445-96f7-413c12a1434b)
## Summary
This PR extends the features plugin to accept feature definition
overrides via Kibana configuration. The functionality is limited to the
Serverless offering only. Additionally, the PR updates Kibana serverless
configurations to include overrides based on the "simplified feature
toggles" proposals discussed with the solution teams.
The configuration might look like this:
```yaml
## Fine-tune the feature privileges.
xpack.features.overrides:
dashboard:
privileges:
### Dashboard's `All` feature privilege should implicitly
### grant `All` access to Maps and Visualize features.
all.composedOf:
- feature: "maps"
privileges: [ "all" ]
- feature: "visualize"
privileges: [ "all" ]
### All Dashboard sub-feature privileges should be hidden:
### reporting capabilities will be granted via dedicated
### Reporting feature and short URL sub-feature privilege
### should be granted for both `All` and `Read`.
subFeatures.privileges:
download_csv_report.disabled: true
url_create:
disabled: true
includeIn: "read"
### Maps feature is disabled since it's automatically granted by Dashboard feature.
maps.disabled: true
```
## How to test
Log in as the `admin` using SAML and navigate to the `Custom roles`
management section to edit role and see tuned role management UX:
<p align="center">
<img
src="ad6e4b07-53bd-4f5a-ae91-66d6534c711a"
/>
<img
src="8ab4d5a3-f719-42d5-a278-3aee87603c33"
/>
</p>
### Search project
```bash
yarn es serverless --projectType=es --ssl -E xpack.security.authc.native_roles.enabled=true
yarn start --serverless=es --ssl --xpack.security.roleManagementEnabled=true
```
Refer to the proposal document, `config/serverless.yml`, and
`config/serverless.es.yml` in this PR to see the specific changes made
for your project type:
Create a custom `custom-search` role and re-login as the user with this
role to test your project type (you need to manually type role name if
the role selector):
<p align="center">
<img
src="5088320b-3cc8-4de9-984c-d70fc6277659"
/>
</p>
### Observability project
```bash
yarn es serverless --projectType=oblt --ssl -E xpack.security.authc.native_roles.enabled=true
yarn start --serverless=oblt --ssl --xpack.security.roleManagementEnabled=true
```
Refer to the proposal document, `config/serverless.yml`, and
`config/serverless.oblt.yml` in this PR to see the specific changes made
for your project type:
Create a custom `custom-o11y` role and re-login as the user with this
role to test your project type (you need to manually type role name if
the role selector):
<p align="center">
<img
src="110572b1-f08a-4427-a687-5c2e0240a36b"
/>
</p>
### Security project
```bash
yarn es serverless --projectType=security --ssl -E xpack.security.authc.native_roles.enabled=true
yarn start --serverless=security --ssl --xpack.security.roleManagementEnabled=true
```
Refer to the proposal document, `config/serverless.yml`, and
`config/serverless.security.yml` in this PR to see the specific changes
made for your project type:
Create a custom `custom-security` role and re-login as the user with
this role to test your project type (you need to manually type role name
if the role selector):
<p align="center">
<img
src="2bec6ae2-8d19-4142-a479-9a81bc1fca14"
/>
</p>
__Fixes: https://github.com/elastic/kibana/issues/178963__
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Added CSP and Cloud Defend plugins into Exclude Package list for oblt
projects in Serverless
Co-authored-by: Maxim Kholod <maxim.kholod@elastic.co>
### Summary
This PR enables the NLP models management page in the ML plugin for
Observability serverless projects and updates how the Obs AI Assistant
checks if ELSER is ready to be used.
Adds a setting in the `aiAssistantManagementSelection` plugin that
allows users to set the preferred Assistant type. The allowed values
are:
- `default`: the status quo, which is: show the Observability AI
Assistant in Observability apps, the Security AI Assistant in Security
solution apps, and none in the other apps.
- `never`: Never show any AI Assistant.
- `observability`: Show the Observability AI Assistant everywhere,
except for Security solution apps.
- `security`: ~Show the Security AI Assistant everywhere, except for
Observability apps.~
it is up to the solutions to respect the setting. See
[x-pack/plugins/observability_solution/observability_ai_assistant_app/public/hooks/is_nav_control_visible.tsx](https://github.com/elastic/kibana/pull/179233/files#diff-753a9aae298da45c5bd96bc2cacaa437ac53e165069ad6323973dd3c2879d4ae)
for an example. It should also be set in the specific Serverless config:
[config/serverless.oblt.yml](https://github.com/elastic/kibana/pull/179233/files#diff-358680d2b72268283dca4f3efb976cd924b7b5bce822e07c8871d061b4bb339e).
Update: this is now labeled as an Observability-only setting.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/176420
## 🍒 Summary
This PR copies the SLO code that was inside the Observability app into
its own app under `observability-solution/slo` folder.
4f6b8dfb-9612-4d30-ad50-4ee5c55a9c32
## ✔️ Acceptance criteria
- URL of new app: `app/slos`
- Design and functionality are not changed.
- Git history has been retained for all files in
`x-pack/plugins/observability_solution/slo`.
- SLO should appear on server less
- SLO code inside `observability_solution/observability` code has been
removed. A new clean up round might be needed though for possible
leftovers.
- Burn rate rule is registered within the new slo app
- SLO embeddables are moved inside the new slo app
- overview
- alerts embeddable
- error budget burn down
- Alerts table configuration registration for slo details page and
alerts table embeddable is still done in the observability app. Response
Ops team is working on removing the need to register the alert table
anyway
- Slo app is wrapped into `ApplicationUsageTrackingProvider` which will
send slo `Application usage` information tracked by the `slo` appId
- Redirect old `app/observability/slos` route to `app/slos`
- Rename old `xpack.observability.slo` keys to `xpack.slo` in the
translation files
## 🌮 How to test
Design and functionality didn't change, so simply navigate to existing
slo pages and try to break it
- Slo list page
- group by
- unified search
- toggle buttons
- actions
- Slo creation
- try group by as well
- Slo detail page
- Actions on top
- navigate to overview and alerts tabs
- Create SLO flyout in Logs Explorer
- Create burn rate rules and verify they appear on rules page
- Verify SLO alerts appear on Alerts page and slo details page
- Embeddables
- Through the dashboard app
- Using the attach to dashboard action on the slo card item on slo list
page and the error budget burn down chart on the slo detail page
- SLOs only for platinum users
- Permissions
- Spaces
## TODO
- [x] Move slo stuff from observability folder to new slo plugin
- [x] Remove old slo stuff from observability folder
- [x] Update references
- [x] Fix typescript and eslint errors
- [x] Paths
- [x] Locators
- [x] Burn rate rule registration
- [x] Embeddable Alerts table configuration registration
- [x] Embeddables
- [x] Translations
- [x] Verify plugin.ts files contain all registration logic
- [x] public
- [x] server
- [x] Final cleanup for observability folder
- [x] Run tests
- [x] Application Usage (Telemetry)
- [x] Permissions
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: shahzad31 <shahzad31comp@gmail.com>
Co-authored-by: Coen Warmer <coen.warmer@gmail.com>
## Summary
At present, we have the MKI Kibana Controller disable Synthetics in
production Serverless by default.
This patch will make Synthetics enabled by default, so at the point when
we want to remove the injected disable config on the controller side,
Synthetics will be available to all production Observability projects
immediately.
## Summary
Introduced the console.ui.embeddedEnabled (boolean) value to allow
disabling the embedded console in serverless security and observability
projects.
This also replaces an index management specific config value to do that
same thing which is removed here as well. Adding this console
configuration will allow us to have one value to set instead of N which
was the pattern I had started to use, but then decided against.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
## Summary
We will depend on the apm-data Elasticsearch plugin for setting up index
templates and ingest pipelines. We have been testing this in serverless
dev and QA with config overrides -- this is just final step to roll it
out to all environments.
See https://github.com/elastic/apm-server/issues/11529
## Summary
Resolves https://github.com/elastic/synthetics-dev/issues/290.
~Creating this PR as a place to experiment with Synthetics running
against the Serverless platform. Main goals are to find areas of the
codebase that require revision and get as much of Synthetics functioning
as possible without access to public locations.~
This adds the necessary config and other features to make Synthetics
workable as part of the oblt serverless project. Notably, we aren't
including the `xpack.uptime.enabled` flag here, because we are not ready
to expose the plugin to production users yet. We're going to enable the
plugin on a per-env basis using other means to start, and when we are
ready to expose the plugin generally in prod, we will add that flag.
Also adds nav and fixes a few other things that were broken for
serverless specifically.
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Adds a new `tags` property to our ML Modules which contains an array of
strings to used to filter the modules returned when calling
`/internal/ml/modules/get_module` or `/internal/ml/modules/recognize`
Adds a new kibana config setting `xpack.ml.compatibleModuleType` which
will enforce a module filter.
This setting supports the values `security`, `observability` or `search`
and will be used by the serverless projects to ensure only modules
relevant to the current project are discoverable.
When matching against the `xpack.ml.compatibleModuleType` setting,
modules with no tags will be returned along with modules which match the
`compatibleModuleType` value.
The endpoints `/internal/ml/modules/get_module` and
`/internal/ml/modules/recognize` can also take a `filter` query
parameter in the form of a comma separated list of values. If any of
these filter values match a module's tags, the module will be returned.
Modules with no tags will not be matched when using the `filter`
parameter.
This PR also updates the security plugin to apply a `security` filter
when retrieving modules and running recognize. This will improve
performance, especially for recognize which runs the queries from every
module on the supplied index pattern.
Examples
**Running the Security serverless project.**
Request:
`/internal/ml/modules/get_module`
Response:
All modules containing `tags: ["security"]`
Plus all modules with no `tags` or `tags: []`
Request:
`/internal/ml/modules/get_module?filter=observability`
Response:
An empty list
**Running stateful es**
Request:
`/internal/ml/modules/get_module`
Response:
All modules
Request:
`/internal/ml/modules/get_module?filter=security`
Response:
Only modules containing `tags: ["security"]`
Request:
`/internal/ml/modules/get_module?filter=security,observability`
Response:
Modules containing `tags: ["security"]` and `tags: ["observability"]`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Branch with Infra UI enabled in Serverless.
This exists for us to have a working environment based on main, but with
infra turned on. We could merge this at a later date when we plan to
enable Infra on serverless.
Fixes https://github.com/elastic/kibana/issues/167850Fixes#168065Fixes#168853
## Summary
Closes https://github.com/elastic/ingest-dev/issues/2396
Added a new kibana task that publishes Agent metrics every minute to
data streams installed by fleet_server package.
Opened the pr for review, there are a few things to finalize, but the
core logic won't change much.
To test locally:
- Install fleet_server package 1.4.0 from
[this](https://github.com/elastic/integrations/pull/8145) pr to get the
mappings
- Start kibana locally, wait for a few minutes for the metrics task to
run (every minute)
- Go to discover, `metrics-*` index pattern, filter on
`data_stream.dataset: fleet_server.*`
- Expect data to be populated in `fleet_server.agent_status` and
`fleet_server.agent_versions` datasets.
<img width="1787" alt="image"
src="615af9df-fe4b-4c17-8c8c-88646c403a18">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
fix https://github.com/elastic/kibana/issues/166233 (this is a short
term fix that removes "analytics" references from the empty state page
in serverless observability analytics apps)
Switches to using the serverless config file to enabled/disable ML
features rather than a function shared from the setup contract.
Storing these flags in a config file means they are already available
when setup runs and so can be used when registering integrations into
other plugins.
Removes the dependency on ML from `security_solution_serverless`,
`serverless_observability` and `serverless_search`
Reverts elastic/kibana#159674
The Elasticsearch team has changed their guidance about `fast_refresh`
and want this setting to be applied from within an Elasticsearch plugin
Closes: https://github.com/elastic/kibana/issues/155330
Adds `index.fast_refresh` to `.apm-custom-link` in order to ensure fast
index refreshes on serverless (1 second periodic refreshes instead of 5
second which is the new default on serverless).
This is pending on Elasticsearch adding support for `index.fast_refresh`
(https://github.com/elastic/elasticsearch/pull/96660)
---------
Co-authored-by: miriam.aparicio <miriam.aparicio@gmail.com>
Co-authored-by: Miriam <31922082+MiriamAparicio@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/155371
## Summary
PR adds Serverless Onboarding flow using Custom Integration. This would
also lay the foundation for us to complete get rid of Home Tutorial App
and move the remaining `onPrem` and `cloud` tutorials which are
currently still loaded using Home Tutorial App.
1. Adds new Custom Integration for Serverless Onboarding (Toggling Home
AApp Tutorial Integration)
2. Since we are migrating away from the Home App Tutorials, lot of
existing code has been duplicated and refactored for the custom
implementation. Home App Tutorial would require the Server to register
all the steps and the client to only register a custom component which
then would be loaded by Home App Tutorial component. We don't need to
follow this approach any more. All the UX logic has now been moved to
the Public folder with only Custom Integration done on the
`server/plugin.ts`.
3. As we are not sure how the solutions will be informed about being
running on Serverless or not, I have introduced a new variable in
`serverless.oblt.yml` file called `xpack.apm.serverlessOnboarding:
true`. With this the development has been done. This can be changed to
actual logic once we know more.
4. A new configuration `xpack.apm.managedServiceUrl` for accessing
Managed Service URL is also being added by Control Plane team as part of
https://elasticco.atlassian.net/browse/CP-2403. Hence this PR expects
this property to be present for Serverless.
5. Unit tests to toggle between `secret_token` and `api_key` depending
on availability has been added. No API Tests were added as no new API
created. Cypress Tests cannot be added due to Serverless
## Need help reviewing the PR ?
1. `config/serverless.oblt.yml` - Adds the new flag which would enable
this flow
2. `x-pack/plugins/apm/common/tutorial/tutorials.ts` - Defines the
configuration required to register the APM's Tutorial Custom Integration
3. `x-pack/plugins/apm/public/components/app/tutorials/commands` - This
directory contains all the agent specific data required to load the
TABLE with settings required for configuring APM MIS.
4. `x-pack/plugins/apm/public/components/app/tutorials/instructions` -
This folder contains all the individual agent specific instructions in
the format used by
[EuiSteps](https://eui.elastic.co/#/navigation/steps#complex-steps)
5. `x-pack/plugins/apm/public/components/routing` - Here we register our
custom route
6. Changes on the server side a quite small and they only register the
custom integration.
7.
`x-pack/plugins/apm/public/components/app/tutorials/serverless_instructions.tsx`
- This file currently defines all the logic for registering Serverless
instructions. We will soon have similar files for `onPrem` and `cloud`
instructions
### Risk Matrix
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| The flow depends on presence of a flag in `kibana.yml` file. | Low |
High | By default this flow will be disabled and would fallback to
traditional onboarding in absence of the flag. |
### Demo
d60f0610-1fea-4540-86f5-2d72ab97f640
### Updated Demo with Create API Button inside the table
e84d8d6c-a048-4638-9b63-45080feca90b
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Since the only way to reach logs onboarding is manually entering the url
it has been decided to enabled this plugin by default.
In serverless it was already enabled by default through the config.
## Summary
The PR introduces the first iteration of the side navigation bar for
serverless observability. Part of
https://github.com/elastic/kibana/issues/153777
### Run locally
```
yarn serverless-oblt
```
#### Screenshots
39ded143-0d4b-4ea6-9534-0ca87b80662d
### Fixes
- Fix rendering the icon for the nested navigation items
-
24ee4dc616
### Notes
- There is an issue where the selected navigation item is not properly
highlighted and loses focus when the user clicks anywhere else on the
page. (atm out of the scope of the PR)
- The navigation tree is subject to change as there is an ongoing
discussion about the naming and order
---------
Co-authored-by: Søren Louv-Jansen <soren.louv@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
