**!!MAJORITY OF THE CHANGED FILES ARE MOVED OR COPIED!!**
### Vision
According to the product vision we will build a new simple UI/UX in the
future https://github.com/elastic/security-team/issues/11790
This PR is a first iteration on enabling Content Connectors Management
UI in Serverless Kibana Stack Management.
Elastic Managed content connectors will be available only for Security
and Observability projects.
### Current PR scope
1. Used initial search_connectors plugin and renamed it to
content_connectors + moved from `x-pack/solutions/search` to
`x-pack/platform/plugins/shared`
2. Copy relevant connectors UI and routes from enterprise_search plugin.
3. Introduce the new Stack Management card/navigation option under the
Data section.
4. Enabled this plugin only in Serverless for Security and Observability
projects.
5. For making PR smaller Pipelines tab was not moved. And according to
Search team vision this functionality should be dropped anyway soon.
6. Extended fleet package logic to include elastic_connectors for
security and o11y serverless projects
7. Added back `search:agentless-connectors-manager` task
In Stack Management navigation:
<img width="2062" alt="Screenshot 2025-04-15 at 3 51 43 PM"
src="https://github.com/user-attachments/assets/5c93ba01-9a6a-4eac-a21d-1370f03b8f35"
/>
Stack Management cards:
<img width="2081" alt="Screenshot 2025-04-10 at 8 41 43 PM"
src="https://github.com/user-attachments/assets/3def1c12-561b-4a84-8241-4dd61cd9313d"
/>
Create Elastic Managed Connector UI (on Agentless):
<img width="1822" alt="Screenshot 2025-04-15 at 3 55 29 PM"
src="https://github.com/user-attachments/assets/6e9fea48-85e7-43df-919d-0e5492d0e704"
/>
Create Self Managed Connector UI:
<img width="2064" alt="Screenshot 2025-04-15 at 3 55 49 PM"
src="https://github.com/user-attachments/assets/d5051898-c8fa-4e41-b9ea-b41d4ed4a0d5"
/>
### Next steps
- [ ] Remove duplicated code between content_connectors,
enterprise_search and serverless_search
- [ ] Extract [common server
libs](https://github.com/elastic/kibana/tree/main/x-pack/solutions/search/plugins/enterprise_search/server/lib)
to the shared package `kbn-search-connectors`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Artem Shelkovnikov <artem.shelkovnikov@elastic.co>
Co-authored-by: Artem Shelkovnikov <lavatroublebubble@gmail.com>
Co-authored-by: Kyle Pollich <kyle.pollich@elastic.co>
fixes https://github.com/elastic/search-team/issues/9171
## Summary
The connectors integration should be GA in ECH, but shouldn't be
available in serverless right now.
~~Prettier also triggered on these files based on our `.prettierrc`
rules, it looks like. Let me know if I should revert all the formatting
changes. There were mixed quotes in a few places, and some indentation
stuff in others.~~
I removed the prettier changes because they invited substantial
conflicts with all the whitespace changes.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [x] Changes serverless behavior, so I'm deploying a serverless search
project to smoke test
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This updates the Search notebooks config path to fix a config conflict,
now that the main search plugin owns `xpack.search`. This config should
only have been used in Serverless and was never pushed to other users,
but I've added a rename and deprecation warning just in case anyone used
it..
## Summary
This PR renames the `enterprise_search` config path from
`enterpriseSearch` to `xpack.search`. This is to migrate away from
customer facing usage of enterprise search and align with other search
plugin config paths like `xpack.serverless.search`.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
As titled. The Project Switcher was created in #155582 as a means of
allowing a developer to switch between project types without having to
restart Kibana. Since that first PR, Kibana *must* be started with a
particular project configuration in mind-- the switcher now does
nothing. This PR removes it and deprecates the config keys.
## Summary
This PR reworks saved query privileges to rely solely on a single global
`savedQueryManagement` privilege, and eliminates app-specific overrides.
This change simplifies the security model for users, fixes bugginess in
the saved query management UI, and reduces code complexity associated
with maintaining two separate security mechanisms (app-specific
overrides and global saved query management privileges).
### Background
Saved queries allow users to store a combination of KQL or Lucene
queries, filters, and time filters to use across various applications in
Kibana. Access to saved query saved objects are currently granted by the
following feature privileges:
```json
[
"feature_discover.all",
"feature_dashboard.all",
"feature_savedQueryManagement.all",
"feature_maps.all",
"feature_savedObjectsManagement.all",
"feature_visualize.all"
]
```
There is also a saved query management UI within the Unified Search bar
shared by applications across Kibana:
<img
src="https://github.com/user-attachments/assets/e4a7539b-3dd4-4d47-9ff8-205281ef50e3"
width="500" />
The way access to this UI is managed in Kibana is currently confusing
and buggy:
- If a user has `feature_discover.all` and `feature_dashboard.all` they
will be able to load and save queries in Discover and Dashboard.
- If a user has `feature_discover.all` and `feature_dashboard.read` they
will be able to load queries in both Discover and Dashboard, but only
save queries in Discover (even though they have write access to the SO,
and API access). Instead they have to navigate to Discover to save a
query before navigating back to Dashboard to load it, making for a
confusing and frustrating UX.
- Access to the UI is even more confusing in apps not listed in the
above feature privileges (e.g. alerting, SLOs). Some of them chose to
check one of the above feature privileges, meaning users who otherwise
should have saved query access won't see the management UI if they don't
also have the exact feature privilege being checked. Other apps just
always show the management UI, leading to bugs and failures when users
without one of the above feature privileges attempt to save queries.
### Existing improvements
In v8.11.0, we introduced a new ["Saved Query
Management"](https://github.com/elastic/kibana/pull/166937) privilege,
allowing users to access saved queries across all of Kibana with a
single global privilege:
<img
src="https://github.com/user-attachments/assets/ccbe79a4-bd0b-4ed6-89c9-117cc1f99ee2"
width="600" />
When this privilege is added to a role, it solves the
`feature_discover.all` and `feature_dashboard.read` issue mentioned
above. However, it does not fix any of the mentioned issues for roles
without the new privilege. We have so far postponed further improvements
to avoid a breaking change.
### Approach
To fully resolve these issues and migrate to a single global privilege,
these changes have been made:
- Remove saved query SO access from all application feature privileges
and instead only allow access through the global saved query management
privilege.
- Stop relying on application feature privileges for toggling the saved
query management UI, and instead rely on the global privilege.
To implement this with minimal breaking changes, we've used the Kibana
privilege migration framework. This allows us to seamlessly migrate
existing roles containing feature privileges that currently provide
access to saved queries, ensuring they are assigned the global saved
query management privilege on upgrade.
As a result, we had to deprecate the following feature privileges,
replacing them with V2 privileges without saved query SO access:
```json
[
"feature_discover.all",
"feature_dashboard.all",
"feature_maps.all",
"feature_visualize.all"
]
```
Each area of code that currently relies on any of these feature
privileges had to be updated to instead access `feature_X_V2` instead
(as well as future code).
This PR still introduces a minor breaking change, since users who have
`feature_discover.all` and `feature_dashboard.read` are now able to save
queries in Dashboard after upgrade, but we believe this is a better UX
(and likely the expected one) and worth a small breaking change.
### Testing
- All existing privileges should continue to work as they do now,
including deprecated V1 feature privileges and customized serverless
privileges. There should be no changes for existing user roles apart
from the minor breaking change outlined above.
- Check that code changes in your area don't introduce breaking changes
to existing behaviour. Many of the changes are just updating client UI
capabilities code from `feature.privilege` to `feature_v2.privilege`,
which is backward compatible.
- The `savedQueryManagement` feature should now globally control access
to saved query management in Unified Search for all new user roles.
Regardless of privileges for Discover, Dashboard, Maps, or Visualize,
new user roles should follow this behaviour:
- If `savedQueryManagement` is `none`, the user cannot see or access the
saved query management UI or APIs.
- If `savedQueryManagement` is `read`, the user can load queries from
the UI and access read APIs, but cannot save queries from the UI or make
changes to queries through APIs.
- If `savedQueryManagement` is `all`, the user can both load and save
queries from the UI and through APIs.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
This PR risks introducing unintended breaking changes to user privileges
related to saved queries if the deprecated features have not been
properly migrated, and users could gain or lose access to saved query
management on upgrade. This would be bad if it happened, but not overly
severe since it wouldn't grant them access to any ES data they couldn't
previously access (only query saved objects). We have automated testing
in place to help ensure features have been migrated correctly, but the
scope of these changes are broad and touch many places in the codebase.
Additionally, the UI capabilities types are not very strict, and are
referenced with string paths in many places, which makes changing them
riskier than changing strictly typed code. A combination of regex
searches and temporarily modifying the `Capabilities` type to cause type
errors for deprecated privileges was used to identify references in
code. Reviewers should consider if there are any other ways that UI
capabilities can be referenced which were not addressed in this PR.
Our automated tests already help mitigate the risk, but it's important
that code owners thoroughly review the changes in their area and
consider if they could have unintended consequences. The Platform
Security team should also review this PR thoroughly, especially since
some changes were made to platform code around privilege handling. The
Data Discovery team will also manually test the behaviour when upgrading
existing user roles with deprecated feature privileges as part of 9.0
upgrade testing.
---------
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: “jeramysoucy” <jeramy.soucy@elastic.co>
## Summary
This PR removes rendering of the Inference Endpoints UI from
`enterprise_search` for stack and instead utilizes the
`search_inference_endpoints` plugin to render the UI for both serverless
and stack. This can be done by utilizing the `search_navigation` plugin
for rendering the classic navigation for stack.
To support this change the `xpack.searchInferenceEndpoints.ui.enabled`
was updated to default to `true` instead of only being set for
serverless search. To account for this change I have added
`xpack.searchInferenceEndpoints.enabled: false` to the serverless
configs for both security and observability to ensure the
`search_inference_endpoints` plugin is disabled in both of those
projects.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This adds exclusions for deprecated "rsa2elk" integration packages to Serverless projects.
The following packages should be excluded from Serverless.
- bluecoat
- cylance
- f5
- fortinet_forticlient
- juniper_junos
- juniper_netscreen
- netscout
- radware
- tomcat
---------
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
## Summary
This adds the AI assistant to Serverless Elasticsearch. It also disables
the knowledge base, and disables a few config values we don't want users
to be able to set in that context.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elena Shostak <165678770+elena-shostak@users.noreply.github.com>
Closes#194933Closes#192282
## Summary
This PR updates the serverless project yml files to
- enable custom roles for Elasticsearch and Security projects
- enable multiple spaces (max 100) for all serverless project types
### Tests
Additionally, this PR adjust the serverless test suites. Originally,
testing of roles and spaces endpoints was achieved from the feature flag
test config. Now that these features are enabled by default, the tests
have been migrated to the standard serverless test configs.
Affected tests:
-
x-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts
-
x-pack/test_serverless/functional/test_suites/common/platform_security/navigation/management_nav_cards.ts
-
x-pack/test_serverless/functional/test_suites/common/platform_security/roles.ts
-
x-pack/test_serverless/functional/test_suites/common/spaces/spaces_management.ts
-
x-pack/test_serverless/functional/test_suites/common/spaces/spaces_selection.ts
- Feature flag configs/indices
- Project specific configs/indices
- Base serverless config
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## Summary
#### Different vCPUs ranges and enabling support for static allocations
based on the serverless project type
- Each serverless config yml, e.g.
[search.es.yml](84b3b79a15/config/serverless.es.yml (L61))
now contains parameters required for start model deployment:
```yml
xpack.ml.nlp:
enabled: true
modelDeployment:
allowStaticAllocations: true
vCPURange:
low:
min: 0
max: 2
static: 2
medium:
min: 1
max: 32
static: 32
high:
min: 1
max: 512
static: 512
```
Note: _There will be no static allocations option for serverless O11y
and serverless Security._
#### The minimum values of vCPUs
- 0 for the Low usage level on both serverless and ESS.
- 1 for the Medium and High usage levels on both serverless and ESS.
#### The default vCPUs usage levels
- Low in serverless.
- Medium in ESS and on-prem
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Kibana should already support packages with spec version 3.2, update the
max version in serverless.
Supersedes part of https://github.com/elastic/kibana/pull/184792.
Co-authored-by: Jen Huang <its.jenetic@gmail.com>
## Summary
Resolves https://github.com/elastic/kibana/issues/182827
As the title says :)
This also corrects `spec.max` to `3.1`, which it should have been all
along.
Integrations available for Observability projects after change:
[Click
here](b9592e60-0bda-4597-b7fc-0e7ee5a673da)
Integrations available for Security projects after change:
[Click
here](f74142ae-88d8-4445-96f7-413c12a1434b)
## Summary
This PR extends the features plugin to accept feature definition
overrides via Kibana configuration. The functionality is limited to the
Serverless offering only. Additionally, the PR updates Kibana serverless
configurations to include overrides based on the "simplified feature
toggles" proposals discussed with the solution teams.
The configuration might look like this:
```yaml
## Fine-tune the feature privileges.
xpack.features.overrides:
dashboard:
privileges:
### Dashboard's `All` feature privilege should implicitly
### grant `All` access to Maps and Visualize features.
all.composedOf:
- feature: "maps"
privileges: [ "all" ]
- feature: "visualize"
privileges: [ "all" ]
### All Dashboard sub-feature privileges should be hidden:
### reporting capabilities will be granted via dedicated
### Reporting feature and short URL sub-feature privilege
### should be granted for both `All` and `Read`.
subFeatures.privileges:
download_csv_report.disabled: true
url_create:
disabled: true
includeIn: "read"
### Maps feature is disabled since it's automatically granted by Dashboard feature.
maps.disabled: true
```
## How to test
Log in as the `admin` using SAML and navigate to the `Custom roles`
management section to edit role and see tuned role management UX:
<p align="center">
<img
src="ad6e4b07-53bd-4f5a-ae91-66d6534c711a"
/>
<img
src="8ab4d5a3-f719-42d5-a278-3aee87603c33"
/>
</p>
### Search project
```bash
yarn es serverless --projectType=es --ssl -E xpack.security.authc.native_roles.enabled=true
yarn start --serverless=es --ssl --xpack.security.roleManagementEnabled=true
```
Refer to the proposal document, `config/serverless.yml`, and
`config/serverless.es.yml` in this PR to see the specific changes made
for your project type:
Create a custom `custom-search` role and re-login as the user with this
role to test your project type (you need to manually type role name if
the role selector):
<p align="center">
<img
src="5088320b-3cc8-4de9-984c-d70fc6277659"
/>
</p>
### Observability project
```bash
yarn es serverless --projectType=oblt --ssl -E xpack.security.authc.native_roles.enabled=true
yarn start --serverless=oblt --ssl --xpack.security.roleManagementEnabled=true
```
Refer to the proposal document, `config/serverless.yml`, and
`config/serverless.oblt.yml` in this PR to see the specific changes made
for your project type:
Create a custom `custom-o11y` role and re-login as the user with this
role to test your project type (you need to manually type role name if
the role selector):
<p align="center">
<img
src="110572b1-f08a-4427-a687-5c2e0240a36b"
/>
</p>
### Security project
```bash
yarn es serverless --projectType=security --ssl -E xpack.security.authc.native_roles.enabled=true
yarn start --serverless=security --ssl --xpack.security.roleManagementEnabled=true
```
Refer to the proposal document, `config/serverless.yml`, and
`config/serverless.security.yml` in this PR to see the specific changes
made for your project type:
Create a custom `custom-security` role and re-login as the user with
this role to test your project type (you need to manually type role name
if the role selector):
<p align="center">
<img
src="2bec6ae2-8d19-4142-a479-9a81bc1fca14"
/>
</p>
__Fixes: https://github.com/elastic/kibana/issues/178963__
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Added CSP and Cloud Defend plugins into Exclude Package list for oblt
projects in Serverless
Co-authored-by: Maxim Kholod <maxim.kholod@elastic.co>
### Summary
This PR enables the NLP models management page in the ML plugin for
Observability serverless projects and updates how the Obs AI Assistant
checks if ELSER is ready to be used.
Adds a setting in the `aiAssistantManagementSelection` plugin that
allows users to set the preferred Assistant type. The allowed values
are:
- `default`: the status quo, which is: show the Observability AI
Assistant in Observability apps, the Security AI Assistant in Security
solution apps, and none in the other apps.
- `never`: Never show any AI Assistant.
- `observability`: Show the Observability AI Assistant everywhere,
except for Security solution apps.
- `security`: ~Show the Security AI Assistant everywhere, except for
Observability apps.~
it is up to the solutions to respect the setting. See
[x-pack/plugins/observability_solution/observability_ai_assistant_app/public/hooks/is_nav_control_visible.tsx](https://github.com/elastic/kibana/pull/179233/files#diff-753a9aae298da45c5bd96bc2cacaa437ac53e165069ad6323973dd3c2879d4ae)
for an example. It should also be set in the specific Serverless config:
[config/serverless.oblt.yml](https://github.com/elastic/kibana/pull/179233/files#diff-358680d2b72268283dca4f3efb976cd924b7b5bce822e07c8871d061b4bb339e).
Update: this is now labeled as an Observability-only setting.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/176420
## 🍒 Summary
This PR copies the SLO code that was inside the Observability app into
its own app under `observability-solution/slo` folder.
4f6b8dfb-9612-4d30-ad50-4ee5c55a9c32
## ✔️ Acceptance criteria
- URL of new app: `app/slos`
- Design and functionality are not changed.
- Git history has been retained for all files in
`x-pack/plugins/observability_solution/slo`.
- SLO should appear on server less
- SLO code inside `observability_solution/observability` code has been
removed. A new clean up round might be needed though for possible
leftovers.
- Burn rate rule is registered within the new slo app
- SLO embeddables are moved inside the new slo app
- overview
- alerts embeddable
- error budget burn down
- Alerts table configuration registration for slo details page and
alerts table embeddable is still done in the observability app. Response
Ops team is working on removing the need to register the alert table
anyway
- Slo app is wrapped into `ApplicationUsageTrackingProvider` which will
send slo `Application usage` information tracked by the `slo` appId
- Redirect old `app/observability/slos` route to `app/slos`
- Rename old `xpack.observability.slo` keys to `xpack.slo` in the
translation files
## 🌮 How to test
Design and functionality didn't change, so simply navigate to existing
slo pages and try to break it
- Slo list page
- group by
- unified search
- toggle buttons
- actions
- Slo creation
- try group by as well
- Slo detail page
- Actions on top
- navigate to overview and alerts tabs
- Create SLO flyout in Logs Explorer
- Create burn rate rules and verify they appear on rules page
- Verify SLO alerts appear on Alerts page and slo details page
- Embeddables
- Through the dashboard app
- Using the attach to dashboard action on the slo card item on slo list
page and the error budget burn down chart on the slo detail page
- SLOs only for platinum users
- Permissions
- Spaces
## TODO
- [x] Move slo stuff from observability folder to new slo plugin
- [x] Remove old slo stuff from observability folder
- [x] Update references
- [x] Fix typescript and eslint errors
- [x] Paths
- [x] Locators
- [x] Burn rate rule registration
- [x] Embeddable Alerts table configuration registration
- [x] Embeddables
- [x] Translations
- [x] Verify plugin.ts files contain all registration logic
- [x] public
- [x] server
- [x] Final cleanup for observability folder
- [x] Run tests
- [x] Application Usage (Telemetry)
- [x] Permissions
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: shahzad31 <shahzad31comp@gmail.com>
Co-authored-by: Coen Warmer <coen.warmer@gmail.com>
## Summary
At present, we have the MKI Kibana Controller disable Synthetics in
production Serverless by default.
This patch will make Synthetics enabled by default, so at the point when
we want to remove the injected disable config on the controller side,
Synthetics will be available to all production Observability projects
immediately.
## Summary
Introduced the console.ui.embeddedEnabled (boolean) value to allow
disabling the embedded console in serverless security and observability
projects.
This also replaces an index management specific config value to do that
same thing which is removed here as well. Adding this console
configuration will allow us to have one value to set instead of N which
was the pattern I had started to use, but then decided against.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
## Summary
We will depend on the apm-data Elasticsearch plugin for setting up index
templates and ingest pipelines. We have been testing this in serverless
dev and QA with config overrides -- this is just final step to roll it
out to all environments.
See https://github.com/elastic/apm-server/issues/11529
## Summary
Resolves https://github.com/elastic/synthetics-dev/issues/290.
~Creating this PR as a place to experiment with Synthetics running
against the Serverless platform. Main goals are to find areas of the
codebase that require revision and get as much of Synthetics functioning
as possible without access to public locations.~
This adds the necessary config and other features to make Synthetics
workable as part of the oblt serverless project. Notably, we aren't
including the `xpack.uptime.enabled` flag here, because we are not ready
to expose the plugin to production users yet. We're going to enable the
plugin on a per-env basis using other means to start, and when we are
ready to expose the plugin generally in prod, we will add that flag.
Also adds nav and fixes a few other things that were broken for
serverless specifically.
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Adds a new `tags` property to our ML Modules which contains an array of
strings to used to filter the modules returned when calling
`/internal/ml/modules/get_module` or `/internal/ml/modules/recognize`
Adds a new kibana config setting `xpack.ml.compatibleModuleType` which
will enforce a module filter.
This setting supports the values `security`, `observability` or `search`
and will be used by the serverless projects to ensure only modules
relevant to the current project are discoverable.
When matching against the `xpack.ml.compatibleModuleType` setting,
modules with no tags will be returned along with modules which match the
`compatibleModuleType` value.
The endpoints `/internal/ml/modules/get_module` and
`/internal/ml/modules/recognize` can also take a `filter` query
parameter in the form of a comma separated list of values. If any of
these filter values match a module's tags, the module will be returned.
Modules with no tags will not be matched when using the `filter`
parameter.
This PR also updates the security plugin to apply a `security` filter
when retrieving modules and running recognize. This will improve
performance, especially for recognize which runs the queries from every
module on the supplied index pattern.
Examples
**Running the Security serverless project.**
Request:
`/internal/ml/modules/get_module`
Response:
All modules containing `tags: ["security"]`
Plus all modules with no `tags` or `tags: []`
Request:
`/internal/ml/modules/get_module?filter=observability`
Response:
An empty list
**Running stateful es**
Request:
`/internal/ml/modules/get_module`
Response:
All modules
Request:
`/internal/ml/modules/get_module?filter=security`
Response:
Only modules containing `tags: ["security"]`
Request:
`/internal/ml/modules/get_module?filter=security,observability`
Response:
Modules containing `tags: ["security"]` and `tags: ["observability"]`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Branch with Infra UI enabled in Serverless.
This exists for us to have a working environment based on main, but with
infra turned on. We could merge this at a later date when we plan to
enable Infra on serverless.
Fixes https://github.com/elastic/kibana/issues/167850Fixes#168065Fixes#168853
## Summary
Closes https://github.com/elastic/ingest-dev/issues/2396
Added a new kibana task that publishes Agent metrics every minute to
data streams installed by fleet_server package.
Opened the pr for review, there are a few things to finalize, but the
core logic won't change much.
To test locally:
- Install fleet_server package 1.4.0 from
[this](https://github.com/elastic/integrations/pull/8145) pr to get the
mappings
- Start kibana locally, wait for a few minutes for the metrics task to
run (every minute)
- Go to discover, `metrics-*` index pattern, filter on
`data_stream.dataset: fleet_server.*`
- Expect data to be populated in `fleet_server.agent_status` and
`fleet_server.agent_versions` datasets.
<img width="1787" alt="image"
src="615af9df-fe4b-4c17-8c8c-88646c403a18">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
fix https://github.com/elastic/kibana/issues/166233 (this is a short
term fix that removes "analytics" references from the empty state page
in serverless observability analytics apps)
Switches to using the serverless config file to enabled/disable ML
features rather than a function shared from the setup contract.
Storing these flags in a config file means they are already available
when setup runs and so can be used when registering integrations into
other plugins.
Removes the dependency on ML from `security_solution_serverless`,
`serverless_observability` and `serverless_search`
Reverts elastic/kibana#159674
The Elasticsearch team has changed their guidance about `fast_refresh`
and want this setting to be applied from within an Elasticsearch plugin
