## Summary
This PR reworks saved query privileges to rely solely on a single global
`savedQueryManagement` privilege, and eliminates app-specific overrides.
This change simplifies the security model for users, fixes bugginess in
the saved query management UI, and reduces code complexity associated
with maintaining two separate security mechanisms (app-specific
overrides and global saved query management privileges).
### Background
Saved queries allow users to store a combination of KQL or Lucene
queries, filters, and time filters to use across various applications in
Kibana. Access to saved query saved objects are currently granted by the
following feature privileges:
```json
[
"feature_discover.all",
"feature_dashboard.all",
"feature_savedQueryManagement.all",
"feature_maps.all",
"feature_savedObjectsManagement.all",
"feature_visualize.all"
]
```
There is also a saved query management UI within the Unified Search bar
shared by applications across Kibana:
<img
src="https://github.com/user-attachments/assets/e4a7539b-3dd4-4d47-9ff8-205281ef50e3"
width="500" />
The way access to this UI is managed in Kibana is currently confusing
and buggy:
- If a user has `feature_discover.all` and `feature_dashboard.all` they
will be able to load and save queries in Discover and Dashboard.
- If a user has `feature_discover.all` and `feature_dashboard.read` they
will be able to load queries in both Discover and Dashboard, but only
save queries in Discover (even though they have write access to the SO,
and API access). Instead they have to navigate to Discover to save a
query before navigating back to Dashboard to load it, making for a
confusing and frustrating UX.
- Access to the UI is even more confusing in apps not listed in the
above feature privileges (e.g. alerting, SLOs). Some of them chose to
check one of the above feature privileges, meaning users who otherwise
should have saved query access won't see the management UI if they don't
also have the exact feature privilege being checked. Other apps just
always show the management UI, leading to bugs and failures when users
without one of the above feature privileges attempt to save queries.
### Existing improvements
In v8.11.0, we introduced a new ["Saved Query
Management"](https://github.com/elastic/kibana/pull/166937) privilege,
allowing users to access saved queries across all of Kibana with a
single global privilege:
<img
src="https://github.com/user-attachments/assets/ccbe79a4-bd0b-4ed6-89c9-117cc1f99ee2"
width="600" />
When this privilege is added to a role, it solves the
`feature_discover.all` and `feature_dashboard.read` issue mentioned
above. However, it does not fix any of the mentioned issues for roles
without the new privilege. We have so far postponed further improvements
to avoid a breaking change.
### Approach
To fully resolve these issues and migrate to a single global privilege,
these changes have been made:
- Remove saved query SO access from all application feature privileges
and instead only allow access through the global saved query management
privilege.
- Stop relying on application feature privileges for toggling the saved
query management UI, and instead rely on the global privilege.
To implement this with minimal breaking changes, we've used the Kibana
privilege migration framework. This allows us to seamlessly migrate
existing roles containing feature privileges that currently provide
access to saved queries, ensuring they are assigned the global saved
query management privilege on upgrade.
As a result, we had to deprecate the following feature privileges,
replacing them with V2 privileges without saved query SO access:
```json
[
"feature_discover.all",
"feature_dashboard.all",
"feature_maps.all",
"feature_visualize.all"
]
```
Each area of code that currently relies on any of these feature
privileges had to be updated to instead access `feature_X_V2` instead
(as well as future code).
This PR still introduces a minor breaking change, since users who have
`feature_discover.all` and `feature_dashboard.read` are now able to save
queries in Dashboard after upgrade, but we believe this is a better UX
(and likely the expected one) and worth a small breaking change.
### Testing
- All existing privileges should continue to work as they do now,
including deprecated V1 feature privileges and customized serverless
privileges. There should be no changes for existing user roles apart
from the minor breaking change outlined above.
- Check that code changes in your area don't introduce breaking changes
to existing behaviour. Many of the changes are just updating client UI
capabilities code from `feature.privilege` to `feature_v2.privilege`,
which is backward compatible.
- The `savedQueryManagement` feature should now globally control access
to saved query management in Unified Search for all new user roles.
Regardless of privileges for Discover, Dashboard, Maps, or Visualize,
new user roles should follow this behaviour:
- If `savedQueryManagement` is `none`, the user cannot see or access the
saved query management UI or APIs.
- If `savedQueryManagement` is `read`, the user can load queries from
the UI and access read APIs, but cannot save queries from the UI or make
changes to queries through APIs.
- If `savedQueryManagement` is `all`, the user can both load and save
queries from the UI and through APIs.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
This PR risks introducing unintended breaking changes to user privileges
related to saved queries if the deprecated features have not been
properly migrated, and users could gain or lose access to saved query
management on upgrade. This would be bad if it happened, but not overly
severe since it wouldn't grant them access to any ES data they couldn't
previously access (only query saved objects). We have automated testing
in place to help ensure features have been migrated correctly, but the
scope of these changes are broad and touch many places in the codebase.
Additionally, the UI capabilities types are not very strict, and are
referenced with string paths in many places, which makes changing them
riskier than changing strictly typed code. A combination of regex
searches and temporarily modifying the `Capabilities` type to cause type
errors for deprecated privileges was used to identify references in
code. Reviewers should consider if there are any other ways that UI
capabilities can be referenced which were not addressed in this PR.
Our automated tests already help mitigate the risk, but it's important
that code owners thoroughly review the changes in their area and
consider if they could have unintended consequences. The Platform
Security team should also review this PR thoroughly, especially since
some changes were made to platform code around privilege handling. The
Data Discovery team will also manually test the behaviour when upgrading
existing user roles with deprecated feature privileges as part of 9.0
upgrade testing.
---------
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: “jeramysoucy” <jeramy.soucy@elastic.co>
## Summary
Adding a note that for logs, in case the name of the filebeat index was
changed, it's needed to adjust `monitoring.ui.logs.index` so that the
logs stats can be retrieved as well
Fixes#201269
## Summary
* Extracts the usage of the EMS styles identifiers to always refer to
the constants at `ems_defaults.ts`
* Adds logic in the Vega and Maps plugins to resolve different styles
depending on the theme, bringing the new `road_map_desaturated_v9` and
`dar_kmap_v9` styles when Borealis theme is enabled.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- Screenshots will be updated as a larger effort afterwards.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- Changes were small enough for this to not require new tests.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
## Release note
Elastic Maps Service basemaps shown in the Custom Visualization
component (Vega) and in the Maps application and components alighn with
the new dark and light theme.
---------
Co-authored-by: Nick Partridge <nick.ryan.partridge@gmail.com>
### Summary
For migration purposes, we need to remove complex tables in the docs. If
we ultimately decide that we need complex tables to support certain
documentation use-cases, we can add that functionality to docs-builder
later.
For
https://github.com/elastic/docs-builder/issues/112#issuecomment-2599284112.
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
## Summary
Resolves: https://github.com/elastic/kibana/issues/151463
Removes all reference to ephemeral tasks from the task manager plugin.
As well as unit and E2E tests while maintaining backwards compatibility
for `xpack.task_manager.ephemeral_tasks` flag to no-op if set. This PR
has some dependencies from the PR to remove ephemeral task support from
the alerting and actions plugin
(https://github.com/elastic/kibana/pull/197421). So it should be merged
after the other PR.
Deprecates the following configuration settings:
- xpack.task_manager.ephemeral_tasks.enabled
- xpack.task_manager.ephemeral_tasks.request_capacity
The user doesn't have to change anything on their end if they don't wish
to. This deprecation is made so if the above settings are defined,
kibana will simply do nothing.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
This PR updates a few references to hosted deployments on Elastic Cloud
to avoid ambiguity with serverless in anticipation of serverless GA.
Some more similar updates will be made on other repos to align.
Rel: https://github.com/elastic/platform-docs-team/issues/485
This PR resets the release notes, upgrade notes, and what's new for 9.0.
It also cleans up a few references/files that were focusing on migration
to 8.0
Some more PRs will happen to prepare the rest of the docs for v9
Closes: https://github.com/elastic/platform-docs-team/issues/564
Closes https://github.com/elastic/kibana/issues/197370
### Test instructions
1) open new kibana installation
2) verify canvas is not available in menu or application search bar
3) use saved object import to import canvas workpad. Reload browser
4) verify canvas is available in menu and application search bar
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Close https://github.com/elastic/kibana/issues/193473
Close https://github.com/elastic/kibana/issues/193474
This PR utilize the documentation packages that are build via the tool
introduced by https://github.com/elastic/kibana/pull/193847, allowing to
install them in Kibana and expose documentation retrieval as an LLM task
that AI assistants (or other consumers) can call.
Users can now decide to install the Elastic documentation from the
assistant's config screen, which will expose a new tool for the
assistant, `retrieve_documentation` (only implemented for the o11y
assistant in the current PR, shall be done for security as a follow up).
For more information, please refer to the self-review.
## General architecture
<img width="1118" alt="Screenshot 2024-10-17 at 09 22 32"
src="https://github.com/user-attachments/assets/3df8c30a-9ccc-49ab-92ce-c204b96d6fc4">
## What this PR does
Adds two plugin:
- `productDocBase`: contains all the logic related to product
documentation installation, status, and search. This is meant to be a
"low level" components only responsible for this specific part.
- `llmTasks`: an higher level plugin that will contain various LLM tasks
to be used by assistants and genAI consumers. The intent is not to have
a single place to put all llm tasks, but more to have a default place
where we can introduce new tasks from. (fwiw, the `nlToEsql` task will
probably be moved to that plugin).
- Add a `retrieve_documentation` tool registration for the o11y
assistant
- Add a component on the o11y assistant configuration page to install
the product doc
(wiring the feature to the o11y assistant was done for testing purposes
mostly, any addition / changes / enhancement should be done by the
owning team - either in this PR or as a follow-up)
## What is NOT included in this PR:
- Wire product base feature to the security assistant (should be done by
the owning team as a follow-up)
- installation
- utilization as tool
- FTR tests: this is somewhat blocked by the same things we need to
figure out for https://github.com/elastic/kibana-team/issues/1271
## Screenshots
### Installation from o11y assistant configuration page
<img width="1476" alt="Screenshot 2024-10-17 at 09 41 24"
src="https://github.com/user-attachments/assets/31daa585-9fb2-400a-a2d1-5917a262367a">
### Example of output
#### Without product documentation installed
<img width="739" alt="Screenshot 2024-10-10 at 09 59 41"
src="https://github.com/user-attachments/assets/993fb216-6c9a-433f-bf44-f6e383d20d9d">
#### With product documentation installed
<img width="718" alt="Screenshot 2024-10-10 at 09 55 38"
src="https://github.com/user-attachments/assets/805ea4ca-8bc9-4355-a434-0ba81f8228a9">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR updates the first few pages of the Discover docs:
- Splits most of the content of the parent page into a child page to
match with the structure of the Dashboards docs
- Refocuses some of the content so that it looks generally applicable to
users and their data rather than a sample data-based tutorial
- Adds missing content flagged in
https://github.com/elastic/kibana/issues/192149
- Adds some content updates specific to 8.16 flagged in
https://github.com/elastic/platform-docs-team/issues/500 (I still need
to figure out what's the best way to document ES|QL stuff)
Rel: https://github.com/elastic/platform-docs-team/issues/500
Closes: https://github.com/elastic/kibana/issues/192149
Closes: https://github.com/elastic/kibana/issues/162265
Closes: https://github.com/elastic/kibana/issues/187417
Note: While there may be more missing features that we want to document,
let's first get these updates in.
Note2: If you're wondering where to preview the changes, find the
preview link in the Github actions comment and navigate to the "master"
version of the Kibana guide.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>
Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
This PR:
- updates navigation instructions to accommodate for the navigation
changes related to solution views.
- updates instructions for adding sample data to rely on the
integrations page instead of the home page, that only exists with the
classic solution view
- updates references to the home page to avoid confusing users using one
of the new solution views
Closes: https://github.com/elastic/platform-docs-team/issues/529
Closes: https://github.com/elastic/platform-docs-team/issues/540
### Overview
This PR updates all app and page references to align with the new style
guide guidelines.
### Related issue
https://github.com/elastic/search-docs-team/issues/205
### Preview
Machine learning
AIOps Labs
---------
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
## Summary
Added a Duplicate dashboards page for the general duplicating process
and then mentioned the new popover for duplicating managed dashboards.
Linked to this page from other relevant mentions.
This PR recreates [@aakash742's
PR](https://github.com/elastic/kibana/pull/196497) to add a note about
Elastic not providing support for community plugins
---------
Co-authored-by: akashsingh <55509676+aakash742@users.noreply.github.com>
Adds a new `MlAuditLogger` service for logging calls to elasticsearch in
kibana's audit log.
Not all calls are logged, only ones which make changes to ML jobs or
trained models, e.g. creating, deleting, starting, stopping etc.
Calls to the es client are wrapped in a logging function so successes
and failures can be caught and logged.
the audit log can be enabed by adding this to the kibana yml or dev.yml
file
`xpack.security.audit.enabled: true`
An example log entry (NDJSON formatted to make it readable):
```
{
"event": {
"action": "ml_start_ad_datafeed",
"type": [
"change"
],
"category": [
"database"
],
"outcome": "success"
},
"labels": {
"application": "elastic/ml"
},
"user": {
"id": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
"name": "elastic",
"roles": [
"superuser"
]
},
"kibana": {
"space_id": "default",
"session_id": "U6HQCDkk+fAEUCXs7i4qM2/MZITPxE02pp8o7h09P68="
},
"trace": {
"id": "4f1b616b-8535-43e1-8516-32ea9fe76d19"
},
"client": {
"ip": "127.0.0.1"
},
"http": {
"request": {
"headers": {
"x-forwarded-for": "127.0.0.1"
}
}
},
"service": {
"node": {
"roles": [
"background_tasks",
"ui"
]
}
},
"ecs": {
"version": "8.11.0"
},
"@timestamp": "2024-10-11T09:07:47.933+01:00",
"message": "Starting anomaly detection datafeed datafeed-11aaaa",
"log": {
"level": "INFO",
"logger": "plugins.security.audit.ecs"
},
"process": {
"pid": 58305,
"uptime": 100.982390291
},
"transaction": {
"id": "77c14aadc6901324"
}
}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Create Search landing page with links to all features available in the
Elastic Cloud hosted UI
- Create skeleton of a table to quickly provide links to all relevant
docs and release notes for each feature
- Currently commented out, to be filled in a follow-up PR by @kosabogi
- Create a stub page for connection details page (Docs equivalent to
[find your CloudID and create API
keys](https://www.elastic.co/search-labs/tutorials/install-elasticsearch/elastic-cloud#finding-your-cloud-id))
- @kosabogi will update this page in follow-up
- Nest existing Playground docs under there
- Create a stub page for AI Assistant for Search docs
- Create a stub page for inference endpoints UI
Towards https://github.com/elastic/kibana/issues/190095,
https://github.com/elastic/kibana/issues/192183,
https://github.com/elastic/kibana/issues/192185
## Summary
This PR updates the following:
- heap-to-capacity converter to take into account larger amounts of RAM,
updated this to be 16GB
- initial`maxAllowedCost` to be the default capacity of 10
- adds `xpack.alerting.maxScheduledPerMinute`,
`xpack.discovery.active_nodes_lookback`, `xpack.discovery.interval`
configs to docker
- updates the TM docs for `xpack.task_manager.capacity`
---------
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
## Summary
[redo of #191129 that got reverted]
This PR updates the structure of the Dashboards docs and refreshes some
outdated parts of the content.
More updates will be made in future PRs to refresh screenshots and to
refresh the content more in depth.
This new structure and edits:
- distribute the pages in more user-oriented identified themes, for
better findability, scanning, and to ease possible integration of some
of these pages into in-app documentation.
- are more future proof to evolve along with upcoming features.
~I'll leave this PR as a draft until I resolve some link dependencies
coming from other docs sources and check some additional bits of
content.~
Preview available on demand on Slack.
Closes: https://github.com/elastic/platform-docs-team/issues/408 (I'll
create separate issues for remaining items)
Closes: https://github.com/elastic/platform-docs-team/issues/413
Closes: https://github.com/elastic/platform-docs-team/issues/418
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
