## Summary
This PR reworks saved query privileges to rely solely on a single global
`savedQueryManagement` privilege, and eliminates app-specific overrides.
This change simplifies the security model for users, fixes bugginess in
the saved query management UI, and reduces code complexity associated
with maintaining two separate security mechanisms (app-specific
overrides and global saved query management privileges).
### Background
Saved queries allow users to store a combination of KQL or Lucene
queries, filters, and time filters to use across various applications in
Kibana. Access to saved query saved objects are currently granted by the
following feature privileges:
```json
[
"feature_discover.all",
"feature_dashboard.all",
"feature_savedQueryManagement.all",
"feature_maps.all",
"feature_savedObjectsManagement.all",
"feature_visualize.all"
]
```
There is also a saved query management UI within the Unified Search bar
shared by applications across Kibana:
<img
src="https://github.com/user-attachments/assets/e4a7539b-3dd4-4d47-9ff8-205281ef50e3"
width="500" />
The way access to this UI is managed in Kibana is currently confusing
and buggy:
- If a user has `feature_discover.all` and `feature_dashboard.all` they
will be able to load and save queries in Discover and Dashboard.
- If a user has `feature_discover.all` and `feature_dashboard.read` they
will be able to load queries in both Discover and Dashboard, but only
save queries in Discover (even though they have write access to the SO,
and API access). Instead they have to navigate to Discover to save a
query before navigating back to Dashboard to load it, making for a
confusing and frustrating UX.
- Access to the UI is even more confusing in apps not listed in the
above feature privileges (e.g. alerting, SLOs). Some of them chose to
check one of the above feature privileges, meaning users who otherwise
should have saved query access won't see the management UI if they don't
also have the exact feature privilege being checked. Other apps just
always show the management UI, leading to bugs and failures when users
without one of the above feature privileges attempt to save queries.
### Existing improvements
In v8.11.0, we introduced a new ["Saved Query
Management"](https://github.com/elastic/kibana/pull/166937) privilege,
allowing users to access saved queries across all of Kibana with a
single global privilege:
<img
src="https://github.com/user-attachments/assets/ccbe79a4-bd0b-4ed6-89c9-117cc1f99ee2"
width="600" />
When this privilege is added to a role, it solves the
`feature_discover.all` and `feature_dashboard.read` issue mentioned
above. However, it does not fix any of the mentioned issues for roles
without the new privilege. We have so far postponed further improvements
to avoid a breaking change.
### Approach
To fully resolve these issues and migrate to a single global privilege,
these changes have been made:
- Remove saved query SO access from all application feature privileges
and instead only allow access through the global saved query management
privilege.
- Stop relying on application feature privileges for toggling the saved
query management UI, and instead rely on the global privilege.
To implement this with minimal breaking changes, we've used the Kibana
privilege migration framework. This allows us to seamlessly migrate
existing roles containing feature privileges that currently provide
access to saved queries, ensuring they are assigned the global saved
query management privilege on upgrade.
As a result, we had to deprecate the following feature privileges,
replacing them with V2 privileges without saved query SO access:
```json
[
"feature_discover.all",
"feature_dashboard.all",
"feature_maps.all",
"feature_visualize.all"
]
```
Each area of code that currently relies on any of these feature
privileges had to be updated to instead access `feature_X_V2` instead
(as well as future code).
This PR still introduces a minor breaking change, since users who have
`feature_discover.all` and `feature_dashboard.read` are now able to save
queries in Dashboard after upgrade, but we believe this is a better UX
(and likely the expected one) and worth a small breaking change.
### Testing
- All existing privileges should continue to work as they do now,
including deprecated V1 feature privileges and customized serverless
privileges. There should be no changes for existing user roles apart
from the minor breaking change outlined above.
- Check that code changes in your area don't introduce breaking changes
to existing behaviour. Many of the changes are just updating client UI
capabilities code from `feature.privilege` to `feature_v2.privilege`,
which is backward compatible.
- The `savedQueryManagement` feature should now globally control access
to saved query management in Unified Search for all new user roles.
Regardless of privileges for Discover, Dashboard, Maps, or Visualize,
new user roles should follow this behaviour:
- If `savedQueryManagement` is `none`, the user cannot see or access the
saved query management UI or APIs.
- If `savedQueryManagement` is `read`, the user can load queries from
the UI and access read APIs, but cannot save queries from the UI or make
changes to queries through APIs.
- If `savedQueryManagement` is `all`, the user can both load and save
queries from the UI and through APIs.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
This PR risks introducing unintended breaking changes to user privileges
related to saved queries if the deprecated features have not been
properly migrated, and users could gain or lose access to saved query
management on upgrade. This would be bad if it happened, but not overly
severe since it wouldn't grant them access to any ES data they couldn't
previously access (only query saved objects). We have automated testing
in place to help ensure features have been migrated correctly, but the
scope of these changes are broad and touch many places in the codebase.
Additionally, the UI capabilities types are not very strict, and are
referenced with string paths in many places, which makes changing them
riskier than changing strictly typed code. A combination of regex
searches and temporarily modifying the `Capabilities` type to cause type
errors for deprecated privileges was used to identify references in
code. Reviewers should consider if there are any other ways that UI
capabilities can be referenced which were not addressed in this PR.
Our automated tests already help mitigate the risk, but it's important
that code owners thoroughly review the changes in their area and
consider if they could have unintended consequences. The Platform
Security team should also review this PR thoroughly, especially since
some changes were made to platform code around privilege handling. The
Data Discovery team will also manually test the behaviour when upgrading
existing user roles with deprecated feature privileges as part of 9.0
upgrade testing.
---------
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: “jeramysoucy” <jeramy.soucy@elastic.co>
## Summary
Adding custom `ScoutLogger` class to use across its services so that we
can better read logs related only to test framework runner.
We can also later migrate from `ToolingLog` to something better
integrated with Playwright to unify logs from different levels
https://github.com/elastic/kibana/issues/203591
To make sure we use the same instance, I converted few core services
(logger, kbnClient, esClient, esArchiver) to singletons.
Log output example:
```
Running 1 test using 1 worker
› should allow removing the dashboard panel after the underlying saved search has been deleted @svlSecurity @svlOblt @svlSearch @ess
debg [scout] [service] logger
info [scout] [config] Reading test servers confiuration from file: /Users/dmle/github/kibana/.scout/servers/local.json
debg [scout] [service] config
debg [scout] [service] esClient
debg [scout] [service] kbnClient
debg [scout] [service] esArchiver
debg [scout] [service] uiSettings
debg [scout] Requesting url (redacted): [http://localhost:5620/api/status]
info [scout] [x-pack/test/functional/es_archives/logstash_functional] Loading "mappings.json"
info [scout] [x-pack/test/functional/es_archives/logstash_functional] Loading "data.json.gz"
info [scout] [x-pack/test/functional/es_archives/logstash_functional] Skipped restore for existing index "logstash-2015.09.22"
info [scout] [x-pack/test/functional/es_archives/logstash_functional] Skipped restore for existing index "logstash-2015.09.20"
info [scout] [x-pack/test/functional/es_archives/logstash_functional] Skipped restore for existing index "logstash-2015.09.21"
```
## Summary
Added support for human readable `name` attribute for saved objects
audit.
- Updated the saved object type/registration with `nameAttribute` option
- Updated the Saved Objects Security Extension to support passing object
names to the audit functions
- Updated the audit logger with a configuration option to opt out of
including saved object names (the SOR and SSC should be aware of this to
avoid operations when they are not necessary)
- Updated Saved Object Repository functions
- [x] `bulkCreate`
- [x] `bulkGet`
- [x] `bulkResolve`
- [x] `bulkUpdate`
- [x] `collectMultiNamespaceReferences`
- [x] `get`
- [x] `updateObjectsSpaces`
- [x] `bulkDelete`
- [x] `delete`
- [x] `removeReferencesTo`
- [x] Updated Secure Spaces Client functions
- [x] `auditObjectsForSpaceDeletion`
Functions that were not updated:
- `authorizeFind` - now we log audit before the actual find with only
types. Find is complex one, that can return a lot of saved objects. The
benefit of adding a separate audit event vs potential performance cost
can be considered negligible.
2f6b9f67d8/src/core/packages/saved-objects/api-server-internal/src/lib/apis/find.ts (L166)
- `deleteByNamespace` - doesn't have an audit log itself, however is
used only along with the `delete` which adds audit log with SO name
2f6b9f67d8/x-pack/platform/plugins/shared/spaces/server/spaces_client/spaces_client.ts (L223-L225)
- `checkConflicts` - audit was intensionally bypassed
2f6b9f67d8/x-pack/platform/plugins/shared/security/server/saved_objects/saved_objects_security_extension.ts (L945-L948)
- `disableLegacyUrlAliases` - function calls `bulkUpdate` in the end
(which adds audit log with SO name already). Adding name to the
`disableLegacyUrlAliases` audit log, will result in double saved objects
get operation which is not feasible.
2f6b9f67d8/x-pack/platform/plugins/shared/spaces/server/spaces_client/spaces_client.ts (L228-L234)
## How to test
Best way to test it is from the `Manage Saved Objects` page with audit
enabled.
- Import some test data set from the main page.
- Go to the `Manage Saved Objects`:
- Update single SO
- Delete singe SO
- Bulk update SOs
- Bulk delete SOs
- Import/export SOs
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Release note
Added support for human readable name attribute for saved objects audit
events.
__Closes: https://github.com/elastic/kibana/issues/200538__
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
- Centralized Scout reporter settings
- Added owner area and config/test file information to reporter events
- Attempt to upload events at the end of a test run
- Enable Scout reporter test events upload for the `pull request` and
`on merge` pipelines
## Summary
1. Extends the server-side prototype pollution protections introduced in
https://github.com/elastic/kibana/pull/190716 to include
`Array.prototype`.
2. Applies the same prototype pollution protections to the client-side.
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] Sealing prototypes on the client can lead to failures in
third-party dependencies. I'm relying on sufficient functional test
coverage to detect issues here. As a result, these protections are
disabled by default for now, and can be controlled via setting
`server.prototypeHardening: true/false`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR resolves#194605 and closes#170132 and brings the following
changes:
- changed ELU metrics evaluation used for autoscaling;
- a rate limiter to throttle incoming requests when under a high load;
- a configuration option to exclude some routes from the rate limiter.
## Summary
sometimes we face failure during saml authentication and error message
is not very helpful:
```
Error: Failed to parse 'set-cookie' header
at getCookieFromResponseHeaders (packages/kbn-test/src/auth/saml_auth.ts:61:11)
at finishSAMLHandshake (packages/kbn-test/src/auth/saml_auth.ts:280:10)
at createLocalSAMLSession (packages/kbn-test/src/auth/saml_auth.ts:333:18)
```
With this change we should know when it happened:
- we create SAML request by calling `/internal/security/login
- we finish SAML handshake by calling `/api/security/saml/callback`
I also hope `response.data` to be useful for investigation
## Summary
As K8S Dashboard is currently hidden on main , the code serves no
purpose other than potentially causing Tech debts whenever a refactor or
a migration happens. As such its better to remove it completely. In case
we want to bring it back later we will just pull it from git history
> [!CAUTION]
> **This should only affect Serverless and Main, 8.x.x should still be
able to see and access K8S Dashboard**
## Related Tickets
- https://github.com/elastic/security-team/issues/11418
- https://github.com/elastic/security-team/issues/10735
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Paulo Silva <paulo.henrique@elastic.co>
Co-authored-by: Maxim Kholod <maxim.kholod@elastic.co>
## Summary
This PR adds `spaceTest` interface to `kbn-scout` to run space aware
tests, that can be executed in parallel. Most of Discover tests were
converted to parallel run because we see runtime improvement with 2
parallel workers.
Experiment 1: **ES data pre-ingested**, running 9 Discover **stateful**
tests in **5 files** locally
| Run setup | Took time |
| ------------- | ------------- |
| 1 worker | `1.3` min |
| 2 workers | `58.7` sec |
| 3 workers | `48.3` sec |
| 4 workers | **tests fail** |
Conclusion: using **2** workers is the optimal solution to continue
Experiment 2: Running Discover tests for stateful/serverless in **Kibana
CI** (starting servers, ingesting ES data, running tests)
| Run setup | 1 worker | 2 workers | diff
| ------------- | ------------- |------------- |------------- |
| stateful, 9 tests / 5 files | `1.7` min | `1.2` min | `-29.4%`|
| svl ES, 8 tests / 4 files | `1.7` min | `1.3` min | `-23.5%`|
| svl Oblt, 8 tests / 4 files | `1.8` min | `1.4` min | `-22.2%`|
| svl Search, 5 tests / 2 files | `59.9` sec | `51.6` sec | `-13.8%`|
Conclusion: parallel run effectiveness benefits from tests being split
in **more test files**.
Experiment 3: Clone existing tests to have **3 times more test files**
and re-run tests for stateful/serverless in **Kibana CI** (starting
servers, ingesting ES data, running tests)
| Run setup | 1 worker | 2 workers | diff
| ------------- | ------------- |------------- |------------- |
| stateful, 27 tests / 15 files | `4.3` min | `2.7` min | `-37.2%`|
| svl ES, 24 tests / 12 files | `4.3` min | `2.7` min | `-37.2%`|
Conclusion: parallel run effectiveness is **increasing** with more test
files in place, **not linear** but with good test design we can expect
**up to 40%** or maybe a bit more.
How parallel run works:
- `scoutSpace` fixture is loaded on Playwright worker setup (using
`auto: true` config), creates a new Kibana Space, expose its id to other
fixtures and deletes the space on teardown.
- `browserAuth` fixture for parallel run caches Cookie per worker/space
like `role:spaceId`. It is needed because Playwright doesn't spin up new
browser for worker, but only new context.
- kbnClient was updated to allow passing `createNewCopies: true` in
query, it is needed to load the same Saved Objects in parallel
workers/spaces and generate new ids to work with them. `scoutSpace`
caches ids and allows to reach saved object by its name. This logic is
different from single thread run, where we can use default ids from
kbnArchives.
How to run parallel tests locally, e.g. for stateful:
```
node scripts/scout run-tests --stateful --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts
```
Clean up visualizations page load size by
* lazy loading actions
* avoid exporting from index files to avoid exporting unused code
* move `urlFor` and `getFullPath` into `url_utils` to avoid including
`utils/saved_visualize_utils` in page load bundle
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Closes https://github.com/elastic/logs-dev/issues/183,
https://github.com/elastic/logs-dev/issues/184 and
https://github.com/elastic/logs-dev/issues/185.
## Summary
This PR aims to support failure store in dataset quality page. The
following acceptance criteria items were resolved
### Dataset quality page
- [x] A column for Failed docs is included in the table
- [x] A tooltip is placed in the title of the column
- [x] A % of documents inside Failure store is calculated for every
dataStream
- [x] If % is lesser than 0.0001 but greater than 0 we should show ⚠
symbol next to the ~0 value (as we do with degraded docs)
- [x] Failed docs percentages greater than 0 should link to discover
🎥 Demo
https://github.com/user-attachments/assets/6d9e3f4c-02d9-43ab-88cb-ae70716b05d9
### Dataset details page
- [x] A metric, Failed docs, is included in the Overview panel under
Data set quality. This metric includes the number of documents inside
the failure store for the specific dataStream.
- [x] A tooltip is placed in the title of the Failed docs metric with
message: `The percentage of docs sent to failure store due to an issue
during ingestion.`
- [x] Degraded docs graph section is transformed to Document trends
allowing the users to switch between Degraded docs and Failed docs
trends over time.
- [x] A new chart for failed documents is created with links to
discover/Logs explorer using the right dataView
🎥 Demo
https://github.com/user-attachments/assets/6a3a1f09-2668-4e83-938e-ecdda798c199
### Failed docs ingestion issue flyout
- [x] Whenever documents are found in failure store we should list
Document indexing failed in Quality issues table
- [x] User should be able to expand Document indexing failed and see
more information in the flyout
- [x] The flyout will show Docs count, an aggregation of the number of
documents inside failure store for the selected timeframe
- [x] The flyout will show Last ocurrence, the datetime registered for
the most recent document in the failure store.
- [x] The flyout will contain a section called Error messages where a
list of unique error messages should be shown, exposing Content (error
message) and Type (Error Type).
- [x] Type should contain a tooltip where message (`Error message
category`) explain users how we are categorising the errors.
- [x] Other issues inside Quality issues table will be appended by field
ignored and the field will be shown in bold.
https://github.com/user-attachments/assets/94dc81f0-9720-4596-b256-c9d289cefd94
Note: This PR was reconstructed from
https://github.com/elastic/kibana/pull/199806 which it supersedes.
## How to test
1. Execute `failed_logs` synthtrace scenario
2. Open dataset quality page
## Follow ups
- Enable in serverless
- Deployment agnostic tests cannot be added until we enable this in
serverless
- FTR tests will be added as part of
https://github.com/elastic/logs-dev/issues/182
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Adds a new section to the overview tab in the log details flyout in
Discover to display stacktrace information for logs and exceptions.
In a follow-up, the stacktrace could be moved to a new tab in the log
details flyout and actions can be added to the stacktrace (and quality)
icons in the document table to open the relevant sections in the flyout.
Closes https://github.com/elastic/kibana/issues/190460
### APM - Log stacktrace (library frames)
<img width="1470" alt="image"
src="https://github.com/user-attachments/assets/8991f882-d329-4bc5-aa37-424576bcee72"
/>
### APM - Exception (with cause)
<img width="1476" alt="image"
src="https://github.com/user-attachments/assets/cfbf24a7-6f82-48f1-b275-5aac977411ac"
/>
### APM - Exception (simple stacktrace)
<img width="1474" alt="image"
src="https://github.com/user-attachments/assets/fc0306c4-5fcd-4b74-bb0d-c1784a48d677"
/>
### Apache Tomcat Integration (Catalina) - Stacktrace
<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/281f1822-faea-4e2d-9515-c11a9ee12f50"
/>
## 📝 Notes for reviewers
- The `@kbn/apm-types` package was marked as platform / shared as it's
being used by the
[unified_doc_viewer](https://github.com/elastic/kibana/blob/main/src/plugins/unified_doc_viewer/kibana.jsonc)
- The code used to render stacktraces in APM was moved into a new
`@kbn/event-stacktrace` package as it is reused in the
`unified_doc_viewer`
- The code used to render metadata table in APM was moved into a new
`@kbn/key-value-metadata-table` package
## 🧪 Testing instructions
The deployed environments have sample logs that can be used (time range:
Jan 1, 2025 - now). For a local setup, please follow the instructions
below:
1. Ingest sample logs with stacktraces
([gist](https://gist.github.com/gbamparop/0da21ca7f65b24c4a9c071ce9e9b97b0)).
Please note that these are test data and some fields that are not used
by stacktraces might not be consistent
2. View relevant logs in Discover (Query: `service.name: "synth-node-0"
OR apache_tomcat :*`, Time range: Jan 1, 2025 - now)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/206967
## Summary
After some changes related to V2 migration of getting the entities,
there was an issue with the new data coming from the endpoint - the
`data_stream.type` is a string instead of an array in case of a single
data stream so this PR adds a fix to support that (and a test)
## Bug fixes
- Service overview page loads for a logs-only data stream
- After adding the fix, I saw another error related to the `useTheme`
and changed it to use the `euiTheme` similar to the other changes
related to the Borealis team upgrade
## Testing
To test the scenario with services and hosts coming from logs (without
APM / metrics) I added a new scenario in synthtrace so to test then we
should:
- Run the new scenario: `node scripts/synthtrace logs_only` (if possible
on a clean ES)
- Enable `observability:entityCentricExperience` in Stack Management >
Advanced Setting
- Go to Inventory and click on a service
- The logs-only views should be available
- Go to Inventory and click on a host
- The logs-only views should be available
https://github.com/user-attachments/assets/cfd5fd40-ac44-4807-9a29-f3ee3015d814
- Test one of the scenarios with mix of APM/metrics/logs
- Run `node scripts/synthtrace infra_hosts_with_apm_hosts`
- Enable `observability:entityCentricExperience` in Stack Management >
Advanced Setting
- Go to Inventory and click on a service from APM
- The APM views (service/traces) should be available
- Go to Inventory and click on a host
- The asset details view should be available and show metrics
https://github.com/user-attachments/assets/894c7c1a-aaa1-42cb-9dcb-05c9a5ca8177
- Infrastructure (Inventory/Hosts, etc) and Applications (Service
Inventory/Traces, etc) should load the data for this scenario and not
for the logs only (also for an oblt cluster connection)
https://github.com/user-attachments/assets/4d092cc6-a8ad-4022-b980-b443be09acc9
Resolves https://github.com/elastic/eui-private/issues/171
Resolves https://github.com/elastic/eui-private/issues/177
## Summary
This PR addresses a prior PR review
[comment](https://github.com/elastic/kibana/pull/203840/files#diff-bb850523655bac7adb30995553acabae9705435fa51e5b8bf13c483152db694a)
by removing `isServerless` from the logic determining what theme should
be used at runtime with a simple YML configuration setting instead.
I added a non-public `uiSettings.experimental.defaultTheme` config
property that defaults to `borealis` and is set to `amsterdam` in
`serverless.yml`. Since the default theme is now (and should be) set to
Borealis, I also updated `DEFAULT_THEME_NAME` and `FALLBACK_THEME_NAME`
to reflect that. This doesn't have any impact on Serverless; it will
keep using Amsterdam.
Additionally, while making these changes, I wanted to simultaneously
improve types and address earlier PR
[comment](https://github.com/elastic/kibana/pull/199748#discussion_r1840402343).
Now `SUPPORTED_THEME_NAMES` array is declared as `const` making the
`ThemeName` type strict instead of resolving a generic `string` type.
Usages were updated to use `ThemeName` instead of `string`, too.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
* Added a few transforms to simplify package paths.
* Fixed typo causing `.mdx` files to not be processed when replacing
references.
* Added preliminary support for `--healthcheck` (to check for broken
references to files and links).
## Summary
Epic: https://github.com/elastic/security-team/issues/7998
In this PR we're breaking out the `timeline` and `notes` features into
their own feature privilege definition. Previously, access to both
features was granted implicitly through the `siem` feature. However, we
found that this level of access control is not sufficient for all
clients who wanted a more fine-grained way to grant access to parts of
security solution.
In order to break out `timeline` and `notes` from `siem`, we had to
deprecate it feature privilege definition for. That is why you'll find
plenty of changes of `siem` to `siemV2` in this PR. We're making use of
the feature privilege's `replacedBy` functionality, allowing for a
seamless migration of deprecated roles.
This means that roles that previously granted `siem.all` are now granted
`siemV2.all`, `timeline.all` and `notes.all` (same for `*.read`).
Existing users are not impacted and should all still have the correct
access. We added tests to make sure this is working as expected.
Alongside the `ui` privileges, this PR also adds dedicated API tags.
Those tags haven been added to the new and previous version of the
privilege definitions to allow for a clean migration:
```mermaid
flowchart LR
subgraph v1
A(siem) --> Y(all)
A --> X(read)
Y -->|api| W(timeline_write / timeline_read / notes_read / notes_write)
X -->|api| V(timeline_read /notes_read)
end
subgraph v2
A-->|replacedBy| C[siemV2]
A-->|replacedBy| E[timeline]
A-->|replacedBy| G[notes]
E --> L(all)
E --> M(read)
L -->|api| N(timeline_write / timeline_read)
M -->|api| P(timeline_read)
G --> Q(all)
G --> I(read)
Q -->|api| R(notes_write / notes_read)
I -->|api| S(notes_read)
end
```
### Visual changes
#### Hidden/disabled elements
Most of the changes are happening "under" the hood and are only
expressed in case a user has a role with `timeline.none` or
`notes.none`. This would hide and/or disable elements that would usually
allow them to interact with either timeline or the notes feature (within
timeline or the event flyout currently).
As an example, this is how the hover actions look for a user with and
without timeline access:
| With timeline access | Without timeline access |
| --- | --- |
| <img width="616" alt="Screenshot 2024-12-18 at 17 22 49"
src="https://github.com/user-attachments/assets/a767fbb5-49c8-422a-817e-23e7fe1f0042"
/> | <img width="724" alt="Screenshot 2024-12-18 at 17 23 29"
src="https://github.com/user-attachments/assets/3490306a-d1c3-41aa-af5b-05a1dd804b47"
/> |
#### Roles
Another visible change of this PR is the addition of `Timeline` and
`Notes` in the edit-role screen:
| Before | After |
| ------- | ------ |
| <img width="746" alt="Screenshot 2024-12-12 at 16 31 43"
src="https://github.com/user-attachments/assets/20a80dd4-c214-48a5-8c6e-3dc19c0cbc43"
/> | <img width="738" alt="Screenshot 2024-12-12 at 16 32 53"
src="https://github.com/user-attachments/assets/afb1eab4-1729-4c4e-9f51-fddabc32b1dd"
/> |
We made sure that for migrated roles that hard `security.all` selected,
this screen correctly shows `security.all`, `timeline.all` and
`notes.all` after the privilege migration.
#### Timeline toast
There are tons of places in security solution where `Investigate / Add
to timeline` are shown. We did our best to disable all of these actions
but there is no guarantee that this PR catches all the places where we
link to timeline (actions). One layer of extra protection is that the
API endpoints don't give access to timelines to users without the
correct privileges. Another one is a Redux middleware that makes sure
timelines cannot be shown in missed cases. The following toast will be
shown instead of the timeline:
<img width="354" alt="Screenshot 2024-12-19 at 10 34 23"
src="https://github.com/user-attachments/assets/1304005e-2753-4268-b6e7-bd7e22d8a1e3"
/>
### Changes to predefined security roles
All predefined security roles have been updated to grant the new
privileges (in ESS and serverless). In accordance with the migration,
all roles with `siem.all` have been assigned `siemV2.all`,
`timeline.all` and `notes.all` (and `*.read` respectively).
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: PhilippeOberti <philippe.oberti@elastic.co>
Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
## Summary
Extracted remaining easy backward-compatible unit test fixes that fail
with React@18 from https://github.com/elastic/kibana/pull/206411
The idea is that the tests should pass for both React@17 and React@18
## Summary
Only log out the number of attempts when the `retryCount` is truthy
Previously we were seeing the attempt counter, constantly reporting 0
for each attempt.
### To Run Locally
```
node scripts/jest --config packages/kbn-ftr-common-functional-services/jest.config.js
```
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
While looking at the `packages` folder at the root of Kibana, I noticed
some files were left over in otherwise empty folders:
- 2 README files were left in the `content-management` folder
- 1 README file and 1 png file were left in the `react` folder
The rest of the content was moved to a new location as part of the
Sustainable Kibana Architecture effort (see [this
PR](https://github.com/elastic/kibana/pull/205593) and [that
one](https://github.com/elastic/kibana/pull/205924)) and I wonder if
those few files were left behind by mistake.
I did not making any changes to the content of the files, I just moved
them to their respective new locations.
Please let me know if these were left behind intentionally, or if they
should be deleted instead of moved!
### Notes
The `appex-sharedux` codeowner only appeared after pushing the second
commit which impacts the `react` folder. I realized that the codeowners
file was pointing to the folder within
`src/platform/packages/shared/content-management/content_insights` and
`src/platform/packages/shared/content-management/favorites` so update it
to point to the parent folder, which now contains the moved README
files. I hope that's ok!
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- Updates `scripts/dependency_ownership` to use the
`@kbn/dev-cli-runner` for consistency with other CI-related CLIs.
- Adds a new `failIfUnowned` flag to exit with an error code if any
dependencies are unowned.
- Adds a new dependency ownership check to `quick_checks` and `renovate`
CI steps.
From a CI run, the additional quick check executes successfully in 3
seconds:
```sh
info [quick-checks] Passed check: /opt/buildkite-agent/builds/bk-agent-prod-gcp-abc123/elastic/kibana-pull-request/kibana/.buildkite/scripts/steps/checks/dependencies_missing_owner.sh in 3s
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* move actions to `registerActionAsync`
* remove global string file loaded in page load bundle
* Break constants into smaller files so only constants required by page
load bundle are exposed.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Extending scout-reporter with `failed-test-reporter`, that saves
failures in json summary file. For each test failure html report file is
generated and linked in summary report:
```
[
{
"name": "stateful - Discover app - saved searches - should customize time range on dashboards",
"htmlReportFilename": "c51fcf067a95b48e2bbf6098a90ab14.html"
},
{
"name": "stateful - Discover app - value suggestions: useTimeRange enabled - dont show up if outside of range",
"htmlReportFilename": "9622dcc1ac732f30e82ad6d20d7eeaa.html"
}
]
```
This PR updates `failed_tests_reporter_cli` to look for potential Scout
test failures and re-generate test failure artifacts in the same format
we already use for FTR ones.
These new artifacts are used to list failures in BK annotation:
<img width="1092" alt="image"
src="https://github.com/user-attachments/assets/09464c55-cdaa-45a4-ab47-c5f0375b701c"
/>
test failure html report example:
<img width="1072" alt="image"
src="https://github.com/user-attachments/assets/81f6e475-1435-445d-82eb-ecf5253c42d3"
/>
Note for reviewer: 3 Scout + 1 FTR tests were "broken" to show/test
reporter, those changes must be reverted before merge. See failed
pipeline
[here](https://buildkite.com/elastic/kibana-pull-request/builds/266822)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 1 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/grid-layout` | `src/platform/packages/private/kbn-grid-layout` |
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./examples/grid_example/tsconfig.type_check.json
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-grid-layout/jest.config.js
./src/platform/packages/private/kbn-repo-packages/package-map.json
./tsconfig.base.json
./tsconfig.base.type_check.json
./tsconfig.refs.json
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/platform/packages/private/kbn-grid-layout/jest.config.js:12
src/platform/packages/private/kbn-grid-layout/tsconfig.json:2
src/platform/packages/private/kbn-grid-layout/tsconfig.type_check.json:2
```
</details>
## Summary
This PR originally aimed at replacing the usages `styled-components`
with `@emotion/react` in the
`security_solution/public/common/components/events_viewer` folder. I
quickly realized removing some of these would require a small refactor.
This lead to making a few more changes, as many properties were actually
unused so a cleanup was welcome.
Only 2 small UI changes are introduced in this PR:
- the inspect icon on the top right corner of the tables are now always
visible instead of only visible on hover. I'm aware that this is a
different behavior from the alerts table in the alerts page, but we also
have other tables (like the one on threat intelligence page) where the
icon is always shown. Waiting on @codearos for confirmation here
- the `Grid view` and `Additional filters` button are reversed due to
the simplification of the code
No other UI changes are introduced. No behavior logic has been changed
either.
The biggest code cleanup are:
- removal of a bunch of unused properties and logic
- deletion of the RightTopMenu component: it was used in both
`StatefulEventsViewerComponent` and `getPersistentControlsHook` but none
of the internal logic was overlapping. I don't know how we got there but
its current implementation was overly complex and completely
unnecessary...
#### Alerts page
#### Rule creation page
#### Host/User/Network events tab
#### Host session view tab
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
It fixes#205051
<del>
Files are excluded because of `euiScrollBar` and `euiScrollBarCorner`
replacement (TBD)
-
x-pack/solutions/observability/plugins/exploratory_view/public/components/shared/exploratory_view/series_editor/series_editor.tsx
-
x-pack/solutions/observability/plugins/exploratory_view/public/components/shared/exploratory_view/series_editor/components/filter_values_list.tsx
</del>
✅ DONE
## Summary
For the most part, all of our colors translated fine into Borealis when
testing our pages. There will be more changes needed in the future to
completely replace all of the `euiThemeVars` usages, especially in
components that are shared with other teams. There are also quite a few
exported custom styled components that can't easily use the
`useEuiTheme` hook since they are not inside a react component. I didn't
want to touch those at this time.
- [x] Replace deprecated tokens to use new naming scheme ( like
successText --> textSuccess)
- [x] Use the hook `useEuiTheme()` over other methods
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves EUI Visual Refresh issue #202491
This PR is part of a list of PRs to perform the changes necessary to get
the new Borealis theme working correctly. It focuses on replacing the
deprecated color "success" colors have been updated to
"accentSecondary".
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 2 plugin(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/entities-data-access-plugin` |
`x-pack/solutions/observability/plugins/entities_data_access` |
| `@kbn/entityManager-app-plugin` |
`x-pack/solutions/observability/plugins/entity_manager_app` |
#### 10 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/core-user-settings-server` |
`src/core/packages/user-settings/server` |
| `@kbn/core-user-settings-server-internal` |
`src/core/packages/user-settings/server-internal` |
| `@kbn/core-user-settings-server-mocks` |
`src/core/packages/user-settings/server-mocks` |
| `@kbn/calculate-auto` |
`src/platform/packages/shared/kbn-calculate-auto` |
| `@kbn/charts-theme` | `src/platform/packages/shared/kbn-charts-theme`
|
| `@kbn/palettes` | `src/platform/packages/shared/kbn-palettes` |
| `@kbn/saved-search-component` |
`src/platform/packages/shared/kbn-saved-search-component` |
| `@kbn/use-tracked-promise` |
`src/platform/packages/shared/kbn-use-tracked-promise` |
| `@kbn/response-ops-rule-form` |
`src/platform/packages/shared/response-ops/rule_form` |
| `@kbn/streams-schema` |
`x-pack/solutions/observability/packages/kbn-streams-schema` |
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./docs/developer/plugin-list.asciidoc
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/core/packages/user-settings/server-internal/jest.config.js
./src/core/packages/user-settings/server-mocks/jest.config.js
./src/platform/packages/private/kbn-repo-packages/package-map.json
./src/platform/packages/shared/kbn-calculate-auto/jest.config.js
./src/platform/packages/shared/kbn-charts-theme/jest.config.js
./src/platform/packages/shared/kbn-palettes/jest.config.js
./src/platform/packages/shared/kbn-saved-search-component/jest.config.js
./src/platform/packages/shared/kbn-use-tracked-promise/jest.config.js
./src/platform/packages/shared/response-ops/rule_form/jest.config.js
./tsconfig.base.json
./x-pack/solutions/observability/packages/kbn-streams-schema/jest.config.js
./x-pack/solutions/observability/plugins/entities_data_access/jest.config.js
./x-pack/solutions/observability/plugins/entity_manager_app/jest.config.js
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/core/packages/user-settings/server-internal/jest.config.js:12
src/core/packages/user-settings/server-internal/tsconfig.json:2
src/core/packages/user-settings/server-mocks/jest.config.js:12
src/core/packages/user-settings/server-mocks/tsconfig.json:2
src/core/packages/user-settings/server/tsconfig.json:2
src/platform/packages/shared/kbn-calculate-auto/jest.config.js:12
src/platform/packages/shared/kbn-calculate-auto/tsconfig.json:2
src/platform/packages/shared/kbn-charts-theme/jest.config.js:12
src/platform/packages/shared/kbn-charts-theme/tsconfig.json:2
src/platform/packages/shared/kbn-palettes/jest.config.js:12
src/platform/packages/shared/kbn-palettes/tsconfig.json:2
src/platform/packages/shared/kbn-saved-search-component/jest.config.js:12
src/platform/packages/shared/kbn-saved-search-component/tsconfig.json:2
src/platform/packages/shared/kbn-use-tracked-promise/jest.config.js:12
src/platform/packages/shared/kbn-use-tracked-promise/tsconfig.json:2
src/platform/packages/shared/response-ops/rule_form/jest.config.js:12
src/platform/packages/shared/response-ops/rule_form/tsconfig.json:2
x-pack/solutions/observability/packages/kbn-streams-schema/jest.config.js:10
x-pack/solutions/observability/packages/kbn-streams-schema/tsconfig.json:2
x-pack/solutions/observability/plugins/entities_data_access/jest.config.js:12
x-pack/solutions/observability/plugins/entities_data_access/tsconfig.json:2
x-pack/solutions/observability/plugins/entity_manager_app/jest.config.js:12
x-pack/solutions/observability/plugins/entity_manager_app/tsconfig.json:2
x-pack/solutions/observability/plugins/entity_manager_app/tsconfig.json:7
```
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Adds logic to support the jest vscode extension by reading the
`--testPathPattern` arg for the purpose of config lookup. This enables
running tests easily in the vscode jest extension.
