We are bumping up against the time limit in MKI runs.
We are seeing this error: `Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called;`
Also, the 3 failing configs are each going over the 2 hour time limit.
So, let's split these three configs:
1. `x-pack/test_serverless/functional/test_suites/search/common_configs/config.group1.ts`
1. `x-pack/test_serverless/functional/test_suites/security/common_configs/config.group1.ts`
1. `x-pack/test_serverless/functional/test_suites/security/common_configs/config.group1.ts`
`ignore_throttled` is automatically added to AD jobs when created. These
are then reused in various searches where the whole `indices_options`
object from the datafeed is passed in the search call.
This PR adds a function to remove `ignore_throttled` in these situations
to avoid triggering deprecation warnings.
## Summary
A recent [bug](https://github.com/elastic/kibana/issues/199902) that
affected some of the pages in Stack Monitoring was caused by changes
related to the locators of the logs-related apps.
The issue wasn't caught by type checks as the affected area in the
monitoring plugin was written in JavaScript.
The goal of this PR is to migrate the logs-related components to
TypeScript.
### Testing
The stateful environment deployed by this PR includes logs and metrics
for stack monitoring. Please make sure to select a larger time range
(e.g. last 14 days).
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
#### 2 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/data-stream-adapter` |
`x-pack/solutions/security/packages/kbn-data-stream-adapter` |
| `@kbn/index-adapter` |
`x-pack/solutions/security/packages/kbn-index-adapter` |
---------
Co-authored-by: PhilippeOberti <philippe.oberti@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes https://github.com/elastic/kibana/issues/203151
Adds a normalization for the `kql_query`, `eql_query`, and `esql_query`
fields that trims the whitespace from the beginning and end of query
strings for a more robust comparison in the diff algorithms. Since
whitespace before or after the query string is purely a formatting
choice and doesn't impact the query itself, we discard the excess
whitespace characters before the direct string comparison.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
## Summary
Implements https://github.com/elastic/observability-dev/issues/4133.
Opening this up for a first pass as the PR is getting quite big. I've
listed below some things that can be improved in further iterations.
## High level notes
- Support for `format` has been added to the field definition
- UI:
- View inherited, mapped, unmapped fields.
- Edit mapped and unmapped fields.
- Map unmapped and unmap mapped fields
- Simulation / preview results
- Filtering
## Followups
- Filter dropdowns (on the right):
- We could potentially use a separate API for the mapping edits, rather
than the core edit route, to be more performant, but for now this is
used to create less surface area / deviation.
- State management is rudimentary right now. It could be improved with a
`useReducer` approach to avoid potential `useState` race conditions, and
then even something like xstate when things are more concrete. No state
syncs with the URL currently.
- Due to the lack of URL state syncing the "Edit in parent stream"
button doesn't navigate with things like a pre-selected field. We could
potentially co-ordinate this between the hooks in the schema editor and
detail view parent, but it's unneeded complexity at the moment.
- We could provide a lot more assistance with `format`. We could provide
a dropdown with options, and then a toggle to do custom. (Actually, it
looks like in the refined designs this is a dropdown, so I'll probably
switch this to a select with predefined options)
## Issues
- There seems to be a bug in the Elasticsearch JS library we use, calls
to `simulate.ingest` don't work as `body` is just set to `undefined`
(chasing this up). You can do the following patch in node_modules just
to get things going (run `yarn start` again):
- Runtime mappings don't seem to work with `match_only_text`:
`mapper_parsing_exception: No handler for type [match_only_text]`
## Open questions
- We might freeze changes to the root stream
- A failure on simulation doesn't do a hard block on saving changes. I
don't think it should, but open to other opinions.
## Screenshots
---------
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds support for explicit indication whether endpoint is
restricted to operator only users.
### Context
1. If user has [all operator
privileges](https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/operator/DefaultOperatorOnlyRegistry.java#L35-#L53)
granted, but is not listed as operator in `operator_users.yml`, ES would
throw an unauthorized error.
2. If user is listed as operator in `operator_users.yml`, but doesn't
have necessary privileges granted, ES would throw an unauthorized error.
3. It’s not possible to determine if a user is operator via any ES API,
i.e. `_has_privileges`.
4. If operator privileges are disabled we skip the the check for it,
that's why we require to explicitly specify additional privileges to
ensure that the route is protected even when operator privileges are
disabled.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Relates: https://github.com/elastic/kibana/issues/196271__
### How to test
1. Add your user to the operators list
1bd8144924/packages/kbn-es/src/serverless_resources/operator_users.yml (L4)
or use existing user from the list to log in.
2. Run ES and Kibana serverless
3. Change any endpoint or create a new one with the following security
config
```
security: {
authz: {
requiredPrivileges: [ReservedPrivilegesSet.operator],
},
},
```
4. Check with enabled and disabled operator privileges (set
`xpack.security.operator_privileges.enabled`)
## Release Note
Added support for explicit indication whether endpoint is restricted to
operator only users at the route definition level.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
#### 2 plugin(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/threat-intelligence-plugin` |
`x-pack/solutions/security/plugins/threat_intelligence` |
| `@kbn/timelines-plugin` |
`x-pack/solutions/security/plugins/timelines` |
#### 2 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/expandable-flyout` |
`x-pack/solutions/security/packages/kbn-expandable-flyout` |
| `@kbn/securitysolution-data-table` |
`x-pack/solutions/security/packages/data_table` |
Co-authored-by: PhilippeOberti <philippe.oberti@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR replaces a couple of places where hardcoded colors are used in
the APM portion of Kibana with EUITheme colors.
Before & After screenshots can be seen in the associated issue, #200960.
However, I was unable to find an example for the
[.../alert_details_app_section/failed_transaction_chart.tsx](https://github.com/elastic/kibana/pull/203348/files#diff-9d9e4bbfe128f4d2f6ff7f027cf746d679a6c06805ef77240cceb2770a837a28).
It seems like this chart in the alert creation flyout will never render
with annotations.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Miriam <31922082+MiriamAparicio@users.noreply.github.com>
## Summary
**Reviewers: Please test the code paths affected by this PR. See the
"Risks" section below.**
Part of work for enabling "high contrast mode" in Kibana. See
https://github.com/elastic/kibana/issues/176219.
**Background:**
Kibana will soon have a user profile setting to allow users to enable
"high contrast mode." This setting will activate a flag with
`<EuiProvider>` that causes EUI components to render with higher
contrast visual elements. Consumer plugins and packages need to be
updated selected places where `<EuiProvider>` is wrapped, to pass the
`UserProfileService` service dependency from the CoreStart contract.
**NOTE:** **EUI currently does not yet support the high-contrast mode
flag**, but support for that is expected to come in around 2 weeks.
These first PRs are simply preparing the code by wiring up the
`UserProvideService`.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [X] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [medium/high] The implementor of this change did not manually test
the affected code paths and relied on type-checking and functional tests
to drive the changes. Code owners for this PR need to manually test the
affected code paths.
- [ ] [medium] The `UserProfileService` dependency comes from the
CoreStart contract. If acquiring the service causes synchronous code to
become asynchronous, check for race conditions or errors in rendering
React components. Code owners for this PR need to manually test the
affected code paths.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
**Reviewers: Please test the code paths affected by this PR. See the
"Risks" section below.**
Part of work for enabling "high contrast mode" in Kibana. See
https://github.com/elastic/kibana/issues/176219.
**Background:**
Kibana will soon have a user profile setting to allow users to enable
"high contrast mode." This setting will activate a flag with
`<EuiProvider>` that causes EUI components to render with higher
contrast visual elements. Consumer plugins and packages need to be
updated selected places where `<EuiProvider>` is wrapped, to pass the
`UserProfileService` service dependency from the CoreStart contract.
**NOTE:** **EUI currently does not yet support the high-contrast mode
flag**, but support for that is expected to come in around 2 weeks.
These first PRs are simply preparing the code by wiring up the
`UserProvideService`.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [X] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [medium/high] The implementor of this change did not manually test
the affected code paths and relied on type-checking and functional tests
to drive the changes. Code owners for this PR need to manually test the
affected code paths.
- [ ] [medium] The `UserProfileService` dependency comes from the
CoreStart contract. If acquiring the service causes synchronous code to
become asynchronous, check for race conditions or errors in rendering
React components. Code owners for this PR need to manually test the
affected code paths.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/200620
1. Remove usage of deprecated color variables
2. Remove usage of `@kbn/ui-theme`
3. A few other changes as requested by @andreadelrio
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR performs multiple changes that all focuses on improving the
quality of the results returned when we translate rules that do not
match with a prebuilt rule and both with/without related integrations.
Changes include:
- Add a filter_index_patterns node, to always ensure `logs-*` is removed
with our `[indexPattern:logs-*]` value, which is similar to how we
detect missing lookups and macros.
- Split `translate_rule` into another `ecs_mapping` node, trying to
ensure translation focuses on changing SPL to ESQL without any focus on
actual field names, while the other node focuses only on the ESQL query
and changing field names.
- The summary now added in the comments have 1 for the translation and
one for the ECS mapping.
- Add default rule batch size `15` with PR comment/question.
- Ensure we only return one integration related rather than an array for
now, to make ESQL more focused on one related integration.
- New prompt to filter out one or more integrations from the returned
RAG; similar to how its done for rules RAG results already.
## Summary
The following improvements have been made:
- Added `--json` flag to CLI command to output result as JSON
- Terminology updated to more accurately reflect object contents
- Code owner teams are always returned as an array
- Search path validation (is under repo root, exists)
- Proper handling of inline comments
- Better logging for `scripts/check_ftr_code_owners.js`
Existing usage of the `@kbn/code-owners` package has been updated
accordingly, without modifying outcomes.
## Summary
Fixing flaky test when choosing a connector with the new EuiComboBox
component.
https://github.com/elastic/kibana/issues/203462
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## 📓 Summary
Related to https://github.com/elastic/kibana/pull/202985
This change reverts a suggestion that was applied but that should only
be valid for v9.
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
## Summary
[Internal link](https://github.com/elastic/security-team/issues/10820)
to the feature details
These changes add a possibility to edit, validate and save custom
migration rules:
* There are new `edit`, `save` and `cancel` buttons in the translation
tab of the details flyout for the non-installed custom rules
* There is a new ES|QL query editing component in the translation tab
which allows edit and validate the ES|QL query
* On saving the ES|QL query the custom migration rule will be updated
and based on the ES|QL validation a new `translation_result` might be
set: `full` if query is valid, `partial` if query has syntax errors,
`untraslated` if query is an empty string.
## Screen recording
https://github.com/user-attachments/assets/59cfc56f-3de8-4f7a-a2f9-79cb3fdee1c7
### Other changes
Next fixes and adjustments were also implemented as part of this PR:
* `Error` status in migration rules table to indicate whether the rule
translation has been failed
* Callouts inside the translation tab in details flyout
* Updated `Fully translated` status title into `Translated`
### Known issue
There is an issue with the autocompletion menu of the ES|QL query editor
component. It is being shifted. It could be because we are using this
component within the flyout and we might need to ask help from the team
which takes care of it.
Part of https://github.com/elastic/kibana/issues/167429
Remove `setCustomEmbeddableFactoryProvider` from embeddable setup API.
`setCustomEmbeddableFactoryProvider` only used in `embeddable_enhanced`
plugin. Replaced with `initializeReactEmbeddableDynamicActions` in react
embeddable system.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
resolves https://github.com/elastic/kibana/issues/201532
Removed the panel containing the logs explorer link. Updated tests and
i18n.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Part of embeddable refactor cleanup
AttributeService is moved from embeddable plugin to visualizations
plugin.
PR reduces visualizations bundle size by avoiding importing
`legacy/embeddable/index.ts` in plugin page load
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
A whole dependency was being pulled in for doing something that could
easily be done just using Node.js own utils and be made into a simple
method. As such, `multistream` has been removed since there is no other
place in the codebase that is using it.
## How to test
* Load Kibana local dev environment using synthtrace data/scenarios. No
scenario should fail to load normally, and all tests using synthtrace
data should pass as expected.
Closes#203860
## Summary
- the Cypress `parallel` runner was updated to set tooling logging level
first from Env. variables before falling back to the value defined in
the Cypress configuration file
- The env. value to set, if wanting to enable a specific logging level,
is `TOOLING_LOG_LEVEL`. The values supported are the same as those used
with `ToolingLog`
([here](b6287708f6/packages/kbn-tooling-log/src/log_levels.ts (L10))):
`silent`, `error`, `warning`, `success`, `info`, `debug`, `verbose`
- This change makes it easier to run Cypress tests locally with (for
example) a logging level of `verbose` for our tooling without having to
modify the Cypress configuration file. Example: `export
TOOLING_LOG_LEVEL=verbose && yarn cypress:dw:open`
- Added two new methods to our scripting VM service clients (for Vagrant
and Multipass):
- `download`: allow you to pull files out of the VM and save them
locally
- `upload`: uploads a local file to the VM. (upload already existed as
`transfer` - which has now been marked as deprecated).
- Added new service function on our Fleet scripting module to enable us
to set the logging level on a Fleet Agent
- Cypress tests were adjusted to automatically set the agent logging to
debug when running in CI
- A new Cypress task that allows for an Agent Diagnostic file (which
includes the Endpoint Log) to be retrieved from the host VM and stored
with the CI job (under the artifacts tab)
- A few tests were updated to include this step for failed test
## 📓 Summary
Closes#202258
This change updates the colors scale for Discover's `log.level`
indicators to differentiate errors from other levels better.
**N.B. As this relies on some hard-coded values defined
[here](https://github.com/elastic/kibana/issues/186273#issuecomment-2505817075),
it is not a definitive version, but a middle step to enhance the scale
in v8.x versions.**
With the introduction of the Borealis theme in v9, a new scale
token-based will replace this.
<img width="934" alt="Screenshot 2024-12-04 at 17 40 32"
src="https://github.com/user-attachments/assets/b3da1300-b39a-4ad0-92c9-fde5dabe91ec">
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
