## Summary
This PR removes the legacy kibana react code-editor, alongside replacing
all import declarations of this legacy component to the one offered by
shared-ux, i.e import declaration source of `'@kbn/kibana-react/public'`
is switched to `@kbn/code-editor`.
Also in this PR an helper for writing jest tests has been included
through the package `@kbn/code-editor-mock`, this would facilitate
mocking the editor, especially given that the code editor leverages
couple of APIs that are aren't included by default in jsdom, among them,
`matchMedia`, `ResizeObserver`. The provided mock is sufficient for most
use cases and can be setup in any package within kibana as a
[`node_module`
mock](https://jestjs.io/docs/manual-mocks#mocking-node-modules) without
having to repeatedly manually mock the editor within individual test
files. An example for how this might be done can be found here
ec5ba25368
### Checklist
<!-- Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
-->
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
<!--
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/167632
This PR provides a simpler api for the Lens embeddable consumers who
want to provide inline editing capabilities. I added an example to help
with the integration. Run kibana with
```
yarn start --run-examples
```
http://localhost:5601/app/lens_embeddable_inline_editing_example
<img width="1381" alt="image"
src="58e7ef2d-2f92-4bab-9cb4-d04a90d87e15">
<img width="2498" alt="image"
src="0a050e8d-f22f-4c48-88e4-20c42683a279">
It also allows the consumers to render the inline editing component in a
custom element in case you don't want to open a push flyout.
I included a readme on how to use the api.
### Note
This is the first PR which uses the new Lens config builder so some of
the changes are not related to the api improvements but they are fixing
some bugs on the builder.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Move all files owned by the Entity Analytics team to
`x-pack/plugins/security_solution/public/entity_analytics`
Initially, files inside `public/entity_analytics/components` look
disorganised because many of them have similar names and do not follow
the same conventions.
I minimized the PR scope to avoid putting too much of a burden on
reviewers.
### Changes
* Moves files to the `entity_analytics` folder
* Updates imports
* Moves the content of some files to a new file inside the
`entity_analytics` folder
* Inlines copies that were previously inside `translation.ts`.
* No changes to the user experience are expected.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
After a PR is merged, CI runs all the tests on `main`. If one fails, a
github issue with the label `failed-test` is created. If you comment
`/skip` on the issue then a workflow action will skip the test directly
on the failing branches.
This workflow can only be used by contributors who have `admin` access
to Kibana, OR to people who are on an allow-list of elastic teams. The
list of teams allowed is `appex-qa`.
This PR adds another team to the list:
@elastic/security-solution-test-skippers
This team was just created to allow a small list of Security Solution
contributors the ability to use this workflow.
Allowing a few of us access to this workflow will allow Security to
triage our own test by writing `/skip` comments.
### Without this PR we have to:
Without this PR we have to do a bunch of stuff to skip our own flaky
tests:
1. Manually edit the files to create a PR on all affected branches.
2. Create PRs using the web app.
3. Get a second developer to approve the PR.
4. Wait for CI.
5. Merge the PR.
This process is slow because of manual steps, and because CI must run
before we can skip the test. At a minimum we should expect that process
to take as long as CI takes (e.g. 90 minutes.) But the PRs that are
meant to skip tests may also in turn be blocked by flaky tests. It also
requires coordination from 2 people to make and approve the PR.
### Code references
[skip-failed-test.yml](https://github.com/elastic/kibana/blob/main/.github/workflows/skip-failed-test.yml#L25-L30)
is where the workflow is defined
[Docs on the syntax for
workflows.](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsuses)
[permission-check/index.ts](https://github.com/elastic/kibana-github-actions/blob/main/permission-check/index.ts#L23-L26)
is where the permission check is implemented. Users need to have:
* either the right permission level
* or be a member of the team
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Gloria Hornero <gloria.hornero@elastic.co>
## Summary
Context:
https://github.com/elastic/kibana/issues/173307#issuecomment-1855858929
Summary of the issue: the automation that assigns team labels to
`failed-test` tickets sometimes doesn't assign a team label or can
assign an incorrect label. This is because it scans the CODEOWNERS file
from bottom to top instead of properly parsing it as GitHub does.
This PR lifts some of the lines up, which should reduce the chance of
the above issue happening again. But that's a temporary workaround.
Closes https://github.com/elastic/kibana/issues/170236.
### Changes
This PR focuses on adding an action per dataset + namespace to navigate
to Observability log explorer. At the same time I took the opportunity
to update `Open in discover` link to include the controls present in
`Observability Log Explorer` state (atm just `namespace`)
1. Extracted `getRouterLinkProps` to a new package for reusability.
2. New `Actions` column was added to table.
3. `LogExplorerLink ` component was introduced, to reuse the navigation
logic between the table and the flyout.
4. `getDiscoverFiltersFromState` was added to combine state filters and
controls into discover state when navigating to discover
#### Demo
a3f38615-d8ae-432b-ba7b-05a6901f870c
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds a script that determines GitHub code ownership for
functional test files in the Kibana repository.
### Why do we need this?
We want to be able to determine test ownership to allow teams to get a
better overview of their tests (number of tests, number of skipped
tests, number of failures in the last x days, etc).
### What does this PR bring?
This PR is a first step on closing the test ownership gaps. It adds
functionality to determine the GitHub code owner for a given file (in
the `@kbn/code-owners` package) and adds a script that makes use of this
to check if all functional test files have a code owner, reporting the
gaps.
### Future plans
The idea is to include the test ownership information in our ingested
test results, such that we can create dashboards, reports, etc based on
it.
At some point (once all ownership gaps are closed), we might consider
running this check on CI to prevent new test files without owners.
### How to run?
```
node scripts/check_ftr_code_owners.js
```
The script lists the functional test files that are not covered by code
owners and also gives a summary like this:
```
ERROR Found 2592 test files without code owner (checked 7550 test files in 12.73 s)
```
## Summary
This PR is part of the effort we are making to have our cypres
serverless tests ready for the second quality gate (QA environment -
real serverless project and also part of the initial effort started by
https://github.com/elastic/kibana/issues/153664.
With the introduced changes, we are creating specific Entity Analytics
executions for both ESS and serverless with the aim of:
- To help to identify quickly the ownership of the tests in case of
failure.
- To help to raise specific flakiness inside the tests of the team.
In this PR:
- We are creating different executions for ESS, serverless and the
quality gate
- We are integrating everything with buildkite and also adding the new
executions to the flaky test suite runner
- We are updating the readme
- We are removing the overall `Security Solution` execution.
## Summary
As elsewhere move our code under one folder and update codeowners.
This is also driven by me wanting to create a common entity analytics
util that will live in `common/entity_analytics` but not won't belong in
asset criticality or risk engine.
- Closes https://github.com/elastic/kibana/issues/172570
## Summary
This PR creates a new package `@kbn/visualization-utils` and moves
`getTimeZone` helper into it. Also the PR removes duplicates of other
similar helpers.
And the histogram in the field popover has now the same time zone
configuration as the the main hits histogram:
<img width="500" alt="Screenshot 2023-12-06 at 18 46 25"
src="2d350c91-1a41-419a-9d80-f203f1c90327">
## For testing
Change `dateFormat:tz` in Advanced Settings and check if histograms are
rendered accordingly.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#161002Closes#170073
## Summary
This PR implements a createModelVersion API in the Encrypted Saved
Objects plugin to support upward migrations for model version encrypted
saved objects.
Much like how the `createMigration` API provided a way to wrap migration
functions to support migration of encrypted saved objects prior to the
model version paradigm, the new `createModelVersion` API provides a way
to wrap a model version definition for the same purpose.
`createModelVersion` manipulates the changes defined for a model version
('unsafe_transform', 'data_backfill', 'data_removal'), merging them into
a single transform function in which the saved object document is
decrypted, transformed, and then encrypted again. The document is
decrypted with the `encrypted saved object type registration` provided
by the required `inputType` parameter. Similarly, the document is by
encrypted with the `encrypted saved object type registration` provided
by the required `outputType` parameter.
An example plugin (`examples/eso_model_version_example`) provides a
demonstration of how the createModelVersion API should be used. The UI
of the example plugin gives an idea of what the encrypted saved objects
look like before and after the model version changes are applied.
## Testing
### Manual Testing
- Modify the example plugin implementation in
`examples/eso_model_version_example` to include different changes or
additional model versions.
### Unit Tests
-
`x-pack/plugins/encrypted_saved_objects/server/create_model_version.test.ts`
### Functional Tests
-
`x-pack/test/encrypted_saved_objects_api_integration/tests/encrypted_saved_objects_api.ts`
-
`x-pack/test/encrypted_saved_objects_api_integration/tests/encrypted_saved_objects_decryption.ts`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Part of https://github.com/elastic/kibana/issues/171570
PR breaks EsError out of data plugin and into @kbn/search-errors
package. This will allow for future efforts to move [lens error
handing](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lens/public/editor_frame_service/error_helper.tsx)
into this package and more closely align error handling across kibana.
An unplanned part of this PR was moving `BfetchRequestError` into its
own package `@kbn/bfetch`. This is required because `@kbn/search-errors`
package can not import code from a plugin. `BfetchRequestError` could
not be part of `@kbn/search-errors` because that created a circular
dependency graph.
Another unexpected problem found while working on this PR is that the
package had to be included in `ui-shared-deps`. The reason for this is
that `renderSearchError` uses `instanceof EsError` check to determine if
an error is an EsError. This check failed when the package was not
included in `ui-shared-deps` because each plugin got its own copy of
`kbn/search-error` plugin. This meant that `data` plugin had its own
copy of `EsError` class when the error is generated, then `discover`
plugin had its own copy of `EsError` class when the instanceof check
occurs. `instanceof` check unexpectedly returned false in this case.
Moving plugin to `ui-shared-deps` results in a single instance of the
plugin. Thanks @mistic for the help with moving package into
`ui-shared-deps`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
## Summary
fixes path to detection-engine tests
from
`/x-pack/test/security_solution_cypress/cypress/e2e/detection_engine
@elastic/security-detection-engine`
to
`/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine
@elastic/security-detection-engine`
## [Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts
This PR implements _Retrieval Augmented Generation_ (RAG) for Alerts in the Security Solution. This feature enables users to ask the assistant questions about the latest and riskiest open alerts in their environment using natural language, for example:
- _How many alerts are currently open?_
- _Which alerts should I look at first?_
- _Did we have any alerts with suspicious activity on Windows machines?_
### More context
Previously, the assistant relied solely on the knowledge of the configured LLM and _singular_ alerts or events passed _by the client_ to the LLM as prompt context. This new feature:
- Enables _multiple_ alerts to be passed by the _server_ as context to the LLM, via [LangChain tools](https://github.com/elastic/kibana/pull/167097)
- Applies the user's [anonymization](https://github.com/elastic/kibana/pull/159857) settings to those alerts
- Only fields allowed by the user will be sent as context to the LLM
- Users may enable or disable anonymization for specific fields (via settings)
- Click the conversation's `Show anonymized` toggle to see the anonymized values sent to / received from the LLM:
### Settings
This feature is enabled and configured via the `Knowledge Base` > `Alerts` settings in the screenshot below:
- The `Alerts` toggle enables or disables the feature
- The slider has a range of `10` - `100` alerts (default: `20`)
When the setting above is enabled, up to `n` alerts (as determined by the slider) that meet the following criteria will be returned:
- the `kibana.alert.workflow_status` must be `open`
- the alert must have been generated in the last `24 hours`
- the alert must NOT be a `kibana.alert.building_block_type` alert
- the `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts
### Feature flag
To use this feature:
1) Add the `assistantRagOnAlerts` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below:
```
xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts']
```
2) Enable the `Alerts` toggle in the Assistant's `Knowledge Base` settings, per the screenshot below:
## How it works
- When the `Alerts` settings toggle is enabled, http `POST` requests to the `/internal/elastic_assistant/actions/connector/{id}/_execute` route include the following new (optional) parameters:
- `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default`
- `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g. `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]`
- `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]`
- `replacements`, a `Record<string, string>` of replacements (generated on the server) that starts empty for a new conversation, and accumulates anonymized values until the conversation is cleared, e.g.
```json
"replacements": {
"e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm",
"cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia",
"7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s",
"f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc",
// ...
},
```
- `size`, the numeric value set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20`
- The `postActionsConnectorExecuteRoute` function in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts` was updated to accept the new optional parameters, and to return an updated `replacements` with every response. (Every new request that is processed on the server may add additional anonymized values to the `replacements` returned in the response.)
- The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts` previously used a hard-coded array of LangChain tools that had just one entry, for the `ESQLKnowledgeBaseTool` tool. That hard-coded array was replaced in this PR with a call to the (new) `getApplicableTools` function:
```typescript
const tools: Tool[] = getApplicableTools({
allow,
allowReplacement,
alertsIndexPattern,
assistantLangChain,
chain,
esClient,
modelExists,
onNewReplacements,
replacements,
request,
size,
});
```
- The `getApplicableTools` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/index.ts` examines the parameters in the `KibanaRequest` and only returns a filtered set of LangChain tools. If the request doesn't contain all the parameters required by a tool, it will NOT be returned by `getApplicableTools`. For example, if the required anonymization parameters are not included in the request, the `open-alerts` tool will not be returned.
- The new `alert-counts` LangChain tool returned by the `getAlertCountsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_tool.ts` provides the LLM the results of an aggregation on the last `24` hours of alerts (in the current Kibana Space), grouped by `kibana.alert.severity`. See the `getAlertsCountQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_query.ts` for details
- The new `open-alerts` LangChain tool returned by the `getOpenAlertsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_tool.ts` provides the LLM up to `size` non-building-block alerts generated in the last `24` hours (in the current Kibana Space) with an `open` workflow status, ordered by `kibana.alert.risk_score` to prioritize the riskiest alerts. See the `getOpenAlertsQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_query.ts` for details.
- On the client, a conversation continues to accumulate additional `replacements` (and send them in subsequent requests) until the conversation is cleared
- Anonymization functions that were only invoked by the browser were moved from the (browser) `kbn-elastic-assistant` package in `x-pack/packages/kbn-elastic-assistant/` to a new common package: `x-pack/packages/kbn-elastic-assistant-common`
- The new `kbn-elastic-assistant-common` package is also consumed by the `elastic_assistant` (server) plugin: `x-pack/plugins/elastic_assistant`
## Summary
This PR adds a bunch of plugins to help manage AI Assistant Management
settings.
It offers a 'selection' plugin inside Stack Management where a user can
select which AI Assistant she wants to manage.
The Security team can hook into this one, so settings for both AI
Assistants can be accessed from inside one place inside Stack
Management.
This PR also adds the plugin to manage settings for the AI Assistant for
Observability, including Knowledge Base management. This plugin is
available both in Stack Management (stateful) and Project Settings
(serverless).
## What it looks like
51392ec5-05c9-4947-9bf2-810d8d0b7525
## Detailed
1. **Adds a Stack Management plugin**
(`/src/plugins/ai_assistant_management/selection`). Its primary function
is to render a selection screen to help users navigate to the settings
plugin for the AI Assistant for a specific solution. This plugin is
displayed in Stack Management, which is only available in stateful
versions of Kibana.
2. **Adds a AI Assistant for Observability Settings plugin**
(`/src/plugins/ai_assistant_management/observability`). This plugin
allows management of specific Observability AI Assistant settings. It is
available in stateful versions of Kibana (via the aforementioned Stack
Management plugin) or in serverless versions via Project Management.
3. **Knowledge Base management for Observability AI Assistant**: The AI
Assistant for Observability Settings plugin has a Knowledge Base tab,
which allows users to add / read / update / delete and bulk import
entries into the Knowledge Base of the Observability AI Assistant.
4. **Moving of KB endpoints in Observability AI Assistant plugin**: KB
endpoints and functions were located in the same folder. As this PR adds
new endpoints for the KB for CRUD operations, it also moves the existing
ones from the function folder into a dedicated one so there's a clearer
distinction between kb and functions.
5. **Adding of GenAI Connector inside Chat Flyout**: If the user has
admin rights, it is possible to set up a GenAI connector from within the
Observability AI Assistant Chat Flyout. This provides a faster and more
seamless onboarding experience. If the user does not, she will be
redirected to the Settings page.
## Bug fixes
* Fixes chat item styling issues (padding, background color).
## How to test
* Check if the Stack Management plugin works on stateful
* Check if the AI Assistant Settings plugin works on stateful +
serverless
* Check if CRUD operations on KB work
* Check if searching on KB entries work
* Check if its possible to navigate to KB tab directly
(`app/management/kibana/aiAssistantManagementObservability?tab=knowledge_base`)
## Todo
- [x] Add sorting to getEntries
- [x] Add params for tab routing
- [x] Add unit tests
- [ ] Add API tests
- [ ] Add fallback for already indexed entries when searching
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR:
* extracts `enterprise_search` index documents component to common
package.
* Uses EUI pagination, converting Elasticsearch`from` & `size` to EUI
pagination standard for documents list
### Screen Recording
f585d9cc-f92c-44f4-aead-23c75c107a0f
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Partially resolves: https://github.com/elastic/kibana/issues/164255,
this is 2/3 of the scoped query changes.
Maintenance window scoped query frontend changes. Adds the ability to
add and edit scoped query for maintenance windows. Due to limitations
with the alerts search bar and each solution fetches AAD fields, we only
allow users to associate scoped query with 1 category (manangement,
o11y, or security solution). The intended usage in this case is for the
user to create multiple maintenance windows if they wish to apply scoped
queries to multiple solutions.
### To test:
go to
`x-pack/plugins/alerting/public/pages/maintenance_windows/constants.ts`
and set `IS_SCOPED_QUERY_ENABLED` to `true`
### Scoped query off, multiple category allowed:
### Scoped query on, multiple category disallowed:
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Create `security_solution/common/api/entity_analytics` to house all
entity analytics related apis, in this case `risk_score`, `risk_engine`
and `asset_criticality`.
This makes ownership clearer, simplifies the codeowners file, and will
enable the entity analytics team to have shared schema components in
`security_solution/common/api/entity_analytics/common` (something I want
to do in an upcoming PR)
I have removed `@elastic/security-threat-hunting-explore`'s ownersip of
`/x-pack/plugins/security_solution/common/api/risk_score` let me know if
you want to keep it!
Closes#171613
## Summary
This PR adds the viewInApp URL to the custom threshold rule type. This
URL will send the user to the log explorer with the selected data view
and the rule's query filter. If there is only one document aggregation,
then the filter related to this aggregation will be added as shown
below:
|Rule|Discover with pre-fill data|
|---|---|
||
For the ad-hoc data view, you should be able to see the selected index
pattern in discover similar to this:
<img
src="046493ae-ba59-46b7-a40f-68d1836d43f1"
width=400 />
### 🧪 How to test
- Check the viewInApp URL both in action variables and the alert table
for the following scenarios:
- A rule with a persisted data view
- A rule with an ad-hoc data view
- A rule with count aggregation and filter
- A rule with an optional query filter
- A rule with non-count aggregation
In all the above scenarios, the starting time in the Discover should be
before the alert's start time.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds upsert, read and delete APIs for asset criticality records. I have
used the OpenAPI code generation to create the types and zod schemas.
The APIs added are as follows:
**POST /internal/risk_score/criticality**
Request Body:
```
{
id_value: "host-1",
id_field: "host.name",
criticality_level: "very_important"
}
```
If the record already exists it will be overwritten, otherwise created
**GET
/internal/risk_score/criticality?id_field=host.name&id_value=host-1**
Response body:
```
{
id_value: "host-1",
id_field: "host.name",
criticality_level: "very_important"
@timestamp: "2023-11-29T11:43:43.175Z"
}
```
**DELETE
/internal/risk_score/criticality?id_field=host.name&id_value=host-1**
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Consolidates UI elements and backend code to create/delete data views
and destination indices related to transforms and data frame analytics.
We ended up with two different approaches for creating data views in the
wizards for transforms and data frame analytics, the original reason was
we were not aware of the `allowNoIndex: true` setting and worked around
that in different ways.
This PR aligns UI workflows and moves related code to a new package
`@kbn/ml-data-view-utils` for data views and
`@kbn/ml-creation-wizard-utils` for the destination index form. The
latter might be used for other shared components across wizard..
In Data Frame Analytics, the checkbox to create a data view was removed
from the last "Create" step, instead the option to create a data view
was moved to the "Details" step.
In Transforms, the UI component to create the destination index was
brought over from DFA where there is a switch option to automatically
use the job ID as the name for the destination index by default.
**Addresses:** https://github.com/elastic/security-team/issues/7981
## Summary
This PR adds an OpenAPI spec bundler to simplify integration with the
Docs Engineering team. The bundler produces a single bundled file by
dereferencing and inlining some of external references and bundling them
and paths into a single file.
## Details
Currently we maintain a number of schema files inside
`x-pack/plugins/security_solution/common/api/**.schema.yaml` and it
might be hard for external teams to keep track of all the changes in our
schemas. By creating a singular schema file, we provide a clear
integration point for others.
The bundler addresses the following issues
- hide endpoints that we don't want to expose (Endpoints related to
features hidden under a feature flag and all internal endpoints should
be excluded from the file)
- hide not finished data structures related to features hidden under a
feature flag or data structures that are not designed to be public (For
example `RuleActionAlertsFilter` or `RuleActionParams` are exposed
directly from the Alerting framework and might be considered
implementation details, we don't want to document interfaces that are
not designed to be public so hiding them is a good option)
- modify spec based on presence of `x-modify` property (Instead of
exposing `x-modify: partial` we need to make the exported data structure
partial and instead of exposing `x-modify: required` we need to make the
exported data structure required)
- remove any internal attributes used for code generation like
`x-codegen-enabled` and `x-modify`
- inline some of the reused data structures (We have a lot of low-level
reusable data structures `in common_attributes.schema.yaml` which might
make the final documentation hardly usable from the UX perspective, so
we can inline them)
and lives in a new `@kbn/openapi-bundler` package under
`packages/kbn-openapi-bundler` folder.
### Related changes
- Implicit version type `version: 2023-10-31` has been changed to
explicit string type `version: '2023-10-31'` for all specs under
`security_solution/common/api` folder. Implicit type causes `js-yaml`
parsing it as a `Data` JS object leading to serializing it like
`2023-10-31T00:00:00.000Z`.
- `ListRequestQuery` schema in
`security_solution/common/api/endpoint/actions/list.schema.yaml ` has
been renamed to `EndpointActionListRequestQuery` to avoid conflicts with
`ListRequestQuery` in
`security_solution/common/api/endpoint/metadata/list_metadata.schema.yaml`.
While it's not an issue to have completely different schemas sharing the
same name in different files it may be an indication of pitfalls in the
API design. I'd say it's an open question if such cases need to be
always resolved automatically or reviewed manually. At this moment the
bundler can't resolve such conflicts.
## How to test?
There is a a new JS script added to Security Solution plugin located at
`x-pack/plugins/security_solution/scripts/openapi/bundle.js` with a
corresponding entry in `package.json` named `openapi:bundle`.
To test the PR change directory to Security Solution plugin's root
folder and run the bundler like below
```sh
cd x-pack/plugins/security_solution
yarn openapi:bundle
```
It should produce a bundled OpenAPI spec at
`x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml`.
## Open issues
- [x] Circular references (implemented in
dfdf0a51ea)
- [x] Mix of OpenAPI `3.0` and `3.1` specs (Maybe convert automatically
to `3.1`?). Folder like OpenAPI bundling format implemented
[here](0ae7ad5abb)
allows to mix OpenAPI `3.0` and `3.1` specs.
## Improvements
- [ ] Flexible configuration
- [ ] CLI support?
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/170428
The bug this is intended to resolve requires some in-depth steps to
reproduce. Follow the instructions in the issue above. Then, merge in
this branch and compare.
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
List APIs tests
- Added a new folder under the `security_solution_api_integration`
called `lists_and_exception_lists` to hold the lists and exception lists
tests, and split the `List` APIs into two groups since the execution
time in Serverless was close to 30 mins
- Modified the
[x-pack/plugins/lists/common/schemas/response/exception_list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-4e3545fdeb8c8d9467cfa1c4aa88194e189193a92fa6f1cf5f859b1ef1beb45c),
[x-pack/plugins/lists/common/schemas/response/exception_list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-c3cc18faf07aab86e307185d41599c3596a3f8b360d3e4829591afa148283238)
,
[x-pack/plugins/lists/common/schemas/response/list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-608579ca5e65da74f41319a58d81ab12cc3d79d389b087806c7b74949fbc6cc3),
[x-pack/plugins/lists/common/schemas/response/list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-efc64eb35937a8da28fc982c527253c0923650ae4163d4bbc203d3ebc2949835)
to accept `elastic user` input because it changes in ESS and Serverless
- Deleted the `x-pack/test/lists_api_integration` folder
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the below files imports from the old `lists_api_integration`
folder to the new `lists_and_exception_lists`
```
lists_api_integration/
- exceptions/operators_data_types/date_numeric_types/date.ts
- exceptions/operators_data_types/date_numeric_types/double.ts
- exceptions/operators_data_types/date_numeric_types/float.ts
- exceptions/operators_data_types/date_numeric_types/integer.ts
- exceptions/operators_data_types/ips/ip.ts
- exceptions/operators_data_types/ips/ip_array.ts
- exceptions/operators_data_types/keyword/keyword.ts
- exceptions/operators_data_types/keyword/keyword_array.ts
- exceptions/operators_data_types/long/long.ts
- exceptions/operators_data_types/text/text.ts
- exceptions/operators_data_types/text/text_array.ts
- exceptions/workflows/create_endpoint_exceptions.ts
- exceptions/workflows/create_rule_exceptions.ts
- exceptions/workflows/find_rule_exception_references.ts
- exceptions/workflows/role_based_add_edit_comments.ts
- exceptions/workflows/role_based_rule_exceptions_workflows.ts
- exceptions/workflows/rule_exception_synchronizations.ts
- rule_execution_logic/execution_logic/esql.ts
- rule_execution_logic/execution_logic/machine_learning.ts
- rule_execution_logic/execution_logic/new_terms.ts
- rule_execution_logic/execution_logic/query.ts
- telemetry/task_based/all_types.ts
- telemetry/task_based/detection_rules.ts
- telemetry/task_based/security_lists.ts
```
```
These files should be moved too soon to the new `lists_api_integration`
detection_engine_api_integration/security_and_spaces/group10
- import_export_rules.ts
- import_rules.ts
- perform_bulk_actions
```
- Updated the below files imports to the
`ftr_provider_context_with_spaces.d.ts`
` - risk_engine/risk_scoring_task/task_execution_nondefault_spaces.ts`
- The QA phase concluded with all tests passing successfully. 🟢
- Updated the CodeOwner file for the newly moved tests
- Add a new util file to `deleteAllExceptions` under the old
`detection_engine_api` folder since the Rule management related-tests
are still need to be moved over to the new folder
- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes partially two remaining tasks from
https://github.com/elastic/kibana/issues/168753
Fixes partially dataview issue from
https://github.com/elastic/kibana/issues/170398
It stretches to maximum approximate 60 characters if any of the labels
in the list is of this length. If the content doesn't need the container
to stretch, it doesn't do it.
<details>
<summary> Field picker in Lens</summary>
minimum width:
<img width="445" alt="Screenshot 2023-11-21 at 15 56 03"
src="2f0f8482-bd00-4ec2-bbde-cbc4f3198eed">
auto-expanded width:
<img width="575" alt="Screenshot 2023-11-21 at 15 58 22"
src="df7bab4d-0a08-4d49-8a91-9386eba15d93">
</details>
<details>
<summary>Layer data view picker in Lens</summary>
<img width="376" alt="Screenshot 2023-11-21 at 16 01 17"
src="b8a98d83-dabc-49bd-a3cc-fc3856de6d3e">
<img width="455" alt="Screenshot 2023-11-21 at 15 58 09"
src="f2c5bde8-7a4a-485f-bf97-fc2179171e50">
<img width="615" alt="Screenshot 2023-11-21 at 15 56 27"
src="0574fc6c-69a3-44e9-9d48-8d427c1c5dba">
</details>
<details> <summary>Data view picker in Unified Search</summary>
<img width="341" alt="Screenshot 2023-11-21 at 16 00 29"
src="1c838ded-0dc5-4632-94e4-1d94586f667c">
<img width="441" alt="Screenshot 2023-11-21 at 15 58 04"
src="87e4f1c0-7922-4b94-a114-f23ece544395">
<img width="561" alt="Screenshot 2023-11-21 at 15 56 20"
src="3ea0f222-5241-4c5b-b00b-4311972754cc">
</details>
<details>
<summary> Data view picker in dashboard Create control flyout</summary>
<img width="677" alt="Screenshot 2023-11-21 at 16 14 00"
src="0455b6ed-555d-4cff-9e34-0de377be6e04">
<img width="682" alt="Screenshot 2023-11-21 at 15 54 56"
src="2a67685c-379d-4c0b-bf56-dbf7c35b3bd4">
</details>
<details>
<summary> Unified search data view select component (tested in
maps)</summary>
<img width="570" alt="Screenshot 2023-11-22 at 14 38 25"
src="bb52ab22-626d-4556-b40c-c9bcc925f426">
</details>
<details>
<summary>Unified search field and value picker</summary>
Adds `panelMinWidth`, removes the custom flex width change behavior
<img width="1142" alt="Screenshot 2023-11-22 at 14 40 26"
src="2450957f-38b7-4a73-b531-7acb29cb56bc">
f4f33624-9287-403e-8472-81f705440f97
</details>
<details>
<summary> Discover breakdown field</summary>
Removes the focus stretching and instead uses the panelMinWidth prop
<img width="419" alt="Screenshot 2023-11-21 at 16 46 50"
src="e35125ad-8823-4bff-954b-8119a352829c">
<img width="619" alt="Screenshot 2023-11-21 at 16 48 20"
src="89e63daf-a59e-43e1-a6ec-91d1b15b0fcd">
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
In this PR, I'm relocating all Kibana Security types (along with a few
schemas necessary for some of these types, unfortunately) that are part
of public contracts to separate packages. This change will enable any
plugin to utilize Security APIs via "static" or
["runtime"](https://github.com/elastic/kibana/pull/167113) dependencies,
regardless of whether Kibana Security already relies on these plugins or
not.
__NOTE TO REVIEWERS:__ I tried to minimize changes as much as I could
via moving only necessary types. I also didn't move deprecated parts of
the Setup/Start contracts to these new packages.
__Triggered by:__ https://github.com/elastic/kibana/pull/168910
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The package data isn't needed anymore, so we can remove that package. On
top, it seems like the package was causing some issues.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Attempting to merge #170852 again now that the release artefact step has
been fixed as part of https://github.com/elastic/kibana/pull/171457
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
basic detection engine-related tests
- Introduced a new license folder to hold the `Basic` Ess tests and the
`Essentials` Serverless tests, is called `basic_essentials_license`
- Added new base configurations files for
`serverless/config.base.essentials` and `ess/config.base.basic`
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the CodeOwner file for the newly moved tests
- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
- The **Privileges** Tests are skipped in Serverless now until the FTR
Roles [PR](https://github.com/elastic/kibana/pull/170131) gets merged
| Action | File | New Path |
|--------|------|----------|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/rules/create_rules|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/rules/create_ml_rules_privileges|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/alerts/open_close_alerts|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/alerts/query_alerts_backword_compatibility|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/alerts/query_alerts|
## Summary
Security Data Analytics would like to know when these files are updated.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Adds the ability to quickly create a categorisation anomaly detection
job from the pattern analysis flyout.
Adds a new `created_by` ID `categorization-wizard-from-pattern-analysis`
which can be picked up by telemetry.
Creates a new package for sharing our AIOPs ui actions IDs. I think we
should move the pattern analysis ID to this package too, but that can be
done in a separate PR.
51349f93-f072-4983-85f0-98741902fb5a
6e618581-8916-4e63-930f-945c96c25e6c
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Update new user details flyout to be consistent with Expandable Alerts
Flyout. The previous user details flyout implementation was hidden
behind a flag and never went live.
### What is included
* Update new user details flyout to use the expandable flyout component
* Update UI components according to the new design
* Keep the feature hidden behind newUserDetailsFlyout flag
* Supporting alert risk inputs
### What is NOT included
* Supporting multiple categories of risk inputs
* Host details flyout
* User and host pages
* Asset integrations (okta and azure)
* Update the flyout on the timeline (It is currently a technical
restriction of the expandable flyout, but the team is working to fix it)
### How to test it?
* Enable experimental flag `newUserDetailsFlyout`
`xpack.securitySolution.enableExperimental: ['newUserDetailsFlyout']`
* Create alerts and open alerts page
* Click on a username
- [x] Test edge cases
- [x] No cases permissions (it hides cases actions)
- [x] Basic license (it hides the risk score summary)
- [x] No risk score data for a user (It hides the risk score summary)
<img width="434" alt="Screenshot 2023-11-13 at 15 56 33"
src="4fc13042-cd3d-487b-9982-bfbf02f003b4">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
