## Summary
This PR is part of the effort we are making to have our cypres
serverless tests ready for the second quality gate (QA environment -
real serverless project and also part of the initial effort started by
https://github.com/elastic/kibana/issues/153664.
With the introduced changes, we are creating specific Entity Analytics
executions for both ESS and serverless with the aim of:
- To help to identify quickly the ownership of the tests in case of
failure.
- To help to raise specific flakiness inside the tests of the team.
In this PR:
- We are creating different executions for ESS, serverless and the
quality gate
- We are integrating everything with buildkite and also adding the new
executions to the flaky test suite runner
- We are updating the readme
- We are removing the overall `Security Solution` execution.
## Summary
As elsewhere move our code under one folder and update codeowners.
This is also driven by me wanting to create a common entity analytics
util that will live in `common/entity_analytics` but not won't belong in
asset criticality or risk engine.
- Closes https://github.com/elastic/kibana/issues/172570
## Summary
This PR creates a new package `@kbn/visualization-utils` and moves
`getTimeZone` helper into it. Also the PR removes duplicates of other
similar helpers.
And the histogram in the field popover has now the same time zone
configuration as the the main hits histogram:
<img width="500" alt="Screenshot 2023-12-06 at 18 46 25"
src="2d350c91-1a41-419a-9d80-f203f1c90327">
## For testing
Change `dateFormat:tz` in Advanced Settings and check if histograms are
rendered accordingly.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#161002Closes#170073
## Summary
This PR implements a createModelVersion API in the Encrypted Saved
Objects plugin to support upward migrations for model version encrypted
saved objects.
Much like how the `createMigration` API provided a way to wrap migration
functions to support migration of encrypted saved objects prior to the
model version paradigm, the new `createModelVersion` API provides a way
to wrap a model version definition for the same purpose.
`createModelVersion` manipulates the changes defined for a model version
('unsafe_transform', 'data_backfill', 'data_removal'), merging them into
a single transform function in which the saved object document is
decrypted, transformed, and then encrypted again. The document is
decrypted with the `encrypted saved object type registration` provided
by the required `inputType` parameter. Similarly, the document is by
encrypted with the `encrypted saved object type registration` provided
by the required `outputType` parameter.
An example plugin (`examples/eso_model_version_example`) provides a
demonstration of how the createModelVersion API should be used. The UI
of the example plugin gives an idea of what the encrypted saved objects
look like before and after the model version changes are applied.
## Testing
### Manual Testing
- Modify the example plugin implementation in
`examples/eso_model_version_example` to include different changes or
additional model versions.
### Unit Tests
-
`x-pack/plugins/encrypted_saved_objects/server/create_model_version.test.ts`
### Functional Tests
-
`x-pack/test/encrypted_saved_objects_api_integration/tests/encrypted_saved_objects_api.ts`
-
`x-pack/test/encrypted_saved_objects_api_integration/tests/encrypted_saved_objects_decryption.ts`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Part of https://github.com/elastic/kibana/issues/171570
PR breaks EsError out of data plugin and into @kbn/search-errors
package. This will allow for future efforts to move [lens error
handing](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lens/public/editor_frame_service/error_helper.tsx)
into this package and more closely align error handling across kibana.
An unplanned part of this PR was moving `BfetchRequestError` into its
own package `@kbn/bfetch`. This is required because `@kbn/search-errors`
package can not import code from a plugin. `BfetchRequestError` could
not be part of `@kbn/search-errors` because that created a circular
dependency graph.
Another unexpected problem found while working on this PR is that the
package had to be included in `ui-shared-deps`. The reason for this is
that `renderSearchError` uses `instanceof EsError` check to determine if
an error is an EsError. This check failed when the package was not
included in `ui-shared-deps` because each plugin got its own copy of
`kbn/search-error` plugin. This meant that `data` plugin had its own
copy of `EsError` class when the error is generated, then `discover`
plugin had its own copy of `EsError` class when the instanceof check
occurs. `instanceof` check unexpectedly returned false in this case.
Moving plugin to `ui-shared-deps` results in a single instance of the
plugin. Thanks @mistic for the help with moving package into
`ui-shared-deps`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
## Summary
fixes path to detection-engine tests
from
`/x-pack/test/security_solution_cypress/cypress/e2e/detection_engine
@elastic/security-detection-engine`
to
`/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine
@elastic/security-detection-engine`
## [Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts
This PR implements _Retrieval Augmented Generation_ (RAG) for Alerts in the Security Solution. This feature enables users to ask the assistant questions about the latest and riskiest open alerts in their environment using natural language, for example:
- _How many alerts are currently open?_
- _Which alerts should I look at first?_
- _Did we have any alerts with suspicious activity on Windows machines?_
### More context
Previously, the assistant relied solely on the knowledge of the configured LLM and _singular_ alerts or events passed _by the client_ to the LLM as prompt context. This new feature:
- Enables _multiple_ alerts to be passed by the _server_ as context to the LLM, via [LangChain tools](https://github.com/elastic/kibana/pull/167097)
- Applies the user's [anonymization](https://github.com/elastic/kibana/pull/159857) settings to those alerts
- Only fields allowed by the user will be sent as context to the LLM
- Users may enable or disable anonymization for specific fields (via settings)
- Click the conversation's `Show anonymized` toggle to see the anonymized values sent to / received from the LLM:
### Settings
This feature is enabled and configured via the `Knowledge Base` > `Alerts` settings in the screenshot below:
- The `Alerts` toggle enables or disables the feature
- The slider has a range of `10` - `100` alerts (default: `20`)
When the setting above is enabled, up to `n` alerts (as determined by the slider) that meet the following criteria will be returned:
- the `kibana.alert.workflow_status` must be `open`
- the alert must have been generated in the last `24 hours`
- the alert must NOT be a `kibana.alert.building_block_type` alert
- the `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts
### Feature flag
To use this feature:
1) Add the `assistantRagOnAlerts` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below:
```
xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts']
```
2) Enable the `Alerts` toggle in the Assistant's `Knowledge Base` settings, per the screenshot below:
## How it works
- When the `Alerts` settings toggle is enabled, http `POST` requests to the `/internal/elastic_assistant/actions/connector/{id}/_execute` route include the following new (optional) parameters:
- `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default`
- `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g. `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]`
- `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]`
- `replacements`, a `Record<string, string>` of replacements (generated on the server) that starts empty for a new conversation, and accumulates anonymized values until the conversation is cleared, e.g.
```json
"replacements": {
"e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm",
"cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia",
"7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s",
"f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc",
// ...
},
```
- `size`, the numeric value set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20`
- The `postActionsConnectorExecuteRoute` function in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts` was updated to accept the new optional parameters, and to return an updated `replacements` with every response. (Every new request that is processed on the server may add additional anonymized values to the `replacements` returned in the response.)
- The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts` previously used a hard-coded array of LangChain tools that had just one entry, for the `ESQLKnowledgeBaseTool` tool. That hard-coded array was replaced in this PR with a call to the (new) `getApplicableTools` function:
```typescript
const tools: Tool[] = getApplicableTools({
allow,
allowReplacement,
alertsIndexPattern,
assistantLangChain,
chain,
esClient,
modelExists,
onNewReplacements,
replacements,
request,
size,
});
```
- The `getApplicableTools` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/index.ts` examines the parameters in the `KibanaRequest` and only returns a filtered set of LangChain tools. If the request doesn't contain all the parameters required by a tool, it will NOT be returned by `getApplicableTools`. For example, if the required anonymization parameters are not included in the request, the `open-alerts` tool will not be returned.
- The new `alert-counts` LangChain tool returned by the `getAlertCountsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_tool.ts` provides the LLM the results of an aggregation on the last `24` hours of alerts (in the current Kibana Space), grouped by `kibana.alert.severity`. See the `getAlertsCountQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_query.ts` for details
- The new `open-alerts` LangChain tool returned by the `getOpenAlertsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_tool.ts` provides the LLM up to `size` non-building-block alerts generated in the last `24` hours (in the current Kibana Space) with an `open` workflow status, ordered by `kibana.alert.risk_score` to prioritize the riskiest alerts. See the `getOpenAlertsQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_query.ts` for details.
- On the client, a conversation continues to accumulate additional `replacements` (and send them in subsequent requests) until the conversation is cleared
- Anonymization functions that were only invoked by the browser were moved from the (browser) `kbn-elastic-assistant` package in `x-pack/packages/kbn-elastic-assistant/` to a new common package: `x-pack/packages/kbn-elastic-assistant-common`
- The new `kbn-elastic-assistant-common` package is also consumed by the `elastic_assistant` (server) plugin: `x-pack/plugins/elastic_assistant`
## Summary
This PR adds a bunch of plugins to help manage AI Assistant Management
settings.
It offers a 'selection' plugin inside Stack Management where a user can
select which AI Assistant she wants to manage.
The Security team can hook into this one, so settings for both AI
Assistants can be accessed from inside one place inside Stack
Management.
This PR also adds the plugin to manage settings for the AI Assistant for
Observability, including Knowledge Base management. This plugin is
available both in Stack Management (stateful) and Project Settings
(serverless).
## What it looks like
51392ec5-05c9-4947-9bf2-810d8d0b7525
## Detailed
1. **Adds a Stack Management plugin**
(`/src/plugins/ai_assistant_management/selection`). Its primary function
is to render a selection screen to help users navigate to the settings
plugin for the AI Assistant for a specific solution. This plugin is
displayed in Stack Management, which is only available in stateful
versions of Kibana.
2. **Adds a AI Assistant for Observability Settings plugin**
(`/src/plugins/ai_assistant_management/observability`). This plugin
allows management of specific Observability AI Assistant settings. It is
available in stateful versions of Kibana (via the aforementioned Stack
Management plugin) or in serverless versions via Project Management.
3. **Knowledge Base management for Observability AI Assistant**: The AI
Assistant for Observability Settings plugin has a Knowledge Base tab,
which allows users to add / read / update / delete and bulk import
entries into the Knowledge Base of the Observability AI Assistant.
4. **Moving of KB endpoints in Observability AI Assistant plugin**: KB
endpoints and functions were located in the same folder. As this PR adds
new endpoints for the KB for CRUD operations, it also moves the existing
ones from the function folder into a dedicated one so there's a clearer
distinction between kb and functions.
5. **Adding of GenAI Connector inside Chat Flyout**: If the user has
admin rights, it is possible to set up a GenAI connector from within the
Observability AI Assistant Chat Flyout. This provides a faster and more
seamless onboarding experience. If the user does not, she will be
redirected to the Settings page.
## Bug fixes
* Fixes chat item styling issues (padding, background color).
## How to test
* Check if the Stack Management plugin works on stateful
* Check if the AI Assistant Settings plugin works on stateful +
serverless
* Check if CRUD operations on KB work
* Check if searching on KB entries work
* Check if its possible to navigate to KB tab directly
(`app/management/kibana/aiAssistantManagementObservability?tab=knowledge_base`)
## Todo
- [x] Add sorting to getEntries
- [x] Add params for tab routing
- [x] Add unit tests
- [ ] Add API tests
- [ ] Add fallback for already indexed entries when searching
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR:
* extracts `enterprise_search` index documents component to common
package.
* Uses EUI pagination, converting Elasticsearch`from` & `size` to EUI
pagination standard for documents list
### Screen Recording
f585d9cc-f92c-44f4-aead-23c75c107a0f
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Partially resolves: https://github.com/elastic/kibana/issues/164255,
this is 2/3 of the scoped query changes.
Maintenance window scoped query frontend changes. Adds the ability to
add and edit scoped query for maintenance windows. Due to limitations
with the alerts search bar and each solution fetches AAD fields, we only
allow users to associate scoped query with 1 category (manangement,
o11y, or security solution). The intended usage in this case is for the
user to create multiple maintenance windows if they wish to apply scoped
queries to multiple solutions.
### To test:
go to
`x-pack/plugins/alerting/public/pages/maintenance_windows/constants.ts`
and set `IS_SCOPED_QUERY_ENABLED` to `true`
### Scoped query off, multiple category allowed:
### Scoped query on, multiple category disallowed:
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Create `security_solution/common/api/entity_analytics` to house all
entity analytics related apis, in this case `risk_score`, `risk_engine`
and `asset_criticality`.
This makes ownership clearer, simplifies the codeowners file, and will
enable the entity analytics team to have shared schema components in
`security_solution/common/api/entity_analytics/common` (something I want
to do in an upcoming PR)
I have removed `@elastic/security-threat-hunting-explore`'s ownersip of
`/x-pack/plugins/security_solution/common/api/risk_score` let me know if
you want to keep it!
Closes#171613
## Summary
This PR adds the viewInApp URL to the custom threshold rule type. This
URL will send the user to the log explorer with the selected data view
and the rule's query filter. If there is only one document aggregation,
then the filter related to this aggregation will be added as shown
below:
|Rule|Discover with pre-fill data|
|---|---|
||
For the ad-hoc data view, you should be able to see the selected index
pattern in discover similar to this:
<img
src="046493ae-ba59-46b7-a40f-68d1836d43f1"
width=400 />
### 🧪 How to test
- Check the viewInApp URL both in action variables and the alert table
for the following scenarios:
- A rule with a persisted data view
- A rule with an ad-hoc data view
- A rule with count aggregation and filter
- A rule with an optional query filter
- A rule with non-count aggregation
In all the above scenarios, the starting time in the Discover should be
before the alert's start time.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds upsert, read and delete APIs for asset criticality records. I have
used the OpenAPI code generation to create the types and zod schemas.
The APIs added are as follows:
**POST /internal/risk_score/criticality**
Request Body:
```
{
id_value: "host-1",
id_field: "host.name",
criticality_level: "very_important"
}
```
If the record already exists it will be overwritten, otherwise created
**GET
/internal/risk_score/criticality?id_field=host.name&id_value=host-1**
Response body:
```
{
id_value: "host-1",
id_field: "host.name",
criticality_level: "very_important"
@timestamp: "2023-11-29T11:43:43.175Z"
}
```
**DELETE
/internal/risk_score/criticality?id_field=host.name&id_value=host-1**
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Consolidates UI elements and backend code to create/delete data views
and destination indices related to transforms and data frame analytics.
We ended up with two different approaches for creating data views in the
wizards for transforms and data frame analytics, the original reason was
we were not aware of the `allowNoIndex: true` setting and worked around
that in different ways.
This PR aligns UI workflows and moves related code to a new package
`@kbn/ml-data-view-utils` for data views and
`@kbn/ml-creation-wizard-utils` for the destination index form. The
latter might be used for other shared components across wizard..
In Data Frame Analytics, the checkbox to create a data view was removed
from the last "Create" step, instead the option to create a data view
was moved to the "Details" step.
In Transforms, the UI component to create the destination index was
brought over from DFA where there is a switch option to automatically
use the job ID as the name for the destination index by default.
**Addresses:** https://github.com/elastic/security-team/issues/7981
## Summary
This PR adds an OpenAPI spec bundler to simplify integration with the
Docs Engineering team. The bundler produces a single bundled file by
dereferencing and inlining some of external references and bundling them
and paths into a single file.
## Details
Currently we maintain a number of schema files inside
`x-pack/plugins/security_solution/common/api/**.schema.yaml` and it
might be hard for external teams to keep track of all the changes in our
schemas. By creating a singular schema file, we provide a clear
integration point for others.
The bundler addresses the following issues
- hide endpoints that we don't want to expose (Endpoints related to
features hidden under a feature flag and all internal endpoints should
be excluded from the file)
- hide not finished data structures related to features hidden under a
feature flag or data structures that are not designed to be public (For
example `RuleActionAlertsFilter` or `RuleActionParams` are exposed
directly from the Alerting framework and might be considered
implementation details, we don't want to document interfaces that are
not designed to be public so hiding them is a good option)
- modify spec based on presence of `x-modify` property (Instead of
exposing `x-modify: partial` we need to make the exported data structure
partial and instead of exposing `x-modify: required` we need to make the
exported data structure required)
- remove any internal attributes used for code generation like
`x-codegen-enabled` and `x-modify`
- inline some of the reused data structures (We have a lot of low-level
reusable data structures `in common_attributes.schema.yaml` which might
make the final documentation hardly usable from the UX perspective, so
we can inline them)
and lives in a new `@kbn/openapi-bundler` package under
`packages/kbn-openapi-bundler` folder.
### Related changes
- Implicit version type `version: 2023-10-31` has been changed to
explicit string type `version: '2023-10-31'` for all specs under
`security_solution/common/api` folder. Implicit type causes `js-yaml`
parsing it as a `Data` JS object leading to serializing it like
`2023-10-31T00:00:00.000Z`.
- `ListRequestQuery` schema in
`security_solution/common/api/endpoint/actions/list.schema.yaml ` has
been renamed to `EndpointActionListRequestQuery` to avoid conflicts with
`ListRequestQuery` in
`security_solution/common/api/endpoint/metadata/list_metadata.schema.yaml`.
While it's not an issue to have completely different schemas sharing the
same name in different files it may be an indication of pitfalls in the
API design. I'd say it's an open question if such cases need to be
always resolved automatically or reviewed manually. At this moment the
bundler can't resolve such conflicts.
## How to test?
There is a a new JS script added to Security Solution plugin located at
`x-pack/plugins/security_solution/scripts/openapi/bundle.js` with a
corresponding entry in `package.json` named `openapi:bundle`.
To test the PR change directory to Security Solution plugin's root
folder and run the bundler like below
```sh
cd x-pack/plugins/security_solution
yarn openapi:bundle
```
It should produce a bundled OpenAPI spec at
`x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml`.
## Open issues
- [x] Circular references (implemented in
dfdf0a51ea)
- [x] Mix of OpenAPI `3.0` and `3.1` specs (Maybe convert automatically
to `3.1`?). Folder like OpenAPI bundling format implemented
[here](0ae7ad5abb)
allows to mix OpenAPI `3.0` and `3.1` specs.
## Improvements
- [ ] Flexible configuration
- [ ] CLI support?
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/170428
The bug this is intended to resolve requires some in-depth steps to
reproduce. Follow the instructions in the issue above. Then, merge in
this branch and compare.
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
List APIs tests
- Added a new folder under the `security_solution_api_integration`
called `lists_and_exception_lists` to hold the lists and exception lists
tests, and split the `List` APIs into two groups since the execution
time in Serverless was close to 30 mins
- Modified the
[x-pack/plugins/lists/common/schemas/response/exception_list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-4e3545fdeb8c8d9467cfa1c4aa88194e189193a92fa6f1cf5f859b1ef1beb45c),
[x-pack/plugins/lists/common/schemas/response/exception_list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-c3cc18faf07aab86e307185d41599c3596a3f8b360d3e4829591afa148283238)
,
[x-pack/plugins/lists/common/schemas/response/list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-608579ca5e65da74f41319a58d81ab12cc3d79d389b087806c7b74949fbc6cc3),
[x-pack/plugins/lists/common/schemas/response/list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-efc64eb35937a8da28fc982c527253c0923650ae4163d4bbc203d3ebc2949835)
to accept `elastic user` input because it changes in ESS and Serverless
- Deleted the `x-pack/test/lists_api_integration` folder
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the below files imports from the old `lists_api_integration`
folder to the new `lists_and_exception_lists`
```
lists_api_integration/
- exceptions/operators_data_types/date_numeric_types/date.ts
- exceptions/operators_data_types/date_numeric_types/double.ts
- exceptions/operators_data_types/date_numeric_types/float.ts
- exceptions/operators_data_types/date_numeric_types/integer.ts
- exceptions/operators_data_types/ips/ip.ts
- exceptions/operators_data_types/ips/ip_array.ts
- exceptions/operators_data_types/keyword/keyword.ts
- exceptions/operators_data_types/keyword/keyword_array.ts
- exceptions/operators_data_types/long/long.ts
- exceptions/operators_data_types/text/text.ts
- exceptions/operators_data_types/text/text_array.ts
- exceptions/workflows/create_endpoint_exceptions.ts
- exceptions/workflows/create_rule_exceptions.ts
- exceptions/workflows/find_rule_exception_references.ts
- exceptions/workflows/role_based_add_edit_comments.ts
- exceptions/workflows/role_based_rule_exceptions_workflows.ts
- exceptions/workflows/rule_exception_synchronizations.ts
- rule_execution_logic/execution_logic/esql.ts
- rule_execution_logic/execution_logic/machine_learning.ts
- rule_execution_logic/execution_logic/new_terms.ts
- rule_execution_logic/execution_logic/query.ts
- telemetry/task_based/all_types.ts
- telemetry/task_based/detection_rules.ts
- telemetry/task_based/security_lists.ts
```
```
These files should be moved too soon to the new `lists_api_integration`
detection_engine_api_integration/security_and_spaces/group10
- import_export_rules.ts
- import_rules.ts
- perform_bulk_actions
```
- Updated the below files imports to the
`ftr_provider_context_with_spaces.d.ts`
` - risk_engine/risk_scoring_task/task_execution_nondefault_spaces.ts`
- The QA phase concluded with all tests passing successfully. 🟢
- Updated the CodeOwner file for the newly moved tests
- Add a new util file to `deleteAllExceptions` under the old
`detection_engine_api` folder since the Rule management related-tests
are still need to be moved over to the new folder
- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes partially two remaining tasks from
https://github.com/elastic/kibana/issues/168753
Fixes partially dataview issue from
https://github.com/elastic/kibana/issues/170398
It stretches to maximum approximate 60 characters if any of the labels
in the list is of this length. If the content doesn't need the container
to stretch, it doesn't do it.
<details>
<summary> Field picker in Lens</summary>
minimum width:
<img width="445" alt="Screenshot 2023-11-21 at 15 56 03"
src="2f0f8482-bd00-4ec2-bbde-cbc4f3198eed">
auto-expanded width:
<img width="575" alt="Screenshot 2023-11-21 at 15 58 22"
src="df7bab4d-0a08-4d49-8a91-9386eba15d93">
</details>
<details>
<summary>Layer data view picker in Lens</summary>
<img width="376" alt="Screenshot 2023-11-21 at 16 01 17"
src="b8a98d83-dabc-49bd-a3cc-fc3856de6d3e">
<img width="455" alt="Screenshot 2023-11-21 at 15 58 09"
src="f2c5bde8-7a4a-485f-bf97-fc2179171e50">
<img width="615" alt="Screenshot 2023-11-21 at 15 56 27"
src="0574fc6c-69a3-44e9-9d48-8d427c1c5dba">
</details>
<details> <summary>Data view picker in Unified Search</summary>
<img width="341" alt="Screenshot 2023-11-21 at 16 00 29"
src="1c838ded-0dc5-4632-94e4-1d94586f667c">
<img width="441" alt="Screenshot 2023-11-21 at 15 58 04"
src="87e4f1c0-7922-4b94-a114-f23ece544395">
<img width="561" alt="Screenshot 2023-11-21 at 15 56 20"
src="3ea0f222-5241-4c5b-b00b-4311972754cc">
</details>
<details>
<summary> Data view picker in dashboard Create control flyout</summary>
<img width="677" alt="Screenshot 2023-11-21 at 16 14 00"
src="0455b6ed-555d-4cff-9e34-0de377be6e04">
<img width="682" alt="Screenshot 2023-11-21 at 15 54 56"
src="2a67685c-379d-4c0b-bf56-dbf7c35b3bd4">
</details>
<details>
<summary> Unified search data view select component (tested in
maps)</summary>
<img width="570" alt="Screenshot 2023-11-22 at 14 38 25"
src="bb52ab22-626d-4556-b40c-c9bcc925f426">
</details>
<details>
<summary>Unified search field and value picker</summary>
Adds `panelMinWidth`, removes the custom flex width change behavior
<img width="1142" alt="Screenshot 2023-11-22 at 14 40 26"
src="2450957f-38b7-4a73-b531-7acb29cb56bc">
f4f33624-9287-403e-8472-81f705440f97
</details>
<details>
<summary> Discover breakdown field</summary>
Removes the focus stretching and instead uses the panelMinWidth prop
<img width="419" alt="Screenshot 2023-11-21 at 16 46 50"
src="e35125ad-8823-4bff-954b-8119a352829c">
<img width="619" alt="Screenshot 2023-11-21 at 16 48 20"
src="89e63daf-a59e-43e1-a6ec-91d1b15b0fcd">
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
In this PR, I'm relocating all Kibana Security types (along with a few
schemas necessary for some of these types, unfortunately) that are part
of public contracts to separate packages. This change will enable any
plugin to utilize Security APIs via "static" or
["runtime"](https://github.com/elastic/kibana/pull/167113) dependencies,
regardless of whether Kibana Security already relies on these plugins or
not.
__NOTE TO REVIEWERS:__ I tried to minimize changes as much as I could
via moving only necessary types. I also didn't move deprecated parts of
the Setup/Start contracts to these new packages.
__Triggered by:__ https://github.com/elastic/kibana/pull/168910
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The package data isn't needed anymore, so we can remove that package. On
top, it seems like the package was causing some issues.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Attempting to merge #170852 again now that the release artefact step has
been fixed as part of https://github.com/elastic/kibana/pull/171457
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
basic detection engine-related tests
- Introduced a new license folder to hold the `Basic` Ess tests and the
`Essentials` Serverless tests, is called `basic_essentials_license`
- Added new base configurations files for
`serverless/config.base.essentials` and `ess/config.base.basic`
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the CodeOwner file for the newly moved tests
- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
- The **Privileges** Tests are skipped in Serverless now until the FTR
Roles [PR](https://github.com/elastic/kibana/pull/170131) gets merged
| Action | File | New Path |
|--------|------|----------|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/rules/create_rules|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/rules/create_ml_rules_privileges|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/alerts/open_close_alerts|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/alerts/query_alerts_backword_compatibility|
| Moved|basic/create_rules|
basic_essentials_license/detection_engine/alerts/query_alerts|
## Summary
Security Data Analytics would like to know when these files are updated.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Adds the ability to quickly create a categorisation anomaly detection
job from the pattern analysis flyout.
Adds a new `created_by` ID `categorization-wizard-from-pattern-analysis`
which can be picked up by telemetry.
Creates a new package for sharing our AIOPs ui actions IDs. I think we
should move the pattern analysis ID to this package too, but that can be
done in a separate PR.
51349f93-f072-4983-85f0-98741902fb5a
6e618581-8916-4e63-930f-945c96c25e6c
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Update new user details flyout to be consistent with Expandable Alerts
Flyout. The previous user details flyout implementation was hidden
behind a flag and never went live.
### What is included
* Update new user details flyout to use the expandable flyout component
* Update UI components according to the new design
* Keep the feature hidden behind newUserDetailsFlyout flag
* Supporting alert risk inputs
### What is NOT included
* Supporting multiple categories of risk inputs
* Host details flyout
* User and host pages
* Asset integrations (okta and azure)
* Update the flyout on the timeline (It is currently a technical
restriction of the expandable flyout, but the team is working to fix it)
### How to test it?
* Enable experimental flag `newUserDetailsFlyout`
`xpack.securitySolution.enableExperimental: ['newUserDetailsFlyout']`
* Create alerts and open alerts page
* Click on a username
- [x] Test edge cases
- [x] No cases permissions (it hides cases actions)
- [x] Basic license (it hides the risk score summary)
- [x] No risk score data for a user (It hides the risk score summary)
<img width="434" alt="Screenshot 2023-11-13 at 15 56 33"
src="4fc13042-cd3d-487b-9982-bfbf02f003b4">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
## 📓 Summary
Closes#170728
This work comes from the need to use agent and cloud provider icons in
the new Log Detail flyout.
Since those icons were already used across the `infra` and `apm`
plugins, this was a good opportunity to extract the shared logic into
packages.
The results of this refactoring are two new packages:
- **@kbn/elastic-agent-utils**: exports small utilities and type
definition used to parse the icon to render and exploits also across the
APM plugin.
- **@kbn/custom-icons**: exports custom icons built on top of EuiIcon,
encapsulating logic related to mapping from data to the relative icon.
Apart from creating the new plugins, this also applies their usage to
the `infra` and `apm` plugins, while the Log Explorer flyout will
benefit from these working on
https://github.com/elastic/kibana/issues/170721.
## 🧪 How to test
### Infra
- Navigate to `Infrastructure -> Hosts`
- Verify the hosts table correctly renders the cloud provider icon for
each table entry.
### APM
- Navigate to `APM -> Services`.
- Verify each table entry correctly displays the related agent icon.
- Navigate to `APM -> Services`.
- Click on a service where t a cloud provider icon is expected to appear
next to the service name.
- Verify the icon is correctly displayed.
- Navigate to `APM -> Services -> Service Map`.
- Create a new group.
- Verify the agent icon is correctly displayed for each entry in the
preview list.
- Navigate to `APM -> Traces`.
- Verify each table entry correctly displays the related agent icon.
- Navigate to `APM -> Settings -> Agent Explorer`.
- Verify each table entry correctly displays the related agent icon.
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
Telemetry
- Moved the utility files associated with telemetry to the new directory
`security_solution_api_integration`. Files not actively used in the
previous folder were moved, while duplicate files remained in their
original positions.
- Updated the CodeOwner file for the newly moved tests
- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
| Action | File | New Path if moved |
|--------|------|----------|
| Delete| security_and_spaces/group4| - |
|
Move|detection_engine_api_integration/security_and_spaces/group4|detections_response/default_license/telemetry/keyword_family|
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
Privileges related tests and Action Throttle
- Moved the utility files associated with telemetry to the new directory
`security_solution_api_integration`. Files not actively used in the
previous folder were moved, while duplicate files remained in their
original positions.
- Updated the CodeOwner file for the newly moved tests
- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
- The **Privileges** Tests are skipped in Serverless now until the FTR
Roles [PR](https://github.com/elastic/kibana/pull/170131) gets merged
| Action | File | New Path if moved |
|--------|------|----------|
| Delete| security_and_spaces/group4| - |
|
Move|detection_engine_api_integration/security_and_spaces/group10/throttle|detections_response/default_license/action/throttle|
|
Move|detection_engine_api_integration/security_and_spaces/group1/check_privileges|detections_response/default_license/actions/check_privileges|
|
Move|detection_engine_api_integration/security_and_spaces/group10/read_privileges|detections_response/default_license/user_roles/read_privileges|
Closes - https://github.com/elastic/kibana/issues/170133
## Summary
This PR adds support for generating logs using the Synthtrace Client
Changes include
1. Changes to Synthtrace package to support new Logs Client and Log
Class for helper methods
2. [Stateful Tests] - Change to our FTR Context config to inject the new
the Log Synthtrace Client
3. [Serverless Tests] - Injected Synthtrace as a service for serverless
tests.
4. A sample test added to `app.ts` to demonstrate how Synthtrace can be
used to generate Log data in both Stateful and Serverless
5. Add support to generate logs via CLI. 2 scenarios added -
`simple_logs.ts` and `logs_and_metrics.ts`
```
# Live Data
node scripts/synthtrace simple_logs.ts --clean --live
# Static Data
node scripts/synthtrace simple_logs.ts --clean
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Yngrid Coello <yngrdyn@gmail.com>
## Summary
This PR refactors the export type classes into their own packages to be
then instantiated in the reporting plugin. This will reduce bloat in the
central reporting plugin.
**Main packages**
- `kbn/reporting-export-types-{png,pdf,csv}` are server packages with
export type declarations
- `kbn/reporting-export-types-{png,pdf,csv}-common` are shared common
packages with type declarations and constants
**Other changes**
- Remove `reporting.getScreenshots()`
- Remove duplicated `schema_utils.ts`
- Consolidate `JOB_STATUS` declaration as an enum
<img width="1063" alt="image"
src="bced8321-93c5-4ebd-b31e-1fd946166241">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Related to [#166340](https://github.com/elastic/kibana/issues/166340)
## Summary
Add mock identity provider and utils to test serverless user roles.
## Screenshot
### 1. Login selector
<img width="767" alt="Screenshot 2023-11-08 at 15 18 18"
src="82b4a29f-65b4-45d2-bed3-6d9f74043c48">
### 2. Single sign on screen
<img width="437" alt="Screenshot 2023-11-09 at 12 30 46"
src="3d5b6f26-5409-4169-a627-bcf6d09836d9">
### 3. User profile page
<img width="1041" alt="Screenshot 2023-11-08 at 17 36 22"
src="50bd4a5a-f9a8-4643-9384-9a352701b011">
## Testing
SAML is only supported by ES when running in SSL mode.
1. To test the mock identity provider run a serverless project in SSL
mode using:
```bash
yarn es serverless --ssl
yarn start --serverless=es --ssl
```
2. Then access Kibana and login in using "Continue as Test User".
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## 📓 Summary
Closes#170938
This work split the `test/functional/apps/infra` test folder accordingly
with the new ownership.
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
