Commit graph

28598 commits

Author SHA1 Message Date
Ash
fa5e944f50
[Security Solution][Endpoint] Re-enable alerts response console test (#172152)
## Summary

Unskip test

**Flaky runner**
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4095
x 100 ( all pass )

### Checklist

Delete any 
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Signed-off-by: Ash <1849116+ashokaditya@users.noreply.github.com>
2023-11-29 17:39:33 +01:00
Dario Gieselaar
450b2c5a2d
[Obs AI Assistant] Refactor hooks, recall on every message (#171965)
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-29 09:27:06 -07:00
Jordan
3437e6d878
[Cloud Security] Azure integration manual fields (#171069) 2023-11-29 09:18:29 -07:00
Ignacio Rivas
d3e9ab7cea
[Index Management] Add index component templates api tests (#171981) 2023-11-29 17:10:09 +01:00
Mykola Harmash
ad2ca2443c
[ObsUX] Add Top Functions to Infra Profilling tab (#171974)
Closes https://github.com/elastic/kibana/issues/171962

## Summary

This integrated Profiling Top Functions embeddable into the Infra's
Profiling tab in asset details.

![CleanShot 2023-11-28 at 14 20
38@2x](408ca866-1bc9-4b66-9ba1-d090cce0f7da)

## How to Test

* Connect local kibana to oblt cluster that has Profiling configured
(e.g. edge)
* Add this to your dev `kibana.yml`
```
xpack.profiling.enabled: true
xpack.infra.profilingEnabled: true

# Direct ES URL on the oblt cluster that you're using, in case of edge it's https://edge-oblt.es.us-west2.gcp.elastic-cloud.com:443
xpack.profiling.elasticsearch.hosts: REMOTE_CLUSTER_ES_URL

# If needed create a new user on the remote oblt cluster
xpack.profiling.elasticsearch.username: REMOTE_CLUSTER_USER
xpack.profiling.elasticsearch.password: REMOTE_CLUSTER_PASWORD
```

* Open kibana, go to Hosts
* Open a flyout for one of the hosts and make sure you see the Profiling
tab with both Flamegraph and Top Functions
* Open Host details as a full page and also make sure you see the same
* Make sure Profiling data updates when you change dates in the date
picker
2023-11-29 16:59:37 +01:00
Mike Pellegrini
cca28e7493
Add model name to default pipeline name (#172110)
Update the default pipeline name to include the model name, like so:

`ml-inference-<index name>-<model name>`

 Unsupported characters in the model name are mapped to `_`.
2023-11-29 10:36:42 -05:00
Ash
5412584188
[Security Solution][Endpoint] Re enable document signing test (#172154)
## Summary

Re-enable document signing test

**Flaky runner**
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4096
x 100 ( all pass )

### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Signed-off-by: Ash <1849116+ashokaditya@users.noreply.github.com>
2023-11-29 08:29:25 -07:00
Maxim Palenov
e3d95e9b72
[Security Solution] OpenAPI docs bundler (#171526)
**Addresses:** https://github.com/elastic/security-team/issues/7981

## Summary

This PR adds an OpenAPI spec bundler to simplify integration with the
Docs Engineering team. The bundler produces a single bundled file by
dereferencing and inlining some of external references and bundling them
and paths into a single file.

## Details

Currently we maintain a number of schema files inside
`x-pack/plugins/security_solution/common/api/**.schema.yaml` and it
might be hard for external teams to keep track of all the changes in our
schemas. By creating a singular schema file, we provide a clear
integration point for others.

The bundler addresses the following issues

- hide endpoints that we don't want to expose (Endpoints related to
features hidden under a feature flag and all internal endpoints should
be excluded from the file)
- hide not finished data structures related to features hidden under a
feature flag or data structures that are not designed to be public (For
example `RuleActionAlertsFilter` or `RuleActionParams` are exposed
directly from the Alerting framework and might be considered
implementation details, we don't want to document interfaces that are
not designed to be public so hiding them is a good option)
- modify spec based on presence of `x-modify` property (Instead of
exposing `x-modify: partial` we need to make the exported data structure
partial and instead of exposing `x-modify: required` we need to make the
exported data structure required)
- remove any internal attributes used for code generation like
`x-codegen-enabled` and `x-modify`
- inline some of the reused data structures (We have a lot of low-level
reusable data structures `in common_attributes.schema.yaml` which might
make the final documentation hardly usable from the UX perspective, so
we can inline them)

and lives in a new `@kbn/openapi-bundler` package under
`packages/kbn-openapi-bundler` folder.

### Related changes

- Implicit version type `version: 2023-10-31` has been changed to
explicit string type `version: '2023-10-31'` for all specs under
`security_solution/common/api` folder. Implicit type causes `js-yaml`
parsing it as a `Data` JS object leading to serializing it like
`2023-10-31T00:00:00.000Z`.
- `ListRequestQuery` schema in
`security_solution/common/api/endpoint/actions/list.schema.yaml ` has
been renamed to `EndpointActionListRequestQuery` to avoid conflicts with
`ListRequestQuery` in
`security_solution/common/api/endpoint/metadata/list_metadata.schema.yaml`.
While it's not an issue to have completely different schemas sharing the
same name in different files it may be an indication of pitfalls in the
API design. I'd say it's an open question if such cases need to be
always resolved automatically or reviewed manually. At this moment the
bundler can't resolve such conflicts.

## How to test?

There is a a new JS script added to Security Solution plugin located at
`x-pack/plugins/security_solution/scripts/openapi/bundle.js` with a
corresponding entry in `package.json` named `openapi:bundle`.

To test the PR change directory to Security Solution plugin's root
folder and run the bundler like below

```sh
cd x-pack/plugins/security_solution
yarn openapi:bundle
```

It should produce a bundled OpenAPI spec at
`x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml`.

## Open issues

- [x] Circular references (implemented in
dfdf0a51ea)
- [x] Mix of OpenAPI `3.0` and `3.1` specs (Maybe convert automatically
to `3.1`?). Folder like OpenAPI bundling format implemented
[here](0ae7ad5abb)
allows to mix OpenAPI `3.0` and `3.1` specs.

## Improvements

- [ ] Flexible configuration
- [ ] CLI support?

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-29 14:07:31 +01:00
Marta Bondyra
37d5aca592
[Maps] Fix long field truncation on Comboboxes (#171829)
## Summary

Fixes https://github.com/elastic/kibana/issues/171509 (fixed in all the
places in maps)

Adds middle truncation and combobox auto-expansion to the content to
approximate of 60 character maximum for `SingleFieldSelect` and
`FieldSelect` components in maps.
I removed custom `renderOption` prop so the combobox can take care of
the proper truncation while searching through the fields. One case I had
to hack was to display a tooltip for a disabled state. I used `prepend`
and some custom styling to do so - it works as before (check out the
screenshot below).

<details>
 <summary> FieldSelect component before</summary>
<img width="471" alt="Screenshot 2023-11-22 at 16 08 21"
src="eb8a682a-0dfe-4bb0-9a88-8dc5863a5aff">

</details>

<details>
 <summary> FieldSelect component after</summary>
<img width="582" alt="Screenshot 2023-11-22 at 16 10 02"
src="b840fb7e-0789-46b7-b783-8e0aadb270ac">

</details>

<details>
 <summary> SingleFieldSelect component before</summary>
<img width="430" alt="Screenshot 2023-11-23 at 10 23 15"
src="fc4af30c-32ad-40d8-a051-4b123be8aa54">
</details>

<details>
 <summary> SingleFieldSelect component after</summary>
<img width="576" alt="Screenshot 2023-11-22 at 16 32 43"
src="2c3f496e-2e1b-4cf9-aa5b-f38c7a7eace2">

#### disabled state 
<img width="829" alt="Screenshot 2023-11-22 at 17 10 48"
src="8db0b076-d3f5-4ec4-9e7c-0e7121b14e04">

</details>

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-29 13:49:41 +01:00
Sander Philipse
ad177dd94f
[Search] Remove indices callout and rename Home nav item (#172103)
## Summary

Removes the indices callout that's been here for over a year, and
renames the Overview nav item to Home.
2023-11-29 11:59:19 +01:00
Stratoula Kalafateli
cb3fd2132f
[ES|QL] Make the dashboard SO lighter (#172130)
## Summary

This PR makes the dashboard SO lighter by removing the initialContext
from the state. It is not needed after the navigation from Discover to
Dashboard and can create SO size problems as it contains a lot of
information for the transition.
2023-11-29 12:55:21 +02:00
Julia Bardi
2ff6a7cfd4
[Fleet] fixed bug in output secret diff logic (#172081)
## Summary

Related to https://github.com/elastic/kibana/issues/104986

Found a bug in `diffOutputSecretPaths` where output secret was deleted
if updating an output without change of service_token. Added unit tests
to cover the logic.

Steps to verify:
- enable feature flags: `xpack.fleet.enableExperimental:
['remoteESOutput', 'outputSecretsStorage']`
- create a remote es output with a service_token
- check that the service_token is stored as secret in `.fleet-secrets`
- update host in remote es output
- verify that the secret is not deleted


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2023-11-29 03:37:41 -07:00
Marco Antonio Ghiani
6bdc094d79
[Logs Explorer] Increase Log details flyout size (#172045)
## 📓 Summary

Closes #171847 

This work adds a new `size` parameter to the customization extension
point for the `flyout`, enabling consumers to increase/decrease it based
on requirements.

It also set the Log Explorer size to `60%` since both `m` and `l` sizes
didn’t fit with the designed size.

Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
2023-11-29 11:08:18 +01:00
Panagiota Mitsopoulou
2914d25a29
[SLO] hide service details link for read only users (#170731)
Fixes https://github.com/elastic/kibana/issues/170610
2023-11-29 02:46:09 -07:00
Tomasz Ciecierski
f085621576
[EDR Workflows] Fix failing test regarding flyout issue in alerts (#172043) 2023-11-29 10:25:34 +01:00
Sebastián Zaffarano
f20e8357e3
Increment axios timeout sending events (#172062) 2023-11-29 02:25:02 -07:00
Antonio
e333c6ee2e
[Cases] Suggest user profiles with read permission (#172047)
Fixes #171446

## Summary

We will now also suggest users with only `Read Cases` permissions for
case assignment.

## Release notes

Fixed a bug that prevented users with read permission from being
assigned to cases.
2023-11-29 08:56:30 +01:00
Alejandro Fernández Haro
ac16c65bae
[Elasticsearch] Limit maxSockets to 800 by default (#151911)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
2023-11-28 23:17:39 -05:00
Adam Demjen
982b447145
[Enterprise Search] Add fetch trained ML models API (#172084)
## Summary

This PR adds an API to the Kibana backend for fetching ML models. The
model objects in the response encapsulate all necessary info for
rendering and managing models in the Search->Pipelines tab.

The API
- fetches deployed ML models via the ML plugin
- combines fetched models with placeholders for promoted models (ELSER,
E5)
- enriches model information with user-friendly title and deployment
status
- filters unsupported models and sorts the result list

Sample request/response:
```json
GET /internal/enterprise_search/ml/models

[
  {
    "deploymentState": "fully_downloaded",
    "nodeAllocationCount": 0,
    "startTime": 0,
    "targetAllocationCount": 0,
    "threadsPerAllocation": 0,
    "isPlaceholder": false,
    "hasStats": false,
    "modelId": ".elser_model_2",
    "type": "text_expansion",
    "title": "Elastic Learned Sparse EncodeR (ELSER)",
    "description": "ELSER is designed to efficiently use context in natural language queries with better results than BM25 alone.",
    "license": "Elastic",
    "isPromoted": true
  },
  {
    "deploymentState": "fully_allocated",
    "nodeAllocationCount": 1,
    "startTime": 1700859252106,
    "targetAllocationCount": 1,
    "threadsPerAllocation": 1,
    "isPlaceholder": false,
    "hasStats": true,
    "modelId": ".multilingual-e5-small",
    "type": "text_embedding",
    "title": "E5 Multilingual Embedding",
    "description": "Multilingual dense vector embedding generator.",
    "license": "MIT",
    "modelDetailsPageUrl": "https://huggingface.co/intfloat/multilingual-e5-small",
    "isPromoted": true
  },
  {
    "deploymentState": "",
    "nodeAllocationCount": 0,
    "startTime": 0,
    "targetAllocationCount": 0,
    "threadsPerAllocation": 0,
    "isPlaceholder": false,
    "hasStats": false,
    "modelId": "sentence-transformers__msmarco-minilm-l-12-v3",
    "type": "text_embedding",
    "title": "Dense Vector Text Embedding",
    "isPromoted": false
  },
  {
    "deploymentState": "fully_allocated",
    "nodeAllocationCount": 0,
    "startTime": 0,
    "targetAllocationCount": 0,
    "threadsPerAllocation": 0,
    "isPlaceholder": false,
    "hasStats": false,
    "modelId": "lang_ident_model_1",
    "type": "classification",
    "title": "Lanugage Identification",
    "isPromoted": false
  },
  {
    "deploymentState": "",
    "nodeAllocationCount": 0,
    "startTime": 0,
    "targetAllocationCount": 0,
    "threadsPerAllocation": 0,
    "isPlaceholder": false,
    "hasStats": false,
    "modelId": "samlowe__roberta-base-go_emotions",
    "type": "text_classification",
    "title": "Text Classification",
    "isPromoted": false
  }
]
```

### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2023-11-28 19:47:46 -05:00
Lisa Cawley
797694df26
[Response Ops][Alerting] Edit text in Elasticsearch query rule and index threshold rule (#171896) 2023-11-28 17:37:29 -07:00
Ersin Erdal
59eb6141e2
Add rule model versions in alerting (#171927)
Towards: #166967

This PR adds `ruleModelVersion` to saved object registry and
`latestRuleVersion` to ruleTypeRegistry.
These new assets will be used in a follow-on PR for skipping the rule
task executions when there is version mismatch.

POC for the issue: https://github.com/elastic/kibana/pull/167128

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 22:50:49 +01:00
Khristinin Nikita
45e88fea3e
Asset cricitality init (#171324)
##  Introduce Asset criticality initialisation and refactor risk engine

### Asset criticality 
use `entityAnalyticsAssetCriticalityEnabled` for testing

Added `AssetCriticalityDataClient` which will create index/mappings for
the risk engine.

### Refactor risk engine

As you can see in this
[discussion](https://github.com/elastic/kibana/pull/171324#discussion_r1394461582)
there raised some concerns about `RiskEngineDataClient` has a lot of
responsibilities.

So in this PR, I took out risk scoring functionality from
`RiskEngineDataClient` to `RiskScoreDataClient`.

### Changes inside `entity_analytics` folder

`risk_engine` folder and `RiskEngineDataClient` will be responsible for:
- Init risk engine and installation of all resources like, SO, removing
legacy dashboards, and calling `RiskScoreDataClient` and
`AssetCriticalityDataClient` for installation of corresponding resources
- Getting the status of the risk engine
- Enable / Disable risk engine and start/remove task
- Saved object configuration manipulation

The `risk_score` folder be responsible for:
- Risk score calculation and persistence
- Task methods
- (**_new_**) `RiskScoreDataClient`  will be responsible for: 
    - resource initialisation like:
      - index template and mappings
      - risk score datasream
      - creating the latest index
      - creating transform
   - return writer for risk scores
   - return risk input index
 
The `asset_criticality` and `AssetCriticalityDataClient` folder be
responsible for:
- asset criticality index and mapping creation
- in future CRUD operations for asset criticality

`routes` folders have API routes for risk engine and risk scoring
functionality, there will be also asset criticality routes in the future

`utils` common utils

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 14:29:54 -07:00
Nicolas Chaulet
0d2d89d066
[Fleet] Remove empty condition (#172101) 2023-11-28 14:10:53 -07:00
Gerard Soldevila
98b2cfbbb0
Enhance plugin documentation (#146678)
The PR tackles a couple of improvements for the new `'notifications'`
plugin documentation:

- Add a link to the plugin API description in the kibana-dev-docs nav
bar.
- Convert the README to `mdx`.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 20:58:46 +01:00
Nathan Reese
d81c367850
[maps] fix tile errors displayed when layer is no longer using tiles (#172019)
Closes https://github.com/elastic/kibana/issues/172013

PR updates TileStatusTracker to clear tile error cache when layer is not
tiled.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 12:38:58 -07:00
Drew Tate
a8647151cb
Unify style for embeddable-stack loaders (#171238)
## Summary

Fix https://github.com/elastic/kibana/issues/170428

The bug this is intended to resolve requires some in-depth steps to
reproduce. Follow the instructions in the issue above. Then, merge in
this branch and compare.


### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 13:30:47 -06:00
Patrick Mueller
9f5651d3bf
[Task Manager] allow multiple task claiming strategies (#171677)
see https://github.com/elastic/kibana/issues/155770

Make the task manager task claiming algorithm selectable, to allow
alternative implementations in the future. No other implementations are
provided here, this is setup for adding the next algorithm. Task Manager
behavior should not be changed by this PR - code has just be re-org'd.

This exposes a new config key which is exposed to Docker -
`xpack.task_manager.claim_strategy`. The only allowed value is
`default`. No plans at present to document this, or allow-list for the
cloud. We may end up changing the config key to just test for serverless
instead, when we implement the next task claiming algorithm (see
referenced issue ^^^, which is aimed for serverless).

The jest tests were coarsely re-org'd. Once we have > 1 algorithm, we'll
like want to re-org a bit more, so we can test all the implementations
"in a loop".
2023-11-28 13:28:56 -05:00
Tiago Costa
4d1adc1b90
skip flaky suite (#171641) 2023-11-28 17:10:33 +00:00
Tiago Costa
92cd3a66e6
skip flaky suite (#171643) 2023-11-28 17:08:06 +00:00
Shahzad
42253b6aba
[Alert As Data] Add match_only_text mapping for reason field (#171969) 2023-11-28 17:58:22 +01:00
Kyle Pollich
e64f475a01
[Fleet] Support integration secrets with required: false (#172078)
## Summary

Support secrets with `required: false` in package manifests.

Closes #172061

## To test

1. Set up an integration in a local package registry with a variable
that has `secret: true` and `required: false`, e.g.

```yml
- name: secret_token
  type: password
  title: (Test) Secret Token
  description: |
    Test non-required secret
  show_user: true
  secret: true
  required: false
```

2. Create a package policy for your test package and note the optional
secret is rendered properly
3. Submit the policy editor form without filling out a value for the
optional secret
4. Observe the request is successful
5. Edit the package policy and set a value for the optional secret
6. Observe that the secret creation logic works as expected

## Screen recording


36e271c5-29d0-49f8-91e8-abc6a7871b20
2023-11-28 11:52:01 -05:00
David Sánchez
1823d94240
[Security Solution][Endpoint] New enroll endpoint host function CI specific for Cypress tests to use cached agent files (#171399)
## Summary

In order to avoid downloading the elastic agent installer file on each
Cypress test, we have introduced a new method CI specific that will
cache elastic agent files and reuse it across all tests.

Old code about `if CI` conditions will be removed in a follow up pr.

It also introduces a CLI script to download a specific version of
elastic agent using the existing methods in place.

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 17:27:25 +01:00
Ying Mao
d9ebfd9af1
[Response Ops][Alerting] Remove echoed field value from bulk error responses when indexing alerts (#172020)
## Summary

When alerts are bulk indexed in the rule registry and the alerts client,
indexing errors may be returned where the entire field value that failed
to be indexed is echoed in the reason. This can cause unnecessarily
verbose logging so we want to sanitize the field value.
2023-11-28 11:25:03 -05:00
Mike Pellegrini
e94a977873
Reorganize new/existing pipeline management screens into tabs (#172027)
Split Configure step’s components in pipeline creation into two tabs:

- Create new: pipeline name, model selection
- Use existing: pipeline selection

Keep the split components' content as-is, they are out of scope for this
task.

Remove the left hand side panels (“Create or select a pipeline”, “Select
a trained ML Model” and blurbs underneath), and make the tab full width.

Also remove the ELSER text expansion callout from the flyout.
2023-11-28 11:15:06 -05:00
Sander Philipse
1aea9c3661
[Search] Add Search Hub as overview page (#172011)
## Summary

This amends the Getting Started page in Search and turns it into the
Search Hub.
<img width="1907" alt="Screenshot 2023-11-27 at 19 14 05"
src="0634d0de-4d6c-43d1-ac16-7d6ad81b0071">
<img width="1385" alt="Screenshot 2023-11-27 at 19 14 13"
src="3fdb40dd-7984-48d3-a1f2-2b21c42fe109">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 16:50:02 +01:00
Wafaa Nasr
a4ed14bec8
[Security Solution][API testing] Move and restructures Lists APIS (#171992)
## Summary

Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
List APIs tests
- Added a new folder under the `security_solution_api_integration`
called `lists_and_exception_lists` to hold the lists and exception lists
tests, and split the `List` APIs into two groups since the execution
time in Serverless was close to 30 mins
- Modified the
[x-pack/plugins/lists/common/schemas/response/exception_list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-4e3545fdeb8c8d9467cfa1c4aa88194e189193a92fa6f1cf5f859b1ef1beb45c),
[x-pack/plugins/lists/common/schemas/response/exception_list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-c3cc18faf07aab86e307185d41599c3596a3f8b360d3e4829591afa148283238)
,
[x-pack/plugins/lists/common/schemas/response/list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-608579ca5e65da74f41319a58d81ab12cc3d79d389b087806c7b74949fbc6cc3),
[x-pack/plugins/lists/common/schemas/response/list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-efc64eb35937a8da28fc982c527253c0923650ae4163d4bbc203d3ebc2949835)
to accept `elastic user` input because it changes in ESS and Serverless
- Deleted the `x-pack/test/lists_api_integration` folder
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the below files imports from the old `lists_api_integration`
folder to the new `lists_and_exception_lists`

    ```
      lists_api_integration/  
        - exceptions/operators_data_types/date_numeric_types/date.ts
        - exceptions/operators_data_types/date_numeric_types/double.ts
        - exceptions/operators_data_types/date_numeric_types/float.ts 
        - exceptions/operators_data_types/date_numeric_types/integer.ts
        - exceptions/operators_data_types/ips/ip.ts
        - exceptions/operators_data_types/ips/ip_array.ts
        - exceptions/operators_data_types/keyword/keyword.ts
        - exceptions/operators_data_types/keyword/keyword_array.ts
        - exceptions/operators_data_types/long/long.ts
        - exceptions/operators_data_types/text/text.ts
        - exceptions/operators_data_types/text/text_array.ts
        - exceptions/workflows/create_endpoint_exceptions.ts
        - exceptions/workflows/create_rule_exceptions.ts
        - exceptions/workflows/find_rule_exception_references.ts
        - exceptions/workflows/role_based_add_edit_comments.ts
        - exceptions/workflows/role_based_rule_exceptions_workflows.ts
        - exceptions/workflows/rule_exception_synchronizations.ts
        - rule_execution_logic/execution_logic/esql.ts
        - rule_execution_logic/execution_logic/machine_learning.ts
        - rule_execution_logic/execution_logic/new_terms.ts
        - rule_execution_logic/execution_logic/query.ts
        - telemetry/task_based/all_types.ts
        - telemetry/task_based/detection_rules.ts
        - telemetry/task_based/security_lists.ts
     ```
     ```
These files should be moved too soon to the new `lists_api_integration`
        detection_engine_api_integration/security_and_spaces/group10
            - import_export_rules.ts 
            - import_rules.ts
            - perform_bulk_actions   
     ```

- Updated the below files imports to the
`ftr_provider_context_with_spaces.d.ts`
` - risk_engine/risk_scoring_task/task_execution_nondefault_spaces.ts`

- The QA phase concluded with all tests passing successfully. 🟢  

- Updated the CodeOwner file for the newly moved tests

- Add a new util file to `deleteAllExceptions` under the old
`detection_engine_api` folder since the Rule management related-tests
are still need to be moved over to the new folder

- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 16:40:09 +01:00
Nicolas Chaulet
348ef4e39c
[Fleet] Persist package upgrade errors (#171797) 2023-11-28 08:37:42 -07:00
Julien Lind
3b8b829581
Update translations.ts to fix typo (#172036)
Closes https://github.com/elastic/kibana/issues/172034
2023-11-28 07:41:36 -07:00
Navarone Feekery
8413078c31
[Search] Update Connector ACL name pattern (#172057)
## Summary

This PR changes the ACL index pattern logic.
Previously an index named `search-foo` would create an ACL index named
`.search-acl-filter-foo`. If a user wants to also use an index named
just `foo`, the ACL index generated would be identical.
These changes simplify the index name creation. Now, indices will look
like:

- `search-foo` -> `.search-acl-filter-search-foo`
- `foo` -> `.search-acl-filter-foo`

Migrations for this have been added already to Enterprise Search.
2023-11-28 07:38:16 -07:00
Paul Tavares
0a72738e4c
[Security Solution][Endpoint] server-side standard interface for response actions clients (#171755)
## Summary

PR introduces a standard interface for Response Actions clients -
currently only Endpoint, but in the near future, other clients will be
introduced like SentinelOne. This PR is in preperation for that feature
in a post v8.12 release.

Changes include:

- Introduction of `EndpointActionsClient` class (first Actions client
using new standard interface)
- Changed Response Actions API handler to:
    - use new `EndpointActionsClient` for processing response actions
- added support for handling file `upload` response action (previously a
separate handler)
    - now handles all errors using the common HTTP error handler
- Deleted `upload` specific API HTTP handler - no longer needed as
common handler will now also process `upload` response actions

**NOTE:** No changes in functionality as a result of this PR. Just
preparation work needed to support Bi-Directional Response Actions.
2023-11-28 09:36:16 -05:00
Julia Bardi
517c815c48
[Fleet] making service_token an output secret (#171875)
## Summary

Related to https://github.com/elastic/kibana/issues/104986

Making remote ES output's service_token a secret.

fleet-server change here:
https://github.com/elastic/fleet-server/pull/3051#discussion_r1406183654

Steps to verify:
- Enable remote ES output and output secrets in `kibana.dev.yml`
locally:
 ```
xpack.fleet.enableExperimental: ['remoteESOutput',
'outputSecretsStorage']
```
- Start es, kibana, fleet-server locally and start a second es locally
 - see detailed steps here: https://github.com/elastic/fleet-server/pull/3051
- Create a remote ES output, verify that the service_token is stored as a secret reference
```
GET .kibana_ingest/_search?q=type:ingest-outputs
```
- Verify that the enrolled agent sends data to the remote ES successfully

<img width="561" alt="image" src="122d9800-a2ec-47f8-97a7-acf64b87172a">
<img width="549" alt="image" src="e1751bdd-5aaf-4f68-9f92-7076b306cdfe">



### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2023-11-28 15:20:36 +01:00
Gergő Ábrahám
a6582337e1
[Defend Workflows][8.12 port] Unblock fleet setup when cannot decrypt uninstall tokens (#172058)
## Summary

This PR is the `8.12` port of:
- #171998

The original PR was opened to `8.11` to make it faster to include it in
`8.12.2`. Now this PR is meant to port the changes to `main`, so:
- we can build upon it,
- and can easily backport any further changes to `8.11.x`

> [!Important]
> The changes cannot be tested on `main` because they are hidden by
other behaviours (namely the retry logic for reading Message SIgning
key) that weren't part of `8.11`. Those behaviours will be also adapted
in follow up PRs.
2023-11-28 15:11:24 +01:00
Dzmitry Lemechko
0c49603883
[ftr] split x-pack accessibility config in 3 groups (#171186)
## Summary

Splitting long running FTR config:

<img width="1573" alt="image"
src="160b97b4-6468-4c2f-9827-17f6bb9a742a">



After split:

| Config Path | Runtime |
| ------------- | ------------- |
| x-pack/test/accessibility/apps/group1/config.ts |  10m 15s  |
| x-pack/test/accessibility/apps/group2/config.ts | 14m 31s |
| x-pack/test/accessibility/apps/group3/config.ts | 11m 30s  |

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 05:35:43 -07:00
Marta Bondyra
085878c289
[Lens][Unified search] Auto expand comboboxes and popovers based on the content (#171573)
## Summary

Fixes partially two remaining tasks from
https://github.com/elastic/kibana/issues/168753
Fixes partially dataview issue from
https://github.com/elastic/kibana/issues/170398
It stretches to maximum approximate 60 characters if any of the labels
in the list is of this length. If the content doesn't need the container
to stretch, it doesn't do it.


<details>
 <summary> Field picker in Lens</summary>

minimum width:
<img width="445" alt="Screenshot 2023-11-21 at 15 56 03"
src="2f0f8482-bd00-4ec2-bbde-cbc4f3198eed">

auto-expanded width: 
<img width="575" alt="Screenshot 2023-11-21 at 15 58 22"
src="df7bab4d-0a08-4d49-8a91-9386eba15d93">
</details>


<details>
  <summary>Layer data view picker in Lens</summary>
<img width="376" alt="Screenshot 2023-11-21 at 16 01 17"
src="b8a98d83-dabc-49bd-a3cc-fc3856de6d3e">
<img width="455" alt="Screenshot 2023-11-21 at 15 58 09"
src="f2c5bde8-7a4a-485f-bf97-fc2179171e50">
<img width="615" alt="Screenshot 2023-11-21 at 15 56 27"
src="0574fc6c-69a3-44e9-9d48-8d427c1c5dba">
</details>

<details> <summary>Data view picker in Unified Search</summary>
<img width="341" alt="Screenshot 2023-11-21 at 16 00 29"
src="1c838ded-0dc5-4632-94e4-1d94586f667c">
<img width="441" alt="Screenshot 2023-11-21 at 15 58 04"
src="87e4f1c0-7922-4b94-a114-f23ece544395">
<img width="561" alt="Screenshot 2023-11-21 at 15 56 20"
src="3ea0f222-5241-4c5b-b00b-4311972754cc">
</details>

<details>
 <summary> Data view picker in dashboard Create control flyout</summary>
<img width="677" alt="Screenshot 2023-11-21 at 16 14 00"
src="0455b6ed-555d-4cff-9e34-0de377be6e04">
<img width="682" alt="Screenshot 2023-11-21 at 15 54 56"
src="2a67685c-379d-4c0b-bf56-dbf7c35b3bd4">
</details>

<details> 
<summary> Unified search data view select component (tested in
maps)</summary>
<img width="570" alt="Screenshot 2023-11-22 at 14 38 25"
src="bb52ab22-626d-4556-b40c-c9bcc925f426">
</details>

<details>
<summary>Unified search field and value picker</summary>
Adds `panelMinWidth`, removes the custom flex width change behavior
<img width="1142" alt="Screenshot 2023-11-22 at 14 40 26"
src="2450957f-38b7-4a73-b531-7acb29cb56bc">


f4f33624-9287-403e-8472-81f705440f97

</details>

<details> 
<summary> Discover breakdown field</summary>

Removes the focus stretching and instead uses the panelMinWidth prop

<img width="419" alt="Screenshot 2023-11-21 at 16 46 50"
src="e35125ad-8823-4bff-954b-8119a352829c">
<img width="619" alt="Screenshot 2023-11-21 at 16 48 20"
src="89e63daf-a59e-43e1-a6ec-91d1b15b0fcd">


</details>

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 13:13:43 +01:00
Mark Hopkin
a4aa7117bb
[Entity Analytics] Add risk engine missing privileges callout (#171250)
## Summary

_note: this is currently behind the experimental feature flag
`riskEnginePrivilegesRouteEnabled`._

Add a callout to the Entity Risk Score Management page if the user
doesn't have sufficient privileges. Here is the callout with a user with
none of the privileges (missing privileges are dynamically shown)

<img width="1177" alt="Screenshot 2023-11-21 at 12 52 21"
src="0c4a17ee-8856-45a5-8798-1cef0e7fe0ad">

as part of this I have added a route `GET
/internal/risk_score/engine/privileges` the response payload looks like
this:
```
{
    "privileges": {
        "kibana": {
            "feature_savedObjectsManagement.all": false
        },
        "elasticsearch": {
            "cluster": {
                "manage_transform": false,
                "manage_index_templates": false
            },
            "index": {
                "risk-score.risk-score-*": {
                    "read": false,
                    "write": false
                }
            }
        }
    },
    "has_all_required": false // does the user have all privileges? 
}
```

Docs issue for associated documentation changes
https://github.com/elastic/security-docs/issues/4307

### Testing    
- cypress tests added for the no banner case (user has all privs), and
the worst case (user has none of the privs)
- API Integration tests added for all of the granular cases 
- Manual test steps
    - 1.  User has correct privileges
        - Create a user with all risk engine privileges
        - navigate to the Entity Risk Score Management page
        - missing privileges banner should not show
    - 2.  User has missing privileges
        - Create a user with some or no risk engine privileges
        - navigate to the Entity Risk Score Management page
        - banner should show and describe all privileges missing

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 11:29:46 +00:00
Pete Hampton
04b6bff6d1
Raise detection rule alert telemetry from 1K/1hr to total or timeout. (#170334)
## Summary

Currently, telemetry instrumentation for prebuilt rule alerts in the
security solution is capped at 1K/1hr.
This PR lists the limit with a
[PiT](https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html)
query, but short circuits with a 15m timeout.

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 03:30:16 -07:00
Christos Nasikas
56887ac1f8
[Cases] Add new sub feature privilege to prevent access to the cases settings page (#170635) 2023-11-28 03:24:45 -07:00
Vadim Kibana
fcdd44ffeb
[Reporting] Do not stretch report delete button (#171862)
## Summary

Closes https://github.com/elastic/kibana/issues/171853

Now there is no extra spacing around the button:

<img width="1469" alt="image"
src="c7cfe3fc-ed30-49ee-a9ce-e1234facb187">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 10:21:37 +01:00
Vadim Kibana
df5383edf2
[Reporting] Prevent server from crashing due to concurrent deletes (#171858)
## Summary

Closes https://github.com/elastic/kibana/issues/171363

- The thing that crashes the server is `refresh: 'wait_for'`.
- I've also changed `await promisify` in one place, as that looked risky
to me.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 10:20:30 +01:00
Faisal Kanout
025fb3031b
[BUG][OBX-UX-MNGMT] Fix IS_NOT_BETWEEN comparator for the custom threshold, Infra, Metric rules (#171925)
## Summary
Fixes https://github.com/elastic/kibana/issues/169524

Fix the painless script that evaluates the IS_NOT_BETWEEN for the Custom
Threshold, Metric, Infra rules.

<img width="1024" alt="Screenshot 2023-11-24 at 15 03 18"
src="38fa171e-9057-4f42-aa12-ce5138eece24">

---------
2023-11-28 10:03:37 +01:00