# Backport
This will backport the following commits from `main` to `8.12`:
- [[Response Ops][Actions] Adding configuration to override default MS
Graph API Scope and Exchange URL values
(#175812)](https://github.com/elastic/kibana/pull/175812)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Ying
Mao","email":"ying.mao@elastic.co"},"sourceCommit":{"committedDate":"2024-02-01T17:41:52Z","message":"[Response
Ops][Actions] Adding configuration to override default MS Graph API
Scope and Exchange URL values (#175812)\n\nResolves
https://github.com/elastic/kibana/issues/166064\r\n\r\n##
Summary\r\n\r\nAdds the following configurations to the `kibana.yml`
config:\r\n* `xpack.actions.microsoftGraphApiScope` - overrides the
default Graph\r\nAPI scope value of
`https://graph.microsoft.com/.default`\r\n*
`xpack.actions.microsoftExchangeUrl` - overrides the default value
of\r\n`https://login.microsoftonline.com`\r\n\r\nThis allows users in
different Azure environments to customize their\r\nendpoints as
needed.\r\n\r\n## To Verify\r\n\r\nWe are unable to test this in a
different environment but we can verify\r\nthat the config overrides the
defaults as expected by setting the config\r\nvalues to something
different and the logging out the params that are\r\nsent to
`getOAuthClientCredentialsAccessToken`
in\r\n`x-pack/plugins/stack_connectors/server/connector_types/email/send_email.ts`.\r\nThen
create an MS Exchange email connector and test it to see that
the\r\nlogged values are overridden as
expected.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"f7e4f7a636763d46cb6a38b21a5eb6e67595ddfe","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Actions","Team:ResponseOps","backport:prev-minor","backport:prev-MAJOR","v8.13.0"],"title":"[Response
Ops][Actions] Adding configuration to override default MS Graph API
Scope and Exchange URL
values","number":175812,"url":"https://github.com/elastic/kibana/pull/175812","mergeCommit":{"message":"[Response
Ops][Actions] Adding configuration to override default MS Graph API
Scope and Exchange URL values (#175812)\n\nResolves
https://github.com/elastic/kibana/issues/166064\r\n\r\n##
Summary\r\n\r\nAdds the following configurations to the `kibana.yml`
config:\r\n* `xpack.actions.microsoftGraphApiScope` - overrides the
default Graph\r\nAPI scope value of
`https://graph.microsoft.com/.default`\r\n*
`xpack.actions.microsoftExchangeUrl` - overrides the default value
of\r\n`https://login.microsoftonline.com`\r\n\r\nThis allows users in
different Azure environments to customize their\r\nendpoints as
needed.\r\n\r\n## To Verify\r\n\r\nWe are unable to test this in a
different environment but we can verify\r\nthat the config overrides the
defaults as expected by setting the config\r\nvalues to something
different and the logging out the params that are\r\nsent to
`getOAuthClientCredentialsAccessToken`
in\r\n`x-pack/plugins/stack_connectors/server/connector_types/email/send_email.ts`.\r\nThen
create an MS Exchange email connector and test it to see that
the\r\nlogged values are overridden as
expected.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"f7e4f7a636763d46cb6a38b21a5eb6e67595ddfe"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.13.0","branchLabelMappingKey":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/175812","number":175812,"mergeCommit":{"message":"[Response
Ops][Actions] Adding configuration to override default MS Graph API
Scope and Exchange URL values (#175812)\n\nResolves
https://github.com/elastic/kibana/issues/166064\r\n\r\n##
Summary\r\n\r\nAdds the following configurations to the `kibana.yml`
config:\r\n* `xpack.actions.microsoftGraphApiScope` - overrides the
default Graph\r\nAPI scope value of
`https://graph.microsoft.com/.default`\r\n*
`xpack.actions.microsoftExchangeUrl` - overrides the default value
of\r\n`https://login.microsoftonline.com`\r\n\r\nThis allows users in
different Azure environments to customize their\r\nendpoints as
needed.\r\n\r\n## To Verify\r\n\r\nWe are unable to test this in a
different environment but we can verify\r\nthat the config overrides the
defaults as expected by setting the config\r\nvalues to something
different and the logging out the params that are\r\nsent to
`getOAuthClientCredentialsAccessToken`
in\r\n`x-pack/plugins/stack_connectors/server/connector_types/email/send_email.ts`.\r\nThen
create an MS Exchange email connector and test it to see that
the\r\nlogged values are overridden as
expected.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"f7e4f7a636763d46cb6a38b21a5eb6e67595ddfe"}}]}]
BACKPORT-->
Co-authored-by: Ying Mao <ying.mao@elastic.co>
# Backport
This will backport the following commits from `main` to `8.12`:
- [[RAM] Fix rule log aggregations when filtering >1k logs
(#172228)](https://github.com/elastic/kibana/pull/172228)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Zacqary Adam
Xeper","email":"Zacqary@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-01-09T23:17:13Z","message":"[RAM]
Fix rule log aggregations when filtering >1k logs (#172228)\n\n##
Summary\r\n\r\nFixes #171409 \r\n\r\nThis fixes weird behavior when
attempting to filter more than 1000 rule\r\nexecution logs.\r\n\r\nThe
execution log aggregator had two problems:\r\n\r\n- It had a max bucket
limit of 1000, while the KPI agg had a max bucket\r\nlimit of 10000,
leading to inconsistent results\r\n- For pagination, it reported the
total number of logs by using a\r\ncardinality aggregator, which wasn't
subject to any bucket limit at all\r\n\r\nThe endpoint responds with a
`total` (the cardinality) and an array of\r\n`data` (a paginated slice
of the aggregated logs). The way the data\r\ntable UI works is that it
takes the `total` value, caps it at 1000, and\r\nthen generates that
many blank rows to fill with whatever's in the\r\n`data`
array.\r\n\r\nSo as seen in the issue, we could easily run into a
situation where:\r\n\r\n1. User enters a filter query that last appeared
more than 1000 logs ago\r\n2. The cardinality agg accurately reports the
number of logs that should\r\nbe fetched, and generates enough blank
rows for these logs\r\n3. The actual execution log agg hits the 1000
bucket limit, and returns\r\nan empty `data` array\r\n\r\nThis PR fixes
this by using a bucket sum aggregation to determine the\r\ndata table's
`total` instead of a cardinality aggregation.\r\n\r\nIt also ups the
bucket limit from 1000 to 10000, to match the bucket\r\nlimit from the
summary KPI endpoint. This prevents a situation where:\r\n\r\n1. User
enters a filter query that last appeared in <1000 logs, but more\r\nthan
1000 logs ago\r\n2. Summary KPI widget, with a bucket limit of 10000,
accurately displays\r\na count of the <1000 logs this query
matches\r\n3. The data table is empty because the execution log bucket
limit tops\r\nout at 1000\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Xavier Mouligneau
<xavier.mouligneau@elastic.co>","sha":"9d32f889683cb6bc099d561d94e02af2feaf0d10","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:ResponseOps","Feature:Alerting/RulesManagement","v8.12.0","v8.13.0"],"title":"[RAM]
Fix rule log aggregations when filtering >1k
logs","number":172228,"url":"https://github.com/elastic/kibana/pull/172228","mergeCommit":{"message":"[RAM]
Fix rule log aggregations when filtering >1k logs (#172228)\n\n##
Summary\r\n\r\nFixes #171409 \r\n\r\nThis fixes weird behavior when
attempting to filter more than 1000 rule\r\nexecution logs.\r\n\r\nThe
execution log aggregator had two problems:\r\n\r\n- It had a max bucket
limit of 1000, while the KPI agg had a max bucket\r\nlimit of 10000,
leading to inconsistent results\r\n- For pagination, it reported the
total number of logs by using a\r\ncardinality aggregator, which wasn't
subject to any bucket limit at all\r\n\r\nThe endpoint responds with a
`total` (the cardinality) and an array of\r\n`data` (a paginated slice
of the aggregated logs). The way the data\r\ntable UI works is that it
takes the `total` value, caps it at 1000, and\r\nthen generates that
many blank rows to fill with whatever's in the\r\n`data`
array.\r\n\r\nSo as seen in the issue, we could easily run into a
situation where:\r\n\r\n1. User enters a filter query that last appeared
more than 1000 logs ago\r\n2. The cardinality agg accurately reports the
number of logs that should\r\nbe fetched, and generates enough blank
rows for these logs\r\n3. The actual execution log agg hits the 1000
bucket limit, and returns\r\nan empty `data` array\r\n\r\nThis PR fixes
this by using a bucket sum aggregation to determine the\r\ndata table's
`total` instead of a cardinality aggregation.\r\n\r\nIt also ups the
bucket limit from 1000 to 10000, to match the bucket\r\nlimit from the
summary KPI endpoint. This prevents a situation where:\r\n\r\n1. User
enters a filter query that last appeared in <1000 logs, but more\r\nthan
1000 logs ago\r\n2. Summary KPI widget, with a bucket limit of 10000,
accurately displays\r\na count of the <1000 logs this query
matches\r\n3. The data table is empty because the execution log bucket
limit tops\r\nout at 1000\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Xavier Mouligneau
<xavier.mouligneau@elastic.co>","sha":"9d32f889683cb6bc099d561d94e02af2feaf0d10"}},"sourceBranch":"main","suggestedTargetBranches":["8.12"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.13.0","branchLabelMappingKey":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/172228","number":172228,"mergeCommit":{"message":"[RAM]
Fix rule log aggregations when filtering >1k logs (#172228)\n\n##
Summary\r\n\r\nFixes #171409 \r\n\r\nThis fixes weird behavior when
attempting to filter more than 1000 rule\r\nexecution logs.\r\n\r\nThe
execution log aggregator had two problems:\r\n\r\n- It had a max bucket
limit of 1000, while the KPI agg had a max bucket\r\nlimit of 10000,
leading to inconsistent results\r\n- For pagination, it reported the
total number of logs by using a\r\ncardinality aggregator, which wasn't
subject to any bucket limit at all\r\n\r\nThe endpoint responds with a
`total` (the cardinality) and an array of\r\n`data` (a paginated slice
of the aggregated logs). The way the data\r\ntable UI works is that it
takes the `total` value, caps it at 1000, and\r\nthen generates that
many blank rows to fill with whatever's in the\r\n`data`
array.\r\n\r\nSo as seen in the issue, we could easily run into a
situation where:\r\n\r\n1. User enters a filter query that last appeared
more than 1000 logs ago\r\n2. The cardinality agg accurately reports the
number of logs that should\r\nbe fetched, and generates enough blank
rows for these logs\r\n3. The actual execution log agg hits the 1000
bucket limit, and returns\r\nan empty `data` array\r\n\r\nThis PR fixes
this by using a bucket sum aggregation to determine the\r\ndata table's
`total` instead of a cardinality aggregation.\r\n\r\nIt also ups the
bucket limit from 1000 to 10000, to match the bucket\r\nlimit from the
summary KPI endpoint. This prevents a situation where:\r\n\r\n1. User
enters a filter query that last appeared in <1000 logs, but more\r\nthan
1000 logs ago\r\n2. Summary KPI widget, with a bucket limit of 10000,
accurately displays\r\na count of the <1000 logs this query
matches\r\n3. The data table is empty because the execution log bucket
limit tops\r\nout at 1000\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Xavier Mouligneau
<xavier.mouligneau@elastic.co>","sha":"9d32f889683cb6bc099d561d94e02af2feaf0d10"}}]}]
BACKPORT-->
Co-authored-by: Zacqary Adam Xeper <Zacqary@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `8.12`:
- [Upgrade openai to 4.24.1
(#173934)](https://github.com/elastic/kibana/pull/173934)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Jon","email":"jon@elastic.co"},"sourceCommit":{"committedDate":"2024-01-02T20:57:17Z","message":"Upgrade
openai to 4.24.1
(#173934)\n\nhttps://github.com/openai/openai-node/releases/tag/v4.0.0\r\n\r\nFor
reviewers: I made a first pass, but I'm unsure on a few of
the\r\nchecks. If there's a ts-expect-error help would be appreciated.
Feel\r\nfree to push directly or whichever workflow is best for
you.\r\n\r\n---------\r\n\r\nCo-authored-by: Dario Gieselaar
<dario.gieselaar@elastic.co>","sha":"5f5f22224cb13ce18397e2b2603c18538ed78947","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-minor","v8.13.0"],"title":"Upgrade
openai to
4.24.1","number":173934,"url":"https://github.com/elastic/kibana/pull/173934","mergeCommit":{"message":"Upgrade
openai to 4.24.1
(#173934)\n\nhttps://github.com/openai/openai-node/releases/tag/v4.0.0\r\n\r\nFor
reviewers: I made a first pass, but I'm unsure on a few of
the\r\nchecks. If there's a ts-expect-error help would be appreciated.
Feel\r\nfree to push directly or whichever workflow is best for
you.\r\n\r\n---------\r\n\r\nCo-authored-by: Dario Gieselaar
<dario.gieselaar@elastic.co>","sha":"5f5f22224cb13ce18397e2b2603c18538ed78947"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.13.0","branchLabelMappingKey":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/173934","number":173934,"mergeCommit":{"message":"Upgrade
openai to 4.24.1
(#173934)\n\nhttps://github.com/openai/openai-node/releases/tag/v4.0.0\r\n\r\nFor
reviewers: I made a first pass, but I'm unsure on a few of
the\r\nchecks. If there's a ts-expect-error help would be appreciated.
Feel\r\nfree to push directly or whichever workflow is best for
you.\r\n\r\n---------\r\n\r\nCo-authored-by: Dario Gieselaar
<dario.gieselaar@elastic.co>","sha":"5f5f22224cb13ce18397e2b2603c18538ed78947"}}]}]
BACKPORT-->
Co-authored-by: Jon <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `8.12`:
- [[ResponseOps] Fix Actions authz for SentinelOne to ensure that the
user explicitly has `ALL` privilege
(#172528)](https://github.com/elastic/kibana/pull/172528)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Paul
Tavares","email":"56442535+paul-tavares@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-12-11T17:31:25Z","message":"[ResponseOps]
Fix Actions authz for SentinelOne to ensure that the user explicitly has
`ALL` privilege (#172528)\n\n## Summary\r\n\r\n- Fixes the Actions
plugin sub-actions execution for SentinelOne\r\nconnector to ensure that
a user must have `ALL` privilege to \"Action
and\r\nConnectors\"","sha":"88ea8498b417d0b0f22364b434d57764ad958030","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","v8.12.0","v8.13.0"],"number":172528,"url":"https://github.com/elastic/kibana/pull/172528","mergeCommit":{"message":"[ResponseOps]
Fix Actions authz for SentinelOne to ensure that the user explicitly has
`ALL` privilege (#172528)\n\n## Summary\r\n\r\n- Fixes the Actions
plugin sub-actions execution for SentinelOne\r\nconnector to ensure that
a user must have `ALL` privilege to \"Action
and\r\nConnectors\"","sha":"88ea8498b417d0b0f22364b434d57764ad958030"}},"sourceBranch":"main","suggestedTargetBranches":["8.12"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.13.0","labelRegex":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/172528","number":172528,"mergeCommit":{"message":"[ResponseOps]
Fix Actions authz for SentinelOne to ensure that the user explicitly has
`ALL` privilege (#172528)\n\n## Summary\r\n\r\n- Fixes the Actions
plugin sub-actions execution for SentinelOne\r\nconnector to ensure that
a user must have `ALL` privilege to \"Action
and\r\nConnectors\"","sha":"88ea8498b417d0b0f22364b434d57764ad958030"}}]}]
BACKPORT-->
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
## Summary
Add possibility to Isolate/Release SentinelOne host from Alert details
flyout.
Add support for displaying S1 Agent status in UI.
Add an experimental flag to S1 Connector.
Rename S1 connector actions from `Agent` to `Host`
Add a feature flag to security_solution to control enrollment of this
feature.
Update parallel script to support all FTR config options
Add `cypress-data-session` plugin to allow better caching of test data
(mostly for Dev experience)
Testing instruction:
1. Ensure you have
2. From root Kibana folder run
https://p.elstc.co/paste/URVrCEcR#aG1X9p3BMCRUDY+IzfIg5mGomcTGxwkYO6RGxSIAyWz
3. In Cypress run
```x-pack/plugins/security_solution/public/management/cypress/e2e/sentinelone/isolate.cy.ts```
4. 💚
<img width="2375" alt="Zrzut ekranu 2023-11-15 o 12 38 27"
src="c7ddc20e-9944-452c-b739-fa2d9fbf072b">
<img width="2374" alt="Zrzut ekranu 2023-11-15 o 12 38 32"
src="ab3ced14-0a5c-4f40-a92e-844feb849bb4">
<img width="2370" alt="Zrzut ekranu 2023-11-15 o 12 38 38"
src="96ccd237-56a6-449e-979d-f4fe8ffbe048">
<img width="2373" alt="Zrzut ekranu 2023-11-15 o 12 38 46"
src="924013aa-79ef-405b-ae73-139cf0644ebf">
<img width="2374" alt="Zrzut ekranu 2023-11-15 o 12 39 17"
src="e1ff5b05-8b80-40a9-84b1-dd21bf9e059c">
<img width="2374" alt="Zrzut ekranu 2023-11-15 o 12 39 58"
src="15fc5d36-970f-47cb-ae2f-f8a19628e6f4">
<img width="2374" alt="Zrzut ekranu 2023-11-15 o 12 40 03"
src="5860a0c9-a6e5-43b9-b37d-aa68e4e71f26">
<img width="2373" alt="Zrzut ekranu 2023-11-15 o 12 40 09"
src="5e2c5d41-c96a-4c32-8d51-a8408efea8e3">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- Adds an additional authz check to the execution of SentinelOne
sub-actions to ensure the user has the `all` privilege to "Actions and
Connectors"
Resolves https://github.com/elastic/kibana/issues/165673
## Summary
`escapeSlack` function in the mustasche_renderer breakes the hyperlinks
by wrapping the URL with backticks.
So the below markdown template does not work.
`<{{context.alertDetailsUrl}}|View alert details>`
This PR removes the code that adds backticks.
With this change action variables with text wrapped in asterisks
\*text\* will render as **text** or wrapped in underscores \_text\_ will
render as _text_ .
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To verify
- Create a slack connector
- Create a rule that uses a slack connector and use the above markdown
template
- Verify that hyperlink is working properly in slack
Part of: https://github.com/elastic/response-ops-team/issues/125
This PR intends to prepare the `GET
${BASE_ACTION_API_PATH}/connector/{id}` API for versioning as shown in
the above issue.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Seeing a bunch of errors like this in serverless logs:
```
Executing Rule default:.es-query:23dedce0-5230-11ee-b332-73fb351d06b8 has resulted in Error: Cannot read properties of undefined (reading 'meta') - TypeError: Cannot read properties of undefined (reading 'meta')\n at /usr/share/kibana/node_modules/@kbn/actions-plugin/server/authorization/get_authorization_mode_by_source.js:45:56\n at Array.reduce (<anonymous>)\n at getBulkAuthorizationModeBySource (/usr/share/kibana/node_modules/@kbn/actions-plugin/server/authorization/get_authorization_mode_by_source.js:43:47)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at ActionsClient.bulkEnqueueExecution (/usr/share/kibana/node_modules/@kbn/actions-plugin/server/actions_client/actions_client.js:588:24)\n at ExecutionHandler.run (/usr/share/kibana/node_modules/@kbn/alerting-plugin/server/task_runner/execution_handler.js:249:28)\n at /usr/share/kibana/node_modules/@kbn/alerting-plugin/server/task_runner/task_runner.js:413:37\n at TaskRunnerTimer.runWithTimer (/usr/share/kibana/node_modules/@kbn/alerting-plugin/server/task_runner/task_runner_timer.js:50:20)\n at TaskRunner.runRule (/usr/share/kibana/node_modules/@kbn/alerting-plugin/server/task_runner/task_runner.js:406:5)\n at TaskRunner.run (/usr/share/kibana/node_modules/@kbn/alerting-plugin/server/task_runner/task_runner.js:629:49)\n at TaskManagerRunner.run (/usr/share/kibana/node_modules/@kbn/task-manager-plugin/server/task_running/task_runner.js:315:170)
```
which seem to be caused by connectivity issues between Kibana and ES. In
this case, when a rule executes and tries to schedule an action, we use
the saved objects client `bulkGet` function to determine if the rule is
a legacy (pre 7.10) version. We were not checking for errors returned
from the `bulkGet` (which returns as a 200 with errors embedded in the
success response) and so were trying to access data from the SO that did
not exist. This PR checks for errors returned from the `bulkGet` and
logs the error accordingly. Also did some small optimizations to the
code
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Addresses https://github.com/elastic/kibana/issues/160411
## Summary
This PR adds functionality for filtering out advanced settings that are
not relevant for serverless.
For context, we need to build an Advanced settings page in serverless
which only contains a set of the existing settings. We will reuse the
section registry (https://github.com/elastic/kibana/pull/163502) from
the original Advanced settings plugin as well as its UI components which
will also be extracted into a separate package. The app will be
registered from inside the `serverless` plugin.
In order to only display the settings that are relevant for serverless,
we need to make some changes to the uiSettings service. The
implementation in this PR leverages the existing `readonly` uiSettings
param and adds the `setAllowlist()` method which is called by the
serverless plugin to set an allowlist of setting keys.
**Testing in serverless:**
1. Set `advanced_settings.enabled: true` to enable the Advanced settings
app in serverless:
5b216c6ea9/config/serverless.yml (L53)
2. Start Es with `yarn es serverless --ssl` and Kibana with `yarn
serverless-{mode} --ssl` in any serverless mode.
3. Navigate to `app/management/kibana/settings`
4. Verify that the app only displays the settings from
`packages/serverless/settings/common/index.ts` (these are the settings,
relevant for all projects in serverless) as well as the settings from
the corresponding project package
`packages/serverless/settings/{mode}_project/index.ts`.
5. Verify that the app is functioning correctly.
**Testing in self-managed:**
1. Start Es with `yarn es snapshot` and Kibana with `yarn start`.
2. Go to Stack Management > Advanced settings
3. Verify that all settings are displayed as usual.
4. Verify that the app is functioning correctly.
If your team is a code owner of any of the serverless project plugins,
please review the corresponding package
`packages/serverless/settings/{search/observanility/security}_project/index.ts`
where you've been added as an owner and test in the serverless solution
accordingly.
<!---
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
-->
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Resolves https://github.com/elastic/kibana/issues/162264
## Summary
Adds a limit on the maximum number of actions that can be queued with a
circuit breaker. The limit in serverless is set to 10,000, and 1,000,000
in the other environments.
- If a rule execution exceeds the limit, the circuit breaker kicks in
and stops triggering actions.
- Alerting rule's status updated to warning when circuit breaker is hit
Did not update the `enqueueExecution` bc it's going to be removed in
https://github.com/elastic/kibana/pull/165120.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To Verify
- Create a 2 rules that have actions
- Set `xpack.actions.queued.max` in kibana.yml to a low number like 2 or
3
- Use the run soon button to queue up actions and hit the circuit
breaker.
- The actions will not be scheduled and the rule status will be set to
warning
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Resolves: #163751
This PR adds a new API to allow the other plugins to define an **enabled
connector types list** in the actions registry.
The list is used during the action execution to decide if the action
type is executable (enabled).
This decision logic sits on the existing two other checks:
1- `isActionTypeEnabled` -> if the connector type is in the
`enabledActionTypes` list in the config
2- `isLicenseValidForActionType` -> if the connector type is allowed for
the active license
As the only user of this feature is just security-solutions for now, we
decided to allow the list to be set only once.
<img width="890" alt="Screenshot 2023-08-31 at 12 00 10"
src="896ab104-6f75-4eee-8f19-9f2eb04bb149">
<img width="592" alt="Screenshot 2023-08-31 at 12 13 38"
src="533a1cfd-947a-4506-b078-149780346088">
<img width="1513" alt="Screenshot 2023-08-31 at 12 00 01"
src="91169dfe-b7f8-4fef-b734-56857a75dbdc">
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Towards https://github.com/elastic/response-ops-team/issues/125
## Summary
Preparing the `GET ${BASE_ACTION_API_PATH}/connector_types` HTTP API for
versioning
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Part of: https://github.com/elastic/response-ops-team/issues/125
This PR intends to prepare the `GET ${BASE_ACTION_API_PATH}/connectors`
API for versioning as shown in the above issue.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
upgrade `axios` to 1.4
- adjust to header usage, and config optionality
- Axios' adapters are now resolved from a string key by axios, no need
to import/instantiate adapters
- most of the changed code stems from changes in Axios' types
- `response.config` is now optional
- there was a change in the type of AxiosHeaders <->
InternalAxiosHeaders
Closes: #162661Closes: #162414
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes#160812
Adds UI for CA and client-side SSL certificate auth to webhook
connector, and adds these properties to the webhook's HttpsAgent on
execution:
<img width="912" alt="Screenshot 2023-07-18 at 11 59 33 AM"
src="b1986c8d-3632-4663-80ff-3fb37f706d07">
### When Editing
<img width="868" alt="Screenshot 2023-07-14 at 3 34 57 PM"
src="e3e0f450-1196-45fd-88a6-1e15b6f2fa36">
Also creates a Field helper for EuiFilePicker
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Bumps node.js to 18.17.0 (replacement for PR #144012 which was later
reverted)
As a result, these categorical additions were needed:
- `node` evocations will need the `--openssl-legacy-provider` flag,
wherever it would use certain crypto functionalities
- tests required updating of the expected HTTPS Agent call arguments,
`noDelay` seems to be a default
- `window.[NAME]` fields cannot be written directly
- some stricter typechecks
This is using our in-house built node.js 18 versions through the URLs
the proxy-cache. (built with
https://github.com/elastic/kibana-custom-nodejs-builds/pull/4)
These urls are served from a bucket, where the RHEL7/Centos7 compatible
node distributables are. (see:
https://github.com/elastic/kibana-ci-proxy-cache/pull/7)
Further todos:
- [x] check docs wording and consistency
- [ ] update the dependency report
- [x] explain custom builds in documentation
- [x] node_sass prebuilts
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Thomas Watson <w@tson.dk>
## Summary
This PR removes the ability to get system actions from `GET` APIs.
Specifically:
- The Get action API throws a 404 error when requesting a system action
- The Bulk Get action API throws a 404 error when requesting a system
action
- The Get All API filters out system actions
- The Get List Types API filters out system actions
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Part of https://github.com/elastic/kibana/issues/159342.
In this PR, I'm preparing the non-alerting (rule types) response ops
task types for serverless by defining an explicit task state schema.
This schema is used to validate the task's state before saving but also
when reading. In the scenario an older Kibana node runs a task after a
newer Kibana node has stored additional task state, the unknown state
properties will be dropped. Additionally, this will prompt developers to
be aware that adding required fields to the task state is a breaking
change that must be handled with care. (see
https://github.com/elastic/kibana/issues/155764).
For more information on how to use `stateSchemaByVersion`, see
https://github.com/elastic/kibana/pull/159048 and
https://github.com/elastic/kibana/blob/main/x-pack/plugins/task_manager/README.md.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Resolves: #155766Resolves: #159302
With this PR we aim to skip a task that has invalid direct and indirect
params.
In order to do that,
1- We validate the task params before calling the subtask's run method
and skip if if the task params are invalid.
2- We skip execution of a subtask (rule, action etc) when the run method
of it returns a `SkipError`
Therefore, validations in the run methods needs to be moved to top of
the run method and executed before anything else to return skip if the
data is invalid.
We also added a config to enable/disable the skip feature, and define
the delay duration of task reschedule.
As this may become an infinitive loop, we are supposed to limit the
attempts.
Follow on issue to implement that: #159302
---------
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR:
- Adds the ability to create system action types
- Creates system connectors on Kibana `start` from the system action
types
- Prevents system action to be created/updated/deleted
- Return system actions from the get/getAll endpoints
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR introduces the `isSystemAction: boolean` to the `ActionResult`
type. It also, adds the same to the `ActionType` type. I added
validation to verify that system actions cannot be registered at the
moment. More PRs are going to follow to incrementally build the feature.
Related: https://github.com/elastic/kibana/issues/160367
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Updates the code in the manual proxy tester to work with an updated
version of it's pre-req npm packages.
This code is not used in production, or ci tests, just used manually for
ad hoc proxy testing.
Since this file is used manually, is currently not type-checked, and
infrequently used, no one noticed it was no longer working, till I did
this morning.
