## Summary
This PR adds details about whats new to the 8.10 release on the whats
new page.
---------
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
# Backport
This will backport the following commits from `main` to `8.10`:
- [[node] Enable openssl legacy provider
(#163190)](https://github.com/elastic/kibana/pull/163190)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Jon","email":"jon@elastic.co"},"sourceCommit":{"committedDate":"2023-08-30T18:51:10Z","message":"[node]
Enable openssl legacy provider (#163190)\n\nThis is to prevent a
breaking change in a minor release of Kibana due to\r\nan underlying
upgrade of Node.js to v18.\r\nThe legacy provider can be disabled by
removing\r\n`--openssl-legacy-provider` in
`config/node.options`.\r\n\r\n[Node.js\r\ndocumentation](https://nodejs.org/docs/latest-v18.x/api/cli.html#--openssl-legacy-provider)\r\n[OpenSSL\r\ndocumentation](https://www.openssl.org/docs/man3.0/man7/OSSL_PROVIDER-legacy.html)\r\n\r\n---------\r\n\r\nCo-authored-by:
Thomas Watson
<w@tson.dk>","sha":"aebd6f392384b4e36241f1a1ad5f3c615b42bcca","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","auto-backport","ci:build-all-platforms","ci:cloud-deploy","v8.10.0","v7.17.13","v8.11.0"],"number":163190,"url":"https://github.com/elastic/kibana/pull/163190","mergeCommit":{"message":"[node]
Enable openssl legacy provider (#163190)\n\nThis is to prevent a
breaking change in a minor release of Kibana due to\r\nan underlying
upgrade of Node.js to v18.\r\nThe legacy provider can be disabled by
removing\r\n`--openssl-legacy-provider` in
`config/node.options`.\r\n\r\n[Node.js\r\ndocumentation](https://nodejs.org/docs/latest-v18.x/api/cli.html#--openssl-legacy-provider)\r\n[OpenSSL\r\ndocumentation](https://www.openssl.org/docs/man3.0/man7/OSSL_PROVIDER-legacy.html)\r\n\r\n---------\r\n\r\nCo-authored-by:
Thomas Watson
<w@tson.dk>","sha":"aebd6f392384b4e36241f1a1ad5f3c615b42bcca"}},"sourceBranch":"main","suggestedTargetBranches":["8.10","7.17"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"7.17","label":"v7.17.13","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/163190","number":163190,"mergeCommit":{"message":"[node]
Enable openssl legacy provider (#163190)\n\nThis is to prevent a
breaking change in a minor release of Kibana due to\r\nan underlying
upgrade of Node.js to v18.\r\nThe legacy provider can be disabled by
removing\r\n`--openssl-legacy-provider` in
`config/node.options`.\r\n\r\n[Node.js\r\ndocumentation](https://nodejs.org/docs/latest-v18.x/api/cli.html#--openssl-legacy-provider)\r\n[OpenSSL\r\ndocumentation](https://www.openssl.org/docs/man3.0/man7/OSSL_PROVIDER-legacy.html)\r\n\r\n---------\r\n\r\nCo-authored-by:
Thomas Watson
<w@tson.dk>","sha":"aebd6f392384b4e36241f1a1ad5f3c615b42bcca"}}]}]
BACKPORT-->
---------
Co-authored-by: Jon <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `8.10`:
- [[DOCS] Add cross-link for other encryption key settings
(#165014)](https://github.com/elastic/kibana/pull/165014)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"James
Rodewig","email":"james.rodewig@elastic.co"},"sourceCommit":{"committedDate":"2023-08-28T19:02:22Z","message":"[DOCS]
Add cross-link for other encryption key settings (#165014)\n\nProblem:
The [Configure security in
Kibana](https://www.elastic.co/guide/en/kibana/current/using-kibana-with-security.html)
docs page only covers the `xpack.security.encryptionKey` setting for
session encryption. Users may not know that encryption for Kibana's
reporting and saved objects features also require encryption
keys.\r\n\r\nSolution: Add a cross-link to the respective encryption key
settings for reporting and saved
objects","sha":"8ac82e38d79c0031d76de472d1b1e0f2f7b9fef2","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Docs","Feature:Reporting","Feature:Saved
Objects","release_note:skip","docs","Feature:Security/Session
Management","v8.10.0","v8.11.0","v8.9.2"],"number":165014,"url":"https://github.com/elastic/kibana/pull/165014","mergeCommit":{"message":"[DOCS]
Add cross-link for other encryption key settings (#165014)\n\nProblem:
The [Configure security in
Kibana](https://www.elastic.co/guide/en/kibana/current/using-kibana-with-security.html)
docs page only covers the `xpack.security.encryptionKey` setting for
session encryption. Users may not know that encryption for Kibana's
reporting and saved objects features also require encryption
keys.\r\n\r\nSolution: Add a cross-link to the respective encryption key
settings for reporting and saved
objects","sha":"8ac82e38d79c0031d76de472d1b1e0f2f7b9fef2"}},"sourceBranch":"main","suggestedTargetBranches":["8.10","8.9"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/165014","number":165014,"mergeCommit":{"message":"[DOCS]
Add cross-link for other encryption key settings (#165014)\n\nProblem:
The [Configure security in
Kibana](https://www.elastic.co/guide/en/kibana/current/using-kibana-with-security.html)
docs page only covers the `xpack.security.encryptionKey` setting for
session encryption. Users may not know that encryption for Kibana's
reporting and saved objects features also require encryption
keys.\r\n\r\nSolution: Add a cross-link to the respective encryption key
settings for reporting and saved
objects","sha":"8ac82e38d79c0031d76de472d1b1e0f2f7b9fef2"}},{"branch":"8.9","label":"v8.9.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
# Backport
This will backport the following commits from `main` to `8.10`:
- [[DOCS] Adds documentation for data comparison view
(#164297)](https://github.com/elastic/kibana/pull/164297)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"István Zoltán
Szabó","email":"szabosteve@gmail.com"},"sourceCommit":{"committedDate":"2023-08-24T14:13:38Z","message":"[DOCS]
Adds documentation for data comparison view (#164297)\n\n##
Summary\r\n\r\nRelated PR:
https://github.com/elastic/kibana/pull/161365\r\nRelated issue:
https://github.com/elastic/platform-docs-team/issues/153\r\n\r\nThis PR
drafts documentation for the new data comparison feature under\r\nthe
Data Visualizer in
Kibana.","sha":"e91103811be0c731af056779ba20a30a89e34253","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Docs",":ml","release_note:skip","docs","v8.10.0","v8.11.0"],"number":164297,"url":"https://github.com/elastic/kibana/pull/164297","mergeCommit":{"message":"[DOCS]
Adds documentation for data comparison view (#164297)\n\n##
Summary\r\n\r\nRelated PR:
https://github.com/elastic/kibana/pull/161365\r\nRelated issue:
https://github.com/elastic/platform-docs-team/issues/153\r\n\r\nThis PR
drafts documentation for the new data comparison feature under\r\nthe
Data Visualizer in
Kibana.","sha":"e91103811be0c731af056779ba20a30a89e34253"}},"sourceBranch":"main","suggestedTargetBranches":["8.10"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/164297","number":164297,"mergeCommit":{"message":"[DOCS]
Adds documentation for data comparison view (#164297)\n\n##
Summary\r\n\r\nRelated PR:
https://github.com/elastic/kibana/pull/161365\r\nRelated issue:
https://github.com/elastic/platform-docs-team/issues/153\r\n\r\nThis PR
drafts documentation for the new data comparison feature under\r\nthe
Data Visualizer in
Kibana.","sha":"e91103811be0c731af056779ba20a30a89e34253"}}]}]
BACKPORT-->
Co-authored-by: István Zoltán Szabó <szabosteve@gmail.com>
# Backport
This will backport the following commits from `main` to `8.10`:
- [[DOCS] Change deep link to ES book for CCS privileges
(#164165)](https://github.com/elastic/kibana/pull/164165)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Abdon
Pijpelink","email":"abdon.pijpelink@elastic.co"},"sourceCommit":{"committedDate":"2023-08-22T07:33:53Z","message":"[DOCS]
Change deep link to ES book for CCS privileges (#164165)\n\n##
Summary\r\n\r\nRelates to
https://github.com/elastic/elasticsearch/pull/98330\r\n\r\n8.10
introduces a new method to configure remote clusters (API key\r\nbased),
which will live next to the existing method (certificate based).\r\nAs a
result, the ES remote cluster docs will be restructured. The
deep\r\nlink from \"[Alerting
set\r\nup](https://www.elastic.co/guide/en/kibana/current/alerting-setup.html#alerting-ccs-setup)\"\r\nto
the setting up CCS privileges section in the ES docs will break
(and\r\nwould potentially bring folks to the wrong page
anyway).\r\n\r\nThis PR changes the link into a link to the higher-level
remote cluster\r\ndocs, from where folks can click through to the
section that is relevant\r\nto
them.","sha":"4b1d833aee778e0e7278edc04260394b0c024de5","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Docs","release_note:skip","docs","v8.10.0","v8.11.0"],"number":164165,"url":"https://github.com/elastic/kibana/pull/164165","mergeCommit":{"message":"[DOCS]
Change deep link to ES book for CCS privileges (#164165)\n\n##
Summary\r\n\r\nRelates to
https://github.com/elastic/elasticsearch/pull/98330\r\n\r\n8.10
introduces a new method to configure remote clusters (API key\r\nbased),
which will live next to the existing method (certificate based).\r\nAs a
result, the ES remote cluster docs will be restructured. The
deep\r\nlink from \"[Alerting
set\r\nup](https://www.elastic.co/guide/en/kibana/current/alerting-setup.html#alerting-ccs-setup)\"\r\nto
the setting up CCS privileges section in the ES docs will break
(and\r\nwould potentially bring folks to the wrong page
anyway).\r\n\r\nThis PR changes the link into a link to the higher-level
remote cluster\r\ndocs, from where folks can click through to the
section that is relevant\r\nto
them.","sha":"4b1d833aee778e0e7278edc04260394b0c024de5"}},"sourceBranch":"main","suggestedTargetBranches":["8.10"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/164165","number":164165,"mergeCommit":{"message":"[DOCS]
Change deep link to ES book for CCS privileges (#164165)\n\n##
Summary\r\n\r\nRelates to
https://github.com/elastic/elasticsearch/pull/98330\r\n\r\n8.10
introduces a new method to configure remote clusters (API key\r\nbased),
which will live next to the existing method (certificate based).\r\nAs a
result, the ES remote cluster docs will be restructured. The
deep\r\nlink from \"[Alerting
set\r\nup](https://www.elastic.co/guide/en/kibana/current/alerting-setup.html#alerting-ccs-setup)\"\r\nto
the setting up CCS privileges section in the ES docs will break
(and\r\nwould potentially bring folks to the wrong page
anyway).\r\n\r\nThis PR changes the link into a link to the higher-level
remote cluster\r\ndocs, from where folks can click through to the
section that is relevant\r\nto
them.","sha":"4b1d833aee778e0e7278edc04260394b0c024de5"}}]}] BACKPORT-->
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
# Backport
This will backport the following commits from `main` to `8.10`:
- [[DOCS] Improves change point detection documentation
(#164277)](https://github.com/elastic/kibana/pull/164277)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"István Zoltán
Szabó","email":"szabosteve@gmail.com"},"sourceCommit":{"committedDate":"2023-08-21T14:33:07Z","message":"[DOCS]
Improves change point detection documentation
(#164277)\n\nCo-authored-by: Dima Arnautov
<arnautov.dima@gmail.com>","sha":"79a0a5e94089121cbce294e0ca899084c63bae0b","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":[":ml","release_note:skip","docs","v8.10.0","v8.11.0"],"number":164277,"url":"https://github.com/elastic/kibana/pull/164277","mergeCommit":{"message":"[DOCS]
Improves change point detection documentation
(#164277)\n\nCo-authored-by: Dima Arnautov
<arnautov.dima@gmail.com>","sha":"79a0a5e94089121cbce294e0ca899084c63bae0b"}},"sourceBranch":"main","suggestedTargetBranches":["8.10"],"targetPullRequestStates":[{"branch":"8.10","label":"v8.10.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/164277","number":164277,"mergeCommit":{"message":"[DOCS]
Improves change point detection documentation
(#164277)\n\nCo-authored-by: Dima Arnautov
<arnautov.dima@gmail.com>","sha":"79a0a5e94089121cbce294e0ca899084c63bae0b"}}]}]
BACKPORT-->
Co-authored-by: István Zoltán Szabó <szabosteve@gmail.com>
resolves https://github.com/elastic/kibana/issues/162886
The default continues to be 90 days for data detetention of event log
documents, and the rollover is now controlled by DLM, as described in
[Data stream lifecycle][].
[Data stream lifecycle]:
https://www.elastic.co/guide/en/elasticsearch/reference/8.9/data-stream-lifecycle.html
## Release note
Fixes the event log data stream to use Data stream lifecycle instead of
Index Lifecycle Management. If you had previously customized the Kibana
event log ILM policy, you should now update the lifecycle of the event
log data stream itself.
## Summary
Bumps node.js to 18.17.0 (replacement for PR #144012 which was later
reverted)
As a result, these categorical additions were needed:
- `node` evocations will need the `--openssl-legacy-provider` flag,
wherever it would use certain crypto functionalities
- tests required updating of the expected HTTPS Agent call arguments,
`noDelay` seems to be a default
- `window.[NAME]` fields cannot be written directly
- some stricter typechecks
This is using our in-house built node.js 18 versions through the URLs
the proxy-cache. (built with
https://github.com/elastic/kibana-custom-nodejs-builds/pull/4)
These urls are served from a bucket, where the RHEL7/Centos7 compatible
node distributables are. (see:
https://github.com/elastic/kibana-ci-proxy-cache/pull/7)
Further todos:
- [x] check docs wording and consistency
- [ ] update the dependency report
- [x] explain custom builds in documentation
- [x] node_sass prebuilts
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Thomas Watson <w@tson.dk>
Closes https://github.com/elastic/kibana/issues/162215
## Summary
This PR changes the default session idle timeout for users to 3 days.
## Changes Made
- Updated default `session.idleTimeout` to `3d`.
- Updated tests to expect the new default timeout
- Updated asciidocs to match the above change
## Release notes
Change the default value of `session.idleTimeout` from 8 hours to 3
days.
👋🏼 @gchaps asked me to file a new PR since my last
https://github.com/elastic/kibana/pull/122573 got too far behind.
## Summary
🙏🏼 per #116017, adds insight on how to temporarily disable Kibana Rules
for clusters which need breathing room.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
This PR updates the security audit logs with some cases values. We added
a new operation for retrieving the `categories` of a case and the users
associated with a case.
## Summary
This PR adds documentation for:
- Drag and drop from Available Fields list
- Wildcards in field searches
- All sources and matching sources in Create Data View
Fixes https://github.com/elastic/kibana/issues/151938
In this PR, I'm re-writing the Task Manager poller so it doesn't run
concurrently when timeouts occur while also fixing the issue where
polling requests would pile up when polling takes time. To support this,
I've also made the following changes:
- Removed the observable monitor and the
`xpack.task_manager.max_poll_inactivity_cycles` setting
- Make the task store `search` and `updateByQuery` functions have no
retries. This prevents the request from retrying 5x whenever a timeout
occurs, causing each call taking up to 2 1/2 minutes before Kibana sees
the error (now down to 30s each). We have polling to manage retries in
these situations.
- Switch the task poller tests to use `sinon` for faking timers
- Removing the `assertStillInSetup` checks on plugin setup. Felt like a
maintenance burden that wasn't necessary to fix with my code changes.
The main code changes are within these files (to review thoroughly so
the polling cycle doesn't suddenly stop):
- x-pack/plugins/task_manager/server/polling/task_poller.ts
- x-pack/plugins/task_manager/server/polling_lifecycle.ts (easier to
review if you disregard whitespace `?w=1`)
## To verify
1. Tasks run normally (create a rule or something that goes through task
manager regularly).
2. When the update by query takes a while, the request is cancelled
after 30s or the time manually configured.
4. When the search for claimed tasks query takes a while, the request is
cancelled after 30s or the time manually configured.
**Tips:**
<details><summary>how to slowdown search for claimed task
queries</summary>
```
diff --git a/x-pack/plugins/task_manager/server/queries/task_claiming.ts b/x-pack/plugins/task_manager/server/queries/task_claiming.ts
index 07042650a37..2caefd63672 100644
--- a/x-pack/plugins/task_manager/server/queries/task_claiming.ts
+++ b/x-pack/plugins/task_manager/server/queries/task_claiming.ts
@@ -247,7 +247,7 @@ export class TaskClaiming {
taskTypes,
});
- const docs = tasksUpdated > 0 ? await this.sweepForClaimedTasks(taskTypes, size) : [];
+ const docs = await this.sweepForClaimedTasks(taskTypes, size);
this.emitEvents(docs.map((doc) => asTaskClaimEvent(doc.id, asOk(doc))));
@@ -346,6 +346,13 @@ export class TaskClaiming {
size,
sort: SortByRunAtAndRetryAt,
seq_no_primary_term: true,
+ aggs: {
+ delay: {
+ shard_delay: {
+ value: '40s',
+ },
+ },
+ },
});
return docs;
```
</details>
<details><summary>how to slow down update by queries</summary>
Not the cleanest way but you'll see occasional request timeouts from the
updateByQuery calls. I had more luck creating rules running every 1s.
```
diff --git a/x-pack/plugins/task_manager/server/task_store.ts b/x-pack/plugins/task_manager/server/task_store.ts
index a06ee7b918a..07aa81e5388 100644
--- a/x-pack/plugins/task_manager/server/task_store.ts
+++ b/x-pack/plugins/task_manager/server/task_store.ts
@@ -126,6 +126,7 @@ export class TaskStore {
// Timeouts are retried and make requests timeout after (requestTimeout * (1 + maxRetries))
// The poller doesn't need retry logic because it will try again at the next polling cycle
maxRetries: 0,
+ requestTimeout: 900,
});
}
@@ -458,6 +459,7 @@ export class TaskStore {
ignore_unavailable: true,
refresh: true,
conflicts: 'proceed',
+ requests_per_second: 1,
body: {
...opts,
max_docs,
```
</details>
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
resolves https://github.com/elastic/kibana/issues/142874
The alerting framework now generates an alert UUID for every alert it
creates. The UUID will be reused for alerts which continue to be active
on subsequent runs, until the alert recovers. When the same alert (alert
instance id) becomes active again, a new UUID will be generated. These
UUIDs then identify a "span" of events for a single alert.
The rule registry plugin was already adding these UUIDs to it's own
alerts-as-data indices, and that code has now been changed to make use
of the new UUID the alerting framework generates.
- adds property in the rule task state
`alertInstances[alertInstanceId].meta.uuid`; this is where the alert
UUID is persisted across runs
- adds a new `Alert` method getUuid(): string` that can be used by rule
executors to obtain the UUID of the alert they just retrieved from the
factory; the rule registry uses this to get the UUID generated by the
alerting framework
- for the event log, adds the property `kibana.alert.uuid` to
`*-instance` event log events; this is the same field the rule registry
writes into the alerts-as-data indices
- various changes to tests to accommodate new UUID data / methods
- migrates the UUID previous stored with lifecycle alerts in the alert
state, via the rule registry *INTO* the new `meta.uuid` field in the
existing alert state.
