* Introduce Email Address Allow Lists For Watcher (#116672)
* New setting plus mutual exclusiveness validation
* New domain list checking
* Email service tests
* Documentation updates
* PR Changes
Fix comment
* Backport missing Settings method for default value with validator
In this PR we introduce cluster settings to manage the global data stream retention.
We introduce two settings `data_streams.lifecycle.retention.max` & `data_streams.lifecycle.retention.default` that configure the respective retentions. The settings are loaded and monitored by the `DataStreamGlobalRetentionSettings`. The validation has also moved there.
We preserved the `DataStreamGlobalRetention` record to reduce the impact of this change. The purpose of this method is to be simply a wrapper record that groups the retention settings together.
Temporarily, the `DataStreamGlobalRetentionSettings` is using the DataStreamFactoryRetention which is marked as deprecated for migration purposes.
* Starting to document various inference settings
* Finish settings
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: Max Hniebergall <137079448+maxhniebergall@users.noreply.github.com>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Update docs/reference/settings/inference-settings.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
---------
Co-authored-by: Max Hniebergall <137079448+maxhniebergall@users.noreply.github.com>
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Remove `es-test-dir` book-scoped variable
* Remove `plugins-examples-dir` book-scoped variable
* Remove `:dependencies-dir:` and `:xes-repo-dir:` book-scoped variables
- In `index.asciidoc`, two variables (`:dependencies-dir:` and `:xes-repo-dir:`) were removed.
- In `sql/index.asciidoc`, the `:sql-tests:` path was updated to fuller path
- In `esql/index.asciidoc`, the `:esql-tests:` path was updated idem
* Replace `es-repo-dir` with `es-ref-dir`
* Move `:include-xpack: true` to few files that use it, remove from index.asciidoc
In #99115 we updated the ssl settings template to cover new features. It
inadvertently introduced a doc bug that show invalied ssl.enabled
settings for watcher HTTP and email. This PR removes them.
Since these docs were originally written there have been a couple
of changes:
1. We now support aarch64 as well as x86_64, so the SSE4.2 guidance
needed clarification.
2. ML is more deeply embedded into Elasticsearch functionality
across nodes that are not ML nodes. For example, ingest pipelines
now routinely use ML, and, in the near future, index mappings
will too in the form of semantic text. Although we cannot mandate
that xpack.ml.enabled is set uniformly across the cluster, as
that would be a breaking change, we should say ever more strongly
that ML must be enabled on all nodes if all ML functionality is to
work correctly. The primary reason for wanting to disable ML is
hardware incompatibility, and if ML is disabled for that reason
then it should not be used at all.
This commit adds an optional setting xpack.security.fips_mode.required_providers
to allow enforcing specific JCE/JSSE security providers while running in FIPS mode.
If running in FIPS mode with this value set and the required provider(s) is not found
an exception will be thrown during startup preventing the service from starting.
This commit adds support to reload the JWT shared secret.
Notably this commit also includes support for a rotatable secret which includes
support for a configurable grace period where the elder value (after rotation)
is still accessible. This allows a time bound leniency where both values are
valid at the same time to help mitigate tightly coupled systems rotations.
The rotatable secret currently only supports checking if it is set or it matches
an external secret. However, future updates will accept a function that
can be be used as input to a 3rd party system that can try the current secret
but automatically fall back to the prior secret if that fails during the grace period.
The implementation of rotatable secret uses a StampedLock with optimistic
reads to help ensure minimal performance impact for the reading and expiry
of the secret.
This makes the data stream lifecycle generally available. This will allow
data streams to take advantage of a native simplified and resilient
lifecycle implementation.
This commit corrects the following issues with JWT and OIDC `jwkset_path` documentation:
* only https is supported for the JWT realm (OIDC support both https and http)
* JWT realm does not use a file watcher to reload the file every 5 seconds
* simplify "path" to "file name" ..technically it is resolved path, but 99% of the time it will be just
a file name in the config directory and "path" is ambiguous
* remove special mention of using the absolute path in cloud. .. this is an unnecessary implementation
detail and the only setting (of many) that calls out the cloud config directly by absolute path
* ensure the 2 different JWT documentations are the same
* make mention of when the JWT file will be reloaded (it is not backed by the file watcher, only OIDC is)
* [DOCS] Describe how to use Elastic Agent to monitor Elasticsearch
* Temporarily fix doc build
* Add question about showing Elastic Agent metrics in the monitoring UI
* Apply changes from review
* Activate link to Kibana docs
* Fix broken link
* Update docs/reference/monitoring/indices.asciidoc
This commit changes the SAML realm to use placeholder metadata (UnresolvedEntity) when the real metadata cannot be loaded over HTTPS - unless metadata.http.fail_on_error is set to true.
All future use of the realm will fail until the metadata is available, but this change allows the node to bootstrap successfully.
This PR is another round of documentation update for the JWT realm with the goal to achieve better clarity, differentiating more between the two token types and encourage readers to choose between them carefully.
Relates: #92409
This PR adds a user action to the SLM health indicator which checks each SLM policy's invocations
since last success field and reports degraded health (YELLOW) in the event that any policy is at or
above the failure threshold (default is 5 failures in a row).
This PR adds a new setting to enable tcp keepalive probes for the
connections used by the oidc back-channel communication. It defaults to
true as tcp keepalive is generally useful for ES.
Relates: #87773
