Add a roles specification to autoscaling policies. This is used to map
the policy to a set of nodes governed by the policy. The list of roles
is mandatory when adding a policy, optional on updates.
This commit also removes the outer level "policy" element from autoscaling
policy PUT and GET requests.
Running the Elasticsearch Docker image with a different GID is
possible but trappy, since at present all the ES files are only
readable by the user and group. This PR documents a Docker CLI flag
that fixes this situation, by ensuring the container user is added
to the default group (which is `root`, GID 0).
I also added a test for this case, and refactored the Docker tests
to use a builder pattern for constructing the `docker run` command.
The existing code was becoming unwieldy and hard to change.
- Replaces more abstract docs about object structure and values source with task-based examples.
- Relocates several sections from the current `misc.asciidoc` file.
- Alphabetically sorts agg categories in the nav.
- Removes the matrix agg family. Moves the stats matrix agg under the metric agg family
Co-authored-by: debadair <debadair@elastic.co>
The docs for the geoip processor database_file option appear to indicate
that all geoip databases are in the config directory. This is leftover
legacy from when this was the case when ingest-geoip was a plugin, but
it is no longer true as the built-in databases now ship inside the
ingest-geoip module that is bundled by default. This commit clarifies
those docs.
Co-authored-by: Jakob Reiter <jakommo@users.noreply.github.com>
After #63811 it became clear to me that `postCollect` is kind of
dangerous and not all that useful. So this removes it.
The trouble with `postCollect` is that it all happened right after we
finished calling `collect` on the `LeafBucketCollectors` but before we
built the aggregation results. But in #63811 we found out that we can't
call `postCollect` on the children of `parent` or `child` aggregators
until we know which *which* aggregation results we're building.
So this removes `postCollect` and moves all of the things we did at
post-collect phase into `buildAggregations` or into hooks called in
those methods.
This commit clarifies that the preferred method for setting the heap
size is via jvm.options.d and that using the ES_JAVA_OPTS environment
variable is discouraged for production deployments.
This commit adjusts the defaults for the tiered data roles so that they
are enabled by default, or if the node has the legacy data role. This
ensures that the default experience is that the tiered data roles are
enabled.
To fully specifiy the behavior for the tiered data roles then:
- starting a new node with the defaults: enabled
- starting a new node with node.roles configured: enabled if and only
if the tiered data roles are explicitly configured, independently
of the node having the data role
- starting a new node with node.data enabled: enabled unless the
tiered data roles are explicitly disabled
- starting a new node with node.data disabled: disabled unless the
tiered data roles are explicitly enabled
Closes#20640.
This PR introduces a new parameter to v2 templates, `allow_auto_create`,
which allows templates to override the cluster setting `auto_create_index`.
Notes:
* `AutoCreateIndex` now looks for a matching v2 template, and if its
`allow_auto_create` setting is true, it overrides the usual logic.
* `TransportBulkAction` previously used `AutoCreateIndex` to check
whether missing indices should be created. We now rely on
`AutoCreateAction`, which was already differentiating between creating
indices and creating data streams. I've updated `AutoCreateAction` to
use `AutoCreateIndex`. Data streams are also influenced by
`allow_auto_create`, in that their default auto-create behaviour can
be disabled with this setting.
* Most of the Java file changes are due to introducing an extra
constructor parameter to `ComposableIndexTemplate`.
* I've added the new setting to various x-pack templates
* I added a YAML test to check that watches can be created even when
`auto_create_index` is `false`.
When running ML, sometimes it is best to automatically adjust the
memory allotted for machine learning based on the nodesize
and how much space is given to the JVM
This commit adds a new static setting xpack.ml.use_auto_machine_memory_percent for
allowing this dynamic calculation. The old setting remains as a backup
just in case the limit cannot be automatically determined due to
lack of information.
Closes#63795
* Allow mixing set-based and regexp-based include and exclude
* Coding style
* Disallow having both set and regexp include (resp. exclude)
* Test correctness of every combination of include/exclude
Renamed decision API to capacity. Responses now prefer objects/maps over
arrays. Removed mention of tier, using policies as the outer map and
total for the policy-wide total capacity.
This adds a new flag `exclude_generated` for GET transform API.
This flag is useful for when a transform needs to be cloned within a cluster or exported/imported between clusters.
It removes certain fields that are not able to be set via the PUT api (e.g. version, create_time).
relates https://github.com/elastic/elasticsearch/issues/63055
