Today's network config docs are split into "Network", "HTTP" and
"Transport" pages, with unclear relationships between them. We often
encounter users with weird configs that indicate they don't really
understand how these settings all relate. In fact these pages are all
very interrelated, and the HTTP and Transport pages are almost all only
for advanced users. This commit brings these docs into a single page and
rewords some things to try and guide users away from the advanced
settings unless their configuration needs all the extra complexity.
It also adds a section entitled "Binding and publishing" which clarifies
the meanings of the `bind_host` and `publish_host` parameters. This is
also a common source of confusion amongst users.
It also clarifies that many of these settings accept a list of
addresses, and warns that this may not be what you want. Closes#67956.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
The default heap size now resides in MachineDependentHeap and is applied
per our auto heap settings logic there vs living in jvm.options. For
this reason we no longer need to fill this in at build time, so we can
ditch these template variables.
The `gateway.recover_after_nodes` setting is one of the lucky few that
gets a mention in the default `elasticsearch.yml` config file. This
setting was deprecated in favour of `gateway.recover_after_data_nodes`
in #53646, but neither of these settings is really important enough to
warrant a place in the default config so this commit removes it.
We have an in-house rule to compare explicitly against `false` instead
of using the logical not operator (`!`). However, this hasn't
historically been enforced, meaning that there are many violations in
the source at present.
We now have a Checkstyle rule that can detect these cases, but before we
can turn it on, we need to fix the existing violations. This is being
done over a series of PRs, since there are a lot to fix.
We handled the exceptions thrown by Security Providers in the case
of short encryption keys in #65464 and this commit adds a couple
of tests to validate that the appropriate exceptions are thrown
when encryption keys derived from short passwords are in use, in
FIPS 140-2 mode.
* Add message to elasticsearch-plugin CLI output to notify users to restart
Elasticsearch after plugin installation
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR is a first attempt to get the build to run on an Apple M1 (ARM 64 / aarch64) machine.
I think the changes are mostly reasonable, apart from some hard-coding to use the Azul JVM,
which at the time of writing seems to be the only available JVM. I'll follow up when our preferred
JVM is available.
A new version of this test dependency is finally available, enabling us
to remove a hack from production code we've long carried because of a
bug in that test dependency. This commit upgrades our tests to use
jimfs-1.2.
As of #64024 we run FIPS CI on a true, FIPS approved only mode JVM.
This mandates that any passwords that are fed into PBKDF2 must have at
least 112 bits of entropy (that is, be 14 characters long).
This commit updates our Keystore CLI tests so that tests either:
1. Use a 14+ character password when in FIPS mode, _or_
2. Are skipped on FIPS mode (because they explicitly test empty
passwords)
Resolves: #66845
We were depending on the BouncyCastle FIPS own mechanics to set
itself in approved only mode since we run with the Security
Manager enabled. The check during startup seems to happen before we
set our restrictive SecurityManager though in
org.elasticsearch.bootstrap.Elasticsearch , and this means that
BCFIPS would not be in approved only mode, unless explicitly
configured so.
This commit sets the appropriate JVM property to explicitly set
BCFIPS in approved only mode in CI and adds tests to ensure that we
will be running with BCFIPS in approved only mode when we expect to.
It also sets xpack.security.fips_mode.enabled to true for all test clusters
used in fips mode and sets the distribution to the default one. It adds a
password to the elasticsearch keystore for all test clusters that run in fips
mode.
Moreover, it changes a few unit tests where we would use bcrypt even in
FIPS 140 mode. These would still pass since we are bundling our own
bcrypt implementation, but are now changed to use FIPS 140 approved
algorithms instead for better coverage.
It also addresses a number of tests that would fail in approved only mode
Mainly:
Tests that use PBKDF2 with a password less than 112 bits (14char). We
elected to change the passwords used everywhere to be at least 14
characters long instead of mandating
the use of pbkdf2_stretch because both pbkdf2 and
pbkdf2_stretch are supported and allowed in fips mode and it makes sense
to test with both. We could possibly figure out the password algorithm used
for each test and adjust password length accordingly only for pbkdf2 but
there is little value in that. It's good practice to use strong passwords so if
our docs and tests use longer passwords, then it's for the best. The approach
is brittle as there is no guarantee that the next test that will be added won't
use a short password, so we add some testing documentation too.
This leaves us with a possible coverage gap since we do support passwords
as short as 6 characters but we only test with > 14 chars but the
validation itself was not tested even before. Tests can be added in a followup,
outside of fips related context.
Tests that use a PKCS12 keystore and were not already muted.
Tests that depend on running test clusters with a basic license or
using the OSS distribution as FIPS 140 support is not available in
neither of these.
Finally, it adds some information around FIPS 140 testing in our testing
documentation reference so that developers can hopefully keep in
mind fips 140 related intricacies when writing/changing docs.
Where CCS is being used it makes sense for ML nodes to have the
remote_cluster_client role. This single extra role is not
significant enough to stop an ML node being considered a
"dedicated ML node".
This commit expands our JVM egonomics to also automatically determine
appropriate heap size based on the total available system memory as well
as the roles assigned to the node. Role determination is done via a
naive parsing of elasticsearch.yml. No settings validation is done and
only the 'node.roles' setting is taken into consideration.
For heap purposes a node falls into one of four (4) categories:
1. A 'master-only' node. This is a node with only the 'master' role.
2. A 'ml-only' node. Similarly, a node with only the 'ml' role.
3. A 'data' node. This is basically the 'other' case. A node with any
set of roles other than only master or only ml is considered a 'data'
node, to include things like coordinating-only or "tie-breaker" nodes.
4. Unknown. This is the case if legacy settings are used. In this
scenario we fallback to the old default heap options of 1GB.
In all cases we short-circuit if a user provides explicit heap options
so we only ever auto-determine heap if no existing heap options exist.
Starting with this commit the default heap settings (1GB) are now
removed from the default jvm.options which means we'll start auto-
setting heap as the new default.
This commit change the warn message returned by elasticsearch-plugin list. The elasticsearch version of plugin is needed for comparison with current es version rather than the plugin version.
Closes#62758.
Include the Stack log4j config in the Docker image, in order to
make it possible to write logs in a container environment in the
same way as for an archive or package deployment. This is useful
in situations where the user is bind-mounting the logs directory
and has their own arrangements for log shipping.
To use stack logging, set the environment variable `ES_LOG_STYLE`
to `file`. It can also be set to `console`, which is the same as
not specifying it at all.
The Docker logging config is now auto-generated at image build time,
by running the default config through a transformer program when
preparing the distribution in an image builder step.
Also, in the docker distribution `build.gradle`, I changed a helper
closure into a class with a static method in order to fix an
issue where the Docker image was always being rebuilt, even when
there were no changes.
When #65409 was merged, it didn't include the new PluginInfo field licensed in the
toString() or toXContent() outputs. This PR adds them and updates the tests accordingly.
Since #51882 we recommend not editing the `jvm.options` file, preferring
instead to override its contents with additional files in
`jvm.options.d`. However the inline comments in this file do not point
users in that direction. This commit adjusts these inline comments.
In certain situations, such as when configured in FIPS 140 mode,
the Java security provider in use might throw a subclass of
java.lang.Error. We currently do not catch these and as a result
the JVM exits, shutting down elasticsearch.
This commit attempts to address this by catching subclasses of Error
that might be thrown for instance when a PBKDF2 implementation
is used from a Security Provider in FIPS 140 mode, with the password
input being less than 14 bytes (112 bits).
- In our PBKDF2 family of hashers, we catch the Error and
throw an ElasticsearchException while creating or verifying the
hash. We throw on verification instead of simply returning false
on purpose so that the message bubbles up and the cause becomes
obvious (otherwise it would be indistinguishable from a wrong
password).
- In KeyStoreWrapper, we catch the Error in order to wrap and re-throw
a GeneralSecurityException with a helpful message. This can happen when
using any of the keystore CLI commands, when the node starts or when we
attempt to reload secure settings.
- In the `elasticsearch-users` tool, we catch the ElasticsearchException that
the Hasher class re-throws and throw an appropriate UserException.
Tests are missing because it's not trivial to set CI in fips approved mode
right now, and thus any tests would need to be muted. There is a parallel
effort in #64024 to enable that and tests will be added in a followup.
In PR #64274, we make it possible to run our Docker images more easily
under a different group, but in so doing increased the Docker layer
sizes dramatically, effectively doubling the size of the images.
Fix this by reworking what commands get run and where, in order to bring
down the final size of the images.
Most of the Elasticsearch Docker build process implements retries
in order to make the process more robust in the face of transient
errors e.g. network failures. However, we missed out the `apk`
command in `master`'s Dockerfile. This PR adds the same retry loop
around `apk` as features elsewhere in the `Dockerfile`.
As part of this, I implemented a helper closure that generates the
same loop construct throughout the `Dockerfile`.
Security manager policies within plugins currently can ask to grant any
permission (though we block some within the security manager itself at
runtime). Yet most of these permissions should never be necessary, and
some we would actively not want any plugins to be allowed to use. This
commit adds validation of plugins' policy files to restrict the
permissions allowed to be granted to a subset that is reasonable for
plugins to need. The allowed permissions are not ideal (still containing
things like suppressAccessChecks), but it is a step forward in defining
a stricter model for plugins that reduces the surface area of potential
abuse.
This PR introduces the concept of "licensed" plugins. Such plugins
may only be installed on installations of the default distribution,
and this is enforced by the plugin installer. This PR also moves
the `quote-aware-fs` plugin to the `x-pack` directory, and marks
it as licensed.
Note that I didn't move the plugin source under `x-pack/plugin`
because all the existing x-pack plugins are actually bundles as
modules into the default distribution, whereas the `quota-aware-fs`
plugin needs to remain a standalone plugin.
This PR adds support for building a Docker context for Iron Bank.
It doesn't actually build the image - we could add that at a later
stage, but this is an attempt to automate at least some of the
process.
Iron Bank is a lot like our UBI build, except it uses a hardened
version of the full UBI image, not the minimal UBI image. They have
particular requirements around how the Docker context should be
arranged. The Docker build cannot fetch its own artefacts, but
instead the context provides a descriptor that locates what is
needed for the build.
I also added a filter so that after performing expansions on the
`Dockerfile`, we squash long runs on newlines together. This makes
the output cleaner, while allowing us to break up the unprocessed
`Dockerfile` for clarity.
The test framework security policy contains permissions for both gradle
and intellij running tests. These currently coexist in the same file,
though only one set of the jars permissions are granted to exist in any
given run. This works because java policy parsing is lenient, so if a
system property referenced in the file does not exist, the entire grant
is silently skipped. This commit splits these permissions into separate
policy files so that we do not rely on leniency, and can (in a followup)
add our own validation to fix java's leniency.
This commit converts build code that downloads distributions or other
artifacts to use the new no-kpi subdomain, and removes the formerly used
no-kpi header.
* Add tests for using ES_JAVA_OPTS with windows service
* Relocate ES_JAVA_OPTS delimiter munging
* Don't use equals for -Xmx and -Xms args
* Write newlines in temporary configs
Running the Elasticsearch Docker image with a different GID is
possible but trappy, since at present all the ES files are only
readable by the user and group. This PR documents a Docker CLI flag
that fixes this situation, by ensuring the container user is added
to the default group (which is `root`, GID 0).
I also added a test for this case, and refactored the Docker tests
to use a builder pattern for constructing the `docker run` command.
The existing code was becoming unwieldy and hard to change.
Plugin policy parsing is currently split, with different code executed
for Elasticsearch startup vs installing a plugin. This commit
refactors the policy parsing to be utilized by both places. The main
benefit is policy files in both places now handle permissions not only
for a global grant, but also codebase specific grants.
Closes#61145.
This PR adds a quota-aware filesystem plugin to Elasticsearch. This plugin
offers a way to provide user quota limits (specifically, total quota size
and available quota size) to Elasticsearch, in an implementation-agnostic
manner.
As part of this work, this PR also introduces the concept of "bootstrap
only" plugins, which are excluded from the normal plugin loading process.
Finally, note that this implementation supports `createLink(...)`, since ES
/ Lucene use hard links where possible.
This commit adjusts the defaults for the tiered data roles so that they
are enabled by default, or if the node has the legacy data role. This
ensures that the default experience is that the tiered data roles are
enabled.
To fully specifiy the behavior for the tiered data roles then:
- starting a new node with the defaults: enabled
- starting a new node with node.roles configured: enabled if and only
if the tiered data roles are explicitly configured, independently
of the node having the data role
- starting a new node with node.data enabled: enabled unless the
tiered data roles are explicitly disabled
- starting a new node with node.data disabled: disabled unless the
tiered data roles are explicitly enabled
Closes#63869. Perform `docker pull` explicitly instead of as part of
`docker build`, and wrap it in a retry loop. This is an attempt to make
the build more resilient to transient errors.
* Removing some more duplication and redundant logic.
* Aligning all timeouts to 30s (60s or even 10 minute timeouts should be unnecessary, if they aren't we should figure out why)
* Remove some usage of `actionGet()` in tests (it's just evil to suppress the stack-trace)
As a precommit task dependency, check that the security audit's layout pattern
from the log4j configuration files for the docker and archive builds are identical.
Following #52519, our Docker build pulls down curl sources in an
Alpine Linux container using wget. However that version of wget
doesn't support any retry flags. Since network issues can cause build
failures, wrap the wget calls in the same retry construct used for
yum commands elsewhere.
Closes#63600.
Closes#51670, closes#50838.
Introduce a tiny base image for Docker builds. It aims to create a basic filesystem with as little as possible, which is mostly glibc, busybox and bash. A statically-built curl is also provided.
We still use CentOS 8 as a base. All the fun stuff happens in the Dockerfile.
* Fix concurrent modification on task realization
* Use taskprovider instead of relying on tasks in distribution setup
* Port more task references in :distribution to task provider
* Fix nullpointer in distribution setup
