This PR replaces the parsing and formatting of SecurityManager policies with the parsing and formatting of Entitlements policy during plugin installation.
Relates to ES-10923
This PR adds a check and a warning message if the user is trying to install a plugin containing the legacy plugin-security.policy used by SecurityManager.
Relates to ES-10846
The node environment has many paths. The accessors for these currently
use a "file" suffix, but they are always directories. This commit
renames the accessors to make it clear these paths are directories.
The libs projects are configured to all begin with `elasticsearch-`.
While this is desireable for the artifacts to contain this consistent
prefix, it means the project names don't match up with their
directories. Additionally, it creates complexities for subproject naming
that must be manually adjusted.
This commit adjusts the project names for those under libs to be their
directory names. The resulting artifacts for these libs are kept the
same, all beginning with `elasticsearch-`.
* Mechanical package change in IntelliJ
* A couple of manual fixups
* Export plugins.loading to deprecation
* Put plugin-cli in a module so can export PluginsUtils to it.
The difference is that our matcher uses .find() to search for a regex match anywhere in the string, whereas the hamcrest one uses .matches() to check the whole string against the regex. This leads to more specific regex checks.
I've left our own one for YAML tests, as that way we don't need to mangle the regex to add .* either side, which might be confusing in test failures.
We're leaking quite a few of these parsers. That doesn't seem to be much
of a problem but results in some memory inefficiencies in Jackson here
and there. This PR bulk fixes a bunch of instances that I could easily
automatically fix. I'll open a follow-up for closing the parser on the
document parsing context which also suffers from this but is non-trivial
to fix.
This commit upgrades the Bouncy Castle jars. Bouncy Castle is used for
some internal build concners as well as a comnand line application.
Most notably Bouncy Castle is also used as the FIPs certified JCE/JSEE provider
we use to test our ability to use a FIPs compliant crypto provider.
The following changes here are a result of the upgraded Bouncy Castle jars:
* TLSv1.3 is now supported when running in FIPs mode
* RSA PKCS#1 v1.5 is no longer allowed in FIPS mode
* Triple DES (3DES) is no longer allowed in FIPS mode
* Minor updates the security manager configuration used to test FIPs (to read permissions from the security provider)
* Minor adjustments to tests to accommodate the above changes.
* Minor adjustments to the gradle build to accommodate new dependencies
Note - update to the documentation will come in a later commit.
The build version is made up of a few parts in non-release builds. Both
the snapshot and pre-release qualifiers are appended to it. These
qualifiers used to be part of Version, but in 7.0 the qualifiers were
made to be found only in the build info. The Build class retains these
qualifiers through the compile ES version extracted from the server jar
at runtime.
Build.qualifiedVersion() is suppose to provide the fully qualified
version, including snapshot and pre-release qualifiers. Yet
Build.version() also includes this information; there is no distinction
since the qualifier was moved to be only in the build info.
This commit separates the pre-release qualifier from the version. It
maintains bwc in talking to older nodes, passing the fully qualified
version there, but in current nodes splits out the pre-release qualifier
into a new member of Build.
The Plugin CLI can generally treat strings opaquely. We had some logic
comparing earlier versions, but what we really care about with most of our
plugins is whether or not they were built with the current version of
Elasticsearch, not whether they were built before or after. (This question
will be trickier with stable plugins, but none of that code is in the CLI.)
The CLI classes can be cleaned up even more once Version is removed from
PluginDescriptor. Some of the tests can't use opaque strings for versions
until PluginDescriptor can handle them.
* Remove Version from Install and List plugin actions
* Remove Version from SyncPluginsAction
Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
In order for build info to be pluggable for serverless, the current
build needs to be lazily determined. This commit moves the CURRENT
constant to a static method.
relates #96861
This commit updates the plugin cli and scanner components to use ASM 9.5.
The update is required to successfully test with JDK 21. Tests in this component programatically run the java source compiler, which generates class files with major version 65, then tries to parse those generated class files. Without this change the tests fail with java.lang.IllegalArgumentException: Unsupported class file major version 65.
stable plugins not build with ES's gradle plugin will not have named_components.json file.
To allow these plugins to expose their named components, a scan can be performed upon install.
relates #88980
Use local-independent `Strings.format` method instead of `String.format(Locale.ROOT, ...)`.
Inline `ESTestCase.forbidden` calls with `Strings.format` for the consistency sake.
Add `Strings.format` alias in `common.Strings`
Removing the custom dependency checksum functionality in favor of Gradle build-in dependency verification support.
- Use sha256 in favor of sha1 as sha1 is not considered safe these days.
Closes https://github.com/elastic/elasticsearch/issues/69736
The new stable plugin api will have a slightly different descriptor file
format. This commit prepares for stable plugins by adding support for
reading those new files. The basic info for a plugin is the same like
name and version info. Other stuff like classname are not necessary. The
one additional property specific to the new plugins is "modular", which
indicates whether the jars of the plugin should be loaded as named
modules (this is akin to setting the module path when running java).
The ingest attachment processor is currently available as a plugin. This
commit moves the processor to the default distribution so it is always
available.
Bootstrap plugins were an internal mechanism added to allow a
filesystemprovider for cloud with the quota-aware-fs plugin. Since that
was removed, bootstrap plugins no longer serve a purpose. They were
never officially documented because they were for internal use only.
This commit removes the bootstrap plugins infrastructure.
When running in Docker, the elasticsearch-plugins.yml allows configuring
plugins that should be installed in the system. Upon Elasticsearch
starting up, plugins are installed/removed to match the configured
plugins. However, this happens late in startup, and it would be nice to
keep the main Elasticsearch process from ever writing outside the
configured data directories. Now that the server cli has been moved to
Java, this is possible.
This commit moves invocation of the plugins sync command into the server
cli. Note that the sync plugins action should probably be reworked as it
can be implement Command directly now. However, this commit tries to be
the minimal change possible to remove plugin cli knowledge from server.
The class PluginInfo represents the plugin-descriptor.properties file
that each plugin must have. This commit renames the class to more
closely match what it represents: the plugin descriptor.
Our plugin loading code uses the PluginBundle class to describe a plugin on
disk. This should be an implementation detail of the plugin loading code, and
not part of the API shared with the plugin CLI.
* Refactor to hide internal code from Plugin CLI
* Add javadoc
* Move static plugin loading methods to util class
We had a set of utility methods in PluginsService that were public and
static so that the plugin cli tool could use them. Here, we move those
methods to a utility class so that the plugin CLI doesn't need to import
PluginsService. For the same reason, we also move
"PluginsService.Bundle" to a top-level class called "PluginBundle."
Finally, we move tests to the appropriate test classes.
Each Command subclass can implement close() so that resources will be
cleaned up on exceptional exit like SIGINT. This is implemented through
a shutdown hook added in the superclass constructor. However, this hook
makes testing difficult because the hook cannot be added in normal
tests, so a flag must be overriden when testing Command classes.
This commit moves the shutdown hook handling into the CliToolLauncher
that creates the command. It also adds non-evil tests that check how the
hook runs, in place of the old evil tests that actually registered a
real shutdown hook.
relates #85758
* Fix test for Windows line endings
* Make more tests line-separator agnostic
* Skip some tests where we need to fix production code
Co-authored-by: Rory Hunter <roryhunter2@gmail.com>
The EnvironmentAwareCommand is the most common base class for CLI tools.
Sometimes it is desirable for one tool to call into another. With the
new CliToolLauncher, this becomes easier to do. However, that only gives
access to the base Command class, which means the Environment would be
reparsed if the implementation is an EnvironmentAwareCommand. This
commit makes the execute method of EnvironmentAwareCommand public so
that other tools can pass the already parsed env.
relates #85758
The sysprops and envVars members of Command provide cli implementations
with information about the jvm process that is running. This is
convenient for runtime, but difficult for tests to mock because they
must subclass the cli class.
This commit adds a ProcessInfo record, and plumbs it through the
main and execute methods. The new record includes system properties,
environment variables and the working directory. By having this be a
single new parameter, additional information can be added in the future
without again needing to modify the method signatures.
relates #85758
The cli lib has the SuppressForbidden annotation, but so does core,
which cli depends on. This commit removes the SuppressForbidden from
cli, in favor of the one from core.
relates #85758
