This updates the PolicyParser to allow static methods to have an ExternalEntitlement annotation. This
removes a limitation where constructors cannot properly support type-erasure with different types of
data structures for internal entitlement generation and external entitlement generation (for example
List<Object> from the parser and List<SomeData> from an internal builder). We continue to enforce
that only one constructor/method may be annotated with ExternalEntitlement per Entitlement class.
Instrumented methods in EntitlementChecker are loosely grouped by the
associated entitlement. This commit sorts the methods within groups to
allow more clear placement of additional instrumented methods.
* Refactor: remove duplicate canWrite methods.
This serves as a good example of how Path and File handling could be
specialized in the future, but as long as they are identical, the duplication
causes more harm than good.
* Refactor: just one neverEntitled.
The original motivation was to avoid allocating a lambda object on each call,
but since that's a highly optimized operation in the JVM, it's unlikely to make
a difference in practice, and this smacks of premature optimization.
We're pretty liberal about lambdas elsewhere, so let's not sweat it here until
we have some evidence that it matters.
* Remove dead code
This commit adds an EntitlementTest annotation that can be used on
classes containing test actions for entitlements. The annotation mirrors
the parameters of CheckAction. Only file check actions are currently
converted, the rest can be moved and annotated as followups. Note that
the check action name is simply the method name, no fancy name
manipulation is done.
This commit adds the data dirs, config dir and temp dir into
entitlement bootstrapping. It doesn't yet use them in entitlement
policies, but makes them available to use within initialization.
This adds basic flag entitlement validation when creating PolicyManager. If a module has the same flag
entitlement as part of it's policy multiple times we will throw an IllegalArgumentException. With this
validation we can safely assume FileEntitlement is the only one we currently have that allows multiple entitlements in a policy.
This commit adds FileEntitlement to entitlements. It does not add checks
for all file access methods yet, instead opting for example read and
write methods.
Each module contains a sorted array of paths with read and write
permissions. Binary search is used to quickly identify the closest path
to determine whether a target path can be read or written.
Some important things about FileEntitlement are the path can either be a
file or a directory. All directories grant recursive permission. The
mode is either read or read_write. All operations like create or delete
are considered write.
relates ES-10354
Co-authored-by: Jack Conradson <osjdconrad@gmail.com>
This change closes a hole where we assumed any check against an unnamed-module from any
classloader was for one of our apm agent. This was not the case and made it so scripts could in theory
have the same entitlements as apm agent. Instead we now check to see if a class is part of the apm
package in an unnamed module to ensure it's actually for the apm agent.
Relates to ES-10192
A new query parameter `?include_source_on_error` was added for create / index, update and bulk REST APIs to control
if to include the document source in the error response in case of parsing errors. The default value is `true`.
This commit adds an AbstractEntitlementsIT and moves the entitlement
cluster setup into a bespoke EntitlementTestRule. That allows most of
the common code to be deduplicated. This change also automatically
creates a temp dir which the test passes along into the test cluster.
The mrjar plugin adds support for sourcesets named in the form mainNN,
which adds the appropriate compiler and other settings for that version
of Java, and produces a multi-release jar. Having multi-release jars
only makes sense for versions of java newer than the minimum compile
version. This commit adds validation that the version is not too old.
Note that the check is slightly relaxed; it allows mainNN where NN is
equal to the min java version. This is due to the desire to keep
code using incubating modules separate because warnings must be
disabled.
When an entitlement is available to policy files, it should have one
constructor that the parser uses. This commit adjusts the policy parser
to scan the constructor to find that one annotated constructor, and
errors if more than one is found.
This commit combines the different entitlement test plugins used for
integration tests into a single plugin. When installing the plugin in
the integ test cluster, the entitlement policy and plugin properties are
adjusted to test the desired combination of settings. This allows the
plugin policy to be dynamically generated by each test, rather than rely
on a static policy.
Instrument write access to System properties by means of the `WriteSystemPropertiesEntitlement`.
`System.setProperties(Properties)` is always denied.
Part of #ES-10359
This adds a new exception called PolicyValdidationException. This exception is intended to be used during construction of entitlements that require parsing additional arguments. This allows PolicyParser to give improved error messaging about a specific entitlement that may have failed during the parsing process.
Support for `ST_EXTENT_AGG` was added in https://github.com/elastic/elasticsearch/pull/118829, and then partially optimized in https://github.com/elastic/elasticsearch/pull/118829. This optimization worked only for cartesian_shape fields, and worked by extracting the Extent from the doc-values and re-encoding it as a WKB `BBOX` geometry. This does not work for geo_shape, where we need to retain all 6 integers stored in the doc-values, in order to perform the datelline choice only at reduce time during the final phase of the aggregation.
Since both geo_shape and cartesian_shape perform the aggregations using integers, and the original Extent values in the doc-values are integers, this PR expands the previous optimization by:
* Saving all Extent values into a multi-valued field in an IntBlock for both cartesian_shape and geo_shape
* Simplifying the logic around merging intermediate states for all cases (geo/cartesian and grouped and non-grouped aggs)
* Widening test cases for testing more combinations of aggregations and types, and fixing a few bugs found
* Enhancing cartesian extent to convert from 6 ints to 4 ints at block loading time (for efficiency)
* Fixing bugs in both cartesian and geo extents for generating intermediate state with missing groups (flaky tests in serverless)
* Moved the int order to always match Rectangle for 4-int and Extent for 6-int cases (improved internal consistency)
Since the PR already changed the meaning of the invalid/infinite values of the intermediate state integers, it was already not compatible with the previous cluster versions. We disabled mixed-cluster testing to prevent errors as a result of that. This leaves us the opportunity to make further changes that are mixed-cluster incompatible, hence the decision to perform this consistency update now.
