Commit graph

3375 commits

Author SHA1 Message Date
David Turner
279498d810
Remove spurious NOMERGE comment (#126231)
Left-over from #122062
2025-04-04 03:46:07 +11:00
Lorenzo Dematté
40dd91b800
[Entitlements] Replace Permissions with Entitlements in InstallPluginAction (#125207)
This PR replaces the parsing and formatting of SecurityManager policies with the parsing and formatting of Entitlements policy during plugin installation.

Relates to ES-10923
2025-04-02 11:03:27 +01:00
David Turner
2d4fb76267
Improve randomIdentifier usage in AWS tests (#125775)
Adds prefixes to various randomly-generated values to make it easier to
pin down where they're coming from in debugging sessions. Also forces
the STS expiry time to be rendered in UTC.
2025-03-28 18:33:05 +11:00
David Turner
36c14bf3a5
Validate region/service in DynamicAwsCredentials (#125671)
Following on from #125559, we can validate the region and service name
in tests that use `DynamicAwsCredentials` too.
2025-03-27 06:14:40 +00:00
Mark Vieira
930b4ab995
Convert remaining plugin projects to new test clusters framework (#125626) 2025-03-26 13:44:07 -07:00
David Turner
8d649f2f07
Validate AWS signer region and service in tests (#125559)
Extends the predicate in `AwsCredentialsUtils` to verify that we are
using a proper AWS v4 signature complete with the correct region and
service, rather than just looking for the access key as a substring.
2025-03-26 02:53:21 +11:00
Yang Wang
6c04abc68c
Fix compilation and tests for customAuthzEngine (#125469)
Relates: #123812
2025-03-24 12:03:06 +11:00
Armin Braun
50437e79d3
Cleanup missing use of StandardCharsets (#125424)
Random annoyance that I figured, I'd just fix globally:
We can do a bit of a cleaner job when doing byte <-> string conversion here and there.
2025-03-21 20:10:15 +01:00
Nikolaj Volgushev
c58ac456b8
Failure Store Access Authorization (#123986)
This PR implements authorization logic for failure store access. It
builds on https://github.com/elastic/elasticsearch/pull/122715.

Access to the failure store is granted by two privileges:
`read_failure_store` and `manage_failure_store`. Either of these
privileges lets a user access a failure store via the `::failures`
selector, as well as access its backing failure indices. 
`read_failure_store` grants read access (for example to search documents
in a failure store), `manage_failure_store` grants access to write
operations, such as rollover. Users with only `read` or `manage` on a
data stream do not get failure store access. Vice versa, users with
`read_failure_store` and `manage_failure_store` do not get access to
regular data in a data stream. 

The PR implements this by making authorization logic selector-aware. It
involves two main changes:

1. Index permission groups now compare the selector under which an index resource is accessed to the selector associated with the group.
2. The `AuthorizedIndices` interface likewise uses selectors to decide which indices to treat as authorized. This part of the change requires a sizable refactor and changes to the interface. 

The high-level behavior for selector-aware search is as follows:

For a user with `read_failure_store` over data stream `logs`:

- `POST /logs::failures/_search` returns the documents in the failure store.
- `POST /logs/_search` returns a 403.
- `POST /logs/_search?ignore_unavailable=true` and `POST /*/_search` return an empty result.

Similarly, for a user with `read` over data stream `logs`:

- `POST /logs::failures/_search` returns a 403.
- `POST /logs/_search` returns documents in the data stream.
- `POST /logs::failures/_search?ignore_unavailable=true` and `POST /*::failures/_search` return an empty result.

A user with both `read` and `read_failure_store` over data stream `logs`
gets access to both `POST /logs::failures/_search` and `POST
/logs/_search`.

The index privilege `all` automatically grants access to both data and
the failures store, as well as all hypothetical future selectors. 

Resolves: ES-10873
2025-03-20 20:10:16 +11:00
Rene Groeschke
6f7a206a8a
Update Gradle Enterprise Plugin (Develocity) to 3.19.2 (#125137)
with develocity updated to 2024.3.4 we are unblocked on updating the gradle enterprise plugin
2025-03-19 19:19:54 +01:00
David Turner
23769da8e3
Work around testFilterByTags failure (#125109)
A SDK bug[^1] means that these values sometimes don't parse as an
integer. We don't need them to do so in order to pass the test, so this
commit relaxes the test to accept any string.

Closes #125090 Closes #125166

[^1]: https://github.com/aws/aws-sdk-java-v2/issues/5968
2025-03-20 02:49:54 +11:00
Patrick Doyle
fd51f44e32
Silence known entitlement warnings (#124883) 2025-03-18 16:52:12 +02:00
David Turner
a2d98e44a1
Upgrade discovery-ec2 to AWS SDK v2 (#122062) 2025-03-18 19:38:16 +11:00
Rene Groeschke
ae569def9c
[Build] Require reason for usesDefaultDistribution (#124707)
This makes using usesDefaultDistribution in our test setup for explicit by requiring a reason why it's needed.
This is helpful as part of revisiting the need for all those usages in our code base.
2025-03-17 08:25:39 +01:00
Lorenzo Dematté
a753d4392a
Revert "missing file entitlement used by google-http-client for oauth2 (#123985)" (#124329)
This reverts commit 404c9e6290.
2025-03-08 23:17:53 +01:00
Armin Braun
b1c75d1868
Move some security APIs to using promises in place of callbacks (#123812)
We have some incredibly deep callstacks in security that seem to visibly
raise context switch costs, make profiling more complicated and
generally make the code rather hard to follow.
Since the methods adjusted here return a result synchronously we can
both save overhead and make things a little easier to follow by using
promises as returns in place of consuming callbacks.
2025-03-05 12:08:29 +01:00
Rene Groeschke
cfa98f30e5
Fix configuration cache compatibility issues (#124073)
- for running :plugins:discovery-ec2:check
- checking if in idea in build scan background action
2025-03-05 21:06:31 +11:00
Rene Groeschke
496c38e5a5
Reapply "Update Gradle wrapper to 8.13 (#122421)" (#123889) (#123896)
This reverts commit 36660f2e5f.
2025-03-05 08:02:13 +01:00
Mark Vieira
cc08d1d2a6
Add inbound_network entitlement to repository-hdfs plugin (#123907) 2025-03-04 09:24:54 -08:00
Lorenzo Dematté
404c9e6290
missing file entitlement used by google-http-client for oauth2 (#123985) 2025-03-04 16:06:44 +01:00
Tim Vernum
23e5c15590
Fix custom authz engine for multi-project (#123937) 2025-03-04 01:21:07 +00:00
Rene Groeschke
36660f2e5f
Revert "Update Gradle wrapper to 8.13 (#122421)" (#123889)
This reverts commit e19b2264af.
2025-03-03 15:51:07 +01:00
Rene Groeschke
e19b2264af
Update Gradle wrapper to 8.13 (#122421)
* Fix Gradle Deprecation warning as declaring an is- property with a Boolean type has been deprecated.
* Make use of new layout.settingsFolder api to address some cross project references
* Fix buildParams snapshot check for multiprojet projects
2025-03-03 14:10:00 +01:00
Lorenzo Dematté
678738acc0
[Entitlements] Add URLConnection instrumentation (#123503) 2025-03-03 08:40:53 +01:00
Lorenzo Dematté
850d48a944
Missing policies for netty, discovery-azure and repository-s3 (#123696) 2025-03-02 22:34:38 +01:00
Tim Vernum
4f918a81b9 Merge main into multi-project 2025-02-20 12:17:12 +11:00
Lorenzo Dematté
20b829fc14
[Entitlements] Add check functions for NIO Files (#122591) 2025-02-19 21:55:06 +01:00
Lorenzo Dematté
340a2ceb29
Fix hdfs-related IT tests for java24 (#122044) 2025-02-19 18:27:09 +01:00
Tim Vernum
81043b5ce0 Merge revision 2eb0d00b8f into multi-project 2025-02-19 16:00:15 +11:00
Alex Kattathra Johnson
5462601fd6
Upgrade AWS SDK to v1.12.746 (#122431) 2025-02-18 02:44:38 +11:00
Tim Vernum
680e7a6979 Merge revision 5c00341c2b into multi-project 2025-02-14 17:17:41 +11:00
Patrick Doyle
f8aa047994
Entitlements: manage_threads (#122261)
* Refactor: protected -> private

* Initial thread-related entitlements

* Entitlements from manual test runs

* Refactor: notEntitled method

* Entitlements reporting mode

* Entitlements from CI

* Revert "Entitlements reporting mode"

This reverts commit 443ca767333269a73accd0cedf9ca6c6ac51698a.

* Remove unnecessary EntitledActions.newThread

* Don't log in entitlements ITs by default

* Import SuppressForbidden

* Respond to PR comments

* Move manage_threads tests to their own file
2025-02-13 13:45:51 -05:00
Oleksandr Kolomiiets
b8d7e99cb9
Use FallbackSyntheticSourceBlockLoader for number fields (#122280) 2025-02-12 16:12:19 -08:00
Yang Wang
04d459009b Merge main into multi-project 2025-02-12 09:57:09 +11:00
Ryan Ernst
30a65fbec5
Fix example plugin using config dir (#122212)
This commit fixes a use of accessing the config dir from the
environment.
2025-02-11 11:06:22 -08:00
Yang Wang
38d74f7408 Merge main into multi-project 2025-02-06 10:04:49 +11:00
Ryan Ernst
0cf42f2388
Rename environment dir accessors (#121803)
The node environment has many paths. The accessors for these currently
use a "file" suffix, but they are always directories. This commit
renames the accessors to make it clear these paths are directories.
2025-02-05 14:03:28 -08:00
Yang Wang
fee57daf53 Merge main into multi-project 2025-02-05 11:03:29 +11:00
Ryan Ernst
f77201fb12
Fix compilation of custom auth engine example (#121089) 2025-02-04 19:34:41 +01:00
Niels Bauman
621a18d947 Merge main into multi-project 2025-01-30 17:26:28 +10:00
Rene Groeschke
2b16515166
Update Gradle wrapper to 8.12.1 (#120960) 2025-01-28 15:41:44 +01:00
Kostas Krikellas
8de9539e29
Lazy initialization for SyntheticSourceSupport.loader() (#120896)
* Lazy initialization for `SyntheticSourceSupport.loader()`

* [CI] Auto commit changes from spotless

* add missing

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-01-27 17:12:42 +02:00
Niels Bauman
682cf0a18f Merge remote-tracking branch 'public/main' into merge-main 2025-01-23 13:27:52 +10:00
Yang Wang
e720fa3ffa
[Test] Ensure sufficient secrets length for FIPS tests (#120581)
Resolves: #120576 Resolves: #120577 Resolves: #120578 Resolves: #120579
2025-01-22 23:16:55 +11:00
Simon Cooper
e6a3aff093 Merge commit '13c75c02f0' into merge-main-22-01-25T12 2025-01-22 12:09:34 +00:00
Moritz Mack
f1447fe821
[Entitlements] Instrument write access to System properties (#120357)
Instrument write access to System properties by means of the `WriteSystemPropertiesEntitlement`.
`System.setProperties(Properties)` is always denied.

Part of #ES-10359
2025-01-21 18:07:40 +01:00
David Turner
49f27e39ea
Migrate discovery-ec2 QA tests to javaRestTest (#119384)
No need for all this Gradle magic any more, we can just test the
discovery behaviour directly using Java REST tests.
2025-01-21 21:20:19 +11:00
Niels Bauman
6c7d2d285b Merge main into multi-project 2025-01-21 11:04:37 +10:00
Lorenzo Dematté
cd86b3b497
[Entitlements] Refactor Network Entitlement (#120391) 2025-01-20 18:29:32 +01:00
Yang Wang
7a1f5c5d4f Merge main into multi-project 2025-01-20 11:17:40 +11:00