Commit graph

372 commits

Author SHA1 Message Date
Lorenzo Dematté
591a08f506
[Entitlements] Validation checks on paths (#126852) (#127055)
With this PR we restrict the paths we allow access to, forbidding plugins to specify/request entitlements for reading or writing to specific protected directories.

I added this validation to EntitlementInitialization, as I wanted to fail fast and this is the earliest occurrence where we have all we need: PathLookup to resolve relative paths, policies (for plugins, server, agents) and the Paths for the specific directories we want to protect.

Relates to ES-10918
2025-04-19 00:44:49 +10:00
Keith Massey
64bcf0419f
Adding a cleanup method to EnterpriseGeoIpDownloaderIT (#125958) (#125994)
(cherry picked from commit 7a9edb5d95)

# Conflicts:
#	modules/ingest-geoip/src/internalClusterTest/java/org/elasticsearch/ingest/geoip/EnterpriseGeoIpDownloaderIT.java
2025-04-01 07:47:24 +11:00
Keith Massey
9a6b47b352
[8.18] Re-enabling EnterpriseGeoIpDownloaderIT with verbose logging (#125884) (#125965) 2025-03-31 14:27:22 -05:00
Mark Vieira
eeea3d7a83
Convert ingest-geoip file based update tests to new testing framework (#125632) (#125700) 2025-03-27 07:56:23 +11:00
Joe Gallo
7461459c49
Fix geoip databases index access after system feature migration (take 3) (#124604) (#124679) 2025-03-13 08:56:24 +11:00
Lorenzo Dematté
5c721b671c
Missing geoip policy (#123597) (#123693) 2025-02-28 18:05:58 +01:00
Joe Gallo
30e972f470
[8.18] Adjust the versions for :modules:ingest-geoip:qa:full-cluster-restart (#123646) 2025-02-27 17:11:27 -05:00
Joe Gallo
44ddc55b75
Register IngestGeoIpMetadata as a NamedXContent (#123079) (#123327) 2025-02-25 13:07:56 +11:00
Joe Gallo
e498ad1793
Fix geoip databases index access after system feature migration (again) (#122938) (#123161) 2025-02-22 03:56:15 +11:00
Lorenzo Dematté
6e8074ed6f
[Entitlements] Instrumentation of NIO Files and Channels (#122816 and #122591) (#122986) (#122993)
* [Entitlements] Add check functions for NIO Files (#122591)

* [Entitlements] Instrumentation of NIO file channels (#122816)

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-02-20 20:45:01 +11:00
Moritz Mack
b00a24fb3d
Add entitlement checks for java.io stream classes (#122406) (#122851)
(cherry picked from commit 8eb89cf7c1)

# Conflicts:
#	libs/entitlement/bridge/src/main/java/org/elasticsearch/entitlement/bridge/EntitlementChecker.java
#	libs/entitlement/src/main/java/org/elasticsearch/entitlement/runtime/api/ElasticsearchEntitlementChecker.java
2025-02-19 04:08:13 +11:00
Joe Gallo
ed3669470e
Handle 404s from the _cat/indices api in FullClusterRestartIT (#122537) (#122564) 2025-02-14 16:22:17 +11:00
Keith Massey
4c0a52e085
[8.18] Fix geoip databases index access after system feature migration (#121196) (#121955)
* Fix geoip databases index access after system feature migration (#121196)

* fixing the test for 8.x

---------

Co-authored-by: Joe Gallo <joe.gallo@elastic.co>
2025-02-07 10:44:31 +11:00
Ryan Ernst
3537349096
Rename environment dir accessors (#121803) (#121836)
* Rename environment dir accessors (#121803)

The node environment has many paths. The accessors for these currently use a "file" suffix, but they are always directories. This commit renames the accessors to make it clear these paths are directories.

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-02-06 10:28:25 +11:00
Joe Gallo
37139f3324
Optimize some per-document hot paths in the geoip processor (#120824) (#121226) 2025-01-30 06:50:04 +11:00
Joe Gallo
5e8194199a
Extend geoip FullClusterRestartIT (#121085) (#121123) 2025-01-29 14:14:43 +11:00
Joe Gallo
ff4f9d7dde
Re-enable geoip FullClusterRestartIT (#120966) (#120976) 2025-01-29 02:15:35 +11:00
Joe Gallo
0f46b562e6
Optimize IngestCtxMap construction (#120833) (#120926) 2025-01-28 04:32:07 +11:00
Simon Cooper
77a179706d
[8.x] Remove IndexNameExpressionResolver from base classes that don't use it (#120531) (#120599)
Backport #120531 to 8.x
2025-01-22 12:30:07 +00:00
Keith Massey
edfdd079ab
Fixing GetDatabaseConfigurationAction response serialization (#119233) (#119470)
Co-authored-by: Joe Gallo <joe.gallo@elastic.co>
2025-01-03 06:13:27 +11:00
Rene Groeschke
6b7cd0339e
Update Gradle wrapper to 8.12 (#118683) (#119363)
This updates the gradle wrapper to 8.12

We addressed deprecation warnings due to the update that includes:

- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions

(cherry picked from commit ba61f8c7f7)
2024-12-31 08:36:31 +01:00
Simon Cooper
41770b7c5e
Collapse transport versions for 8.16 (#117991) (#118053) 2024-12-05 10:19:16 +00:00
Rene Groeschke
617644f781
[8.x] [Gradle] Remove static use of BuildParams (#115122) (#117436)
* [Gradle] Remove static use of BuildParams (#115122)

Static fields dont do well in Gradle with configuration cache enabled.

- Use buildParams extension in build scripts
- Keep BuildParams.ci for now for easy serverless migration
-  Tweak testing doc

(cherry picked from commit 13c8aaeffa)

# Conflicts:
#	TESTING.asciidoc
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/RestTestBasePlugin.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/AbstractYamlRestCompatTestPlugin.java
#	build.gradle
#	modules/ingest-geoip/qa/full-cluster-restart/build.gradle
#	qa/mixed-cluster/build.gradle
#	x-pack/plugin/ent-search/qa/full-cluster-restart/build.gradle
#	x-pack/plugin/eql/qa/rest/build.gradle
#	x-pack/plugin/fleet/qa/rest/build.gradle
#	x-pack/plugin/kql/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/ml/qa/multi-cluster-tests-with-security/build.gradle
#	x-pack/plugin/security/qa/multi-cluster/build.gradle
#	x-pack/plugin/sql/qa/jdbc/build.gradle
#	x-pack/plugin/transform/qa/multi-cluster-tests-with-security/build.gradle

* fix merge issue

* Fix more merge conflicts

* Fix merge conflicts

* Update TESTING.asciidoc
2024-11-25 18:28:51 +01:00
Joe Gallo
bb33952367
Optimize PipelineConfiguration-checking ClusterStateListeners (#117038) (#117098) 2024-11-21 02:45:42 +11:00
Keith Massey
dc7b91115b
Adding testUsedMaxMindResponseClassesAreAccountedFor back to MaxMindSupportTests (#115706) (#115781) 2024-10-29 02:54:51 +11:00
Keith Massey
81a93a46e7
Fixing DatabaseNodeServiceIT testNonGzippedDatabase and testGzippedDatabase race condition (#115463) (#115707)
Co-authored-by: Joe Gallo <joegallo@gmail.com>
2024-10-26 10:22:46 +11:00
Keith Massey
6f4ffa8eb3
Adding additional checks for IPInfo results (#115481) (#115695) 2024-10-26 07:56:48 +11:00
Joe Gallo
4282fa12c9
Refactor the download_database_on_pipeline_creation checks (#115421) (#115461) 2024-10-24 08:04:37 +11:00
Joe Gallo
8f568b5649
Optimize downloader task executor (#115355) (#115365) 2024-10-23 08:36:13 +11:00
Joe Gallo
b744181885
Fix IPinfo geolocation schema (#115147) (#115162) 2024-10-19 12:30:16 +11:00
Joe Gallo
0b535c20ce
Close exchanges in HttpClientTests (#115059) (#115113) 2024-10-19 05:29:22 +11:00
Simon Cooper
a7883aea9b
Squash transport versions for 8.15 (#114827) (#114971)
Backport #114827 to 8.x
2024-10-17 11:48:44 +01:00
Keith Massey
59ed4058ec
Reducing error-level stack trace logging for normal events in GeoIpDownloader (#114924) (#114943) 2024-10-17 06:53:48 +11:00
Joe Gallo
c3e7eed0eb
Download IPinfo ip location databases (#114847) (#114872)
Backport of #114847
2024-10-16 13:34:15 +11:00
Joe Gallo
45881c4c9f
Support IPinfo databases in the ip_location processor (#114735) (#114776) 2024-10-15 22:20:01 +11:00
David Turner
98209e44de
Simplify XContent output of epoch times (#114491) (#114736)
Today the overloads of `XContentBuilder#timeField` do two rather
different things: one formats an object as a `String` representation of
a time (where the object is either an unambiguous time object or else a
`long`) and the other formats only a `long` as one or two fields
depending on the `?human` flag.

This is trappy in a number of ways:

- `long` means an absolute (epoch) time, but sometimes folks will
  mistakenly use this for time intervals too.

- `long` means only milliseconds, there is no facility to specify a
  different unit.

- the dependence on the `?human` flag in exactly one of the overloads is
  kinda weird.

This commit removes the confusion by dropping support for considering a
`Long` as a valid representation of a time at all, and instead requiring
callers to either convert it into a proper time object or else call a
method that is explicitly expecting an epoch time in milliseconds.
2024-10-15 03:46:31 +11:00
Joe Gallo
47bf5b9e2f
Refactor IPinfoIpDataLookupsTests tests (and others) (#114667) (#114672) 2024-10-12 11:40:33 +11:00
Joe Gallo
25ffc1526a
Support IPinfo database configurations (#114548) (#114663) 2024-10-12 09:15:57 +11:00
Keith Massey
f61c9d0afc
Move tests out of geo ip processor tests (#114656) (#114666) 2024-10-12 09:15:12 +11:00
Patrick Doyle
15343e4ec0
Backport ChunkedXContentBuilder to 8.16.0 (#114659)
* Create a fluent builder to help implement ChunkedToXContent (#112389)

Rather than manually adding startObject/endObject, and having to line everything up manually, this handles the start/end for you.
A few implementations are converted already. In the long run, I would like this to replace ChunkedXContentHelper.

* Convert a few more implementations to ChunkedXContentBuilder (#113125)

Remove the complex methods from ChunkedXContentHelper

* Further conversions to ChunkedXContentBuilder (#114237)

---------

Co-authored-by: Simon Cooper <simon.cooper@elastic.co>
2024-10-12 07:54:35 +11:00
Keith Massey
3adce49fdb
Adding support for registered country fields for maxmind geoip databases (#114521) (#114543)
Co-authored-by: Joe Gallo <joegallo@gmail.com>
2024-10-11 10:41:24 +11:00
Joe Gallo
3b8e33768a
Verify Maxmind database types in the geoip processor (#114527) (#114532) 2024-10-11 08:24:24 +11:00
Joe Gallo
57f083e1ca
IPinfo privacy detection support (#114456) (#114462) 2024-10-10 11:28:55 +11:00
Joe Gallo
80fb2ca371
IPinfo geolocation support (#114311) (#114373) 2024-10-09 10:48:17 +11:00
Keith Massey
90f8d5a451
Fixing IpinfoIpDataLookupsTests for Windows (#114340) (#114361) 2024-10-09 09:46:55 +11:00
Keith Massey
204950ad06
Supporting more maxmind fields in the geoip processor (#114268) (#114270) 2024-10-08 13:26:26 +11:00
Joe Gallo
586ea41217
Add postal_code support to the City and Enterprise databases (#114193) (#114263) 2024-10-08 08:23:45 +11:00
Joe Gallo
3ee06081ce
IPinfo ASN and Country (#114192) (#114256)
Adding the building blocks to support IPinfo ASN and Country data
2024-10-08 05:44:22 +11:00
Keith Massey
54165408e1
ingest-geoip: establish boundaries (#109655) (#114250)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
2024-10-08 03:48:21 +11:00
Keith Massey
0b224dcef4
[8.x] Listing all available databases in the _ingest/geoip/database API (#113498) (#114081)
* Listing all available databases in the _ingest/geoip/database API (#113498)

* yamlRestCompatTestTransform renamed to yamlRestTestV7CompatTransform
2024-10-04 09:40:39 +10:00