Commit graph

85864 commits

Author SHA1 Message Date
Tim Grein
f054dca0b3 Add working dense text embeddings integration with default endpoint. Some tests WIP 2025-06-23 13:19:47 +02:00
Rene Groeschke
6e4cb8142b
Add initial esql test benchmark tests (#126224)
* Add initial esql test benchmark tests
* Fix build-benchmark pipeline
* Update gredle profiler
2025-04-08 13:28:34 +02:00
Richard Dennehy
ceaa01a538
Add Issuer to failed SAML Signature validation logs when available (#126310)
* Add Issuer to failed SAML Signature validation logs when available

* [CI] Auto commit changes from spotless

* Fix tests

* Update docs/changelog/126310.yaml

* address review comments

* replace String.format call

* update formatIssuer to describeIssuer

* [CI] Auto commit changes from spotless

* truncate long issuers in log messages

* [CI] Auto commit changes from spotless

* handle null issuer value

* address review comments

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-04-08 10:50:45 +01:00
Luca Cavanna
a6ffeeeb71
Remove outdated TODO from TopDocsAndMaxScore (#126386)
There are no plans to remove max_score, as highlighted in #32981 .
This commit removes a related TODO given we don't plan on addressing it.
2025-04-08 10:41:38 +02:00
Luigi Dell'Aquila
262a793329
Re-enable generative tests (#126421) 2025-04-08 10:21:36 +02:00
Ievgen Degtiarenko
c2d0c59b7a
Pre-size collections (#126382) 2025-04-08 10:01:21 +02:00
Slobodan Adamović
284121ad9f
Set keyUsage for generated HTTP certificates and self-signed CA (#126376)
The `elasticsearch-certutil http` command, and security auto-configuration, 
generate the HTTP certificate and CA without setting the `keyUsage` extension.

This PR fixes this by setting (by default):
- `keyCertSign` and `cRLSign` for self-signed CAs 
- `digitalSignature` and `keyEncipherment` for HTTP certificates and CSRs

These defaults can be overridden when running `elasticsearch-certutil http` 
command. The user will be prompted to change them as they wish.

For `elasticsearch-certutil ca`, the default value can be overridden by passing 
the `--keysage` option, e.g.
```
elasticsearch-certutil ca --keyusage "digitalSignature,keyCertSign,cRLSign" -pem    
```

Fixes #117769
2025-04-08 09:44:09 +02:00
Ignacio Vera
47e352fda0
Throw nicer exception in SpanBooleanQueryRewriteWithMaxClause (#126387)
Throw an ElasticsearchStatusException with a RestStatus.BAD_REQUEST code instead of a generic RuntimeException.
2025-04-08 06:42:00 +02:00
Yang Wang
997a7b8fab
FileWatchingService should not throw for missing file (#126264)
Missing file is a valid state for FileWatchingService so that the
exception should be suppressed.
2025-04-08 09:56:35 +10:00
Ryan Ernst
991e80d56e
Remove unnecessary generic params from action classes (#126364)
Transport actions have associated request and response classes. However,
the base type restrictions are not necessary to duplicate when creating
a map of transport actions. Relatedly, the ActionHandler class doesn't
actually need strongly typed action type and classes since they are lost
when shoved into the node client map. This commit removes these type
restrictions and generic parameters.
2025-04-07 16:22:56 -07:00
elasticsearchmachine
9feac7833e Mute org.elasticsearch.smoketest.MlWithSecurityIT test {yaml=ml/trained_model_cat_apis/Test cat trained models} #125750 2025-04-08 07:45:47 +10:00
Nhat Nguyen
6c641c0677
Extrapolate rate aggregation (#126331)
This change performs extrapolation for rate aggregation similarly
to how PromQL does.
2025-04-07 14:31:57 -07:00
Kaarina Tungseth
e3e03bc28b
Removes known issues page content (#126429)
* Removes known issues page

* Adds empty known issues page
2025-04-07 15:58:52 -05:00
David Turner
cedcb5ccfe
Replace TransportResponse.Empty with ActionResponse.Empty (#126400)
No need to distinguish these things any more, we can just use
`ActionResponse.Empty` everywhere.
2025-04-08 06:58:06 +10:00
Joe Gallo
bead858ccd
Correctly handle nulls in nested paths in the remove processor (#126417) 2025-04-07 16:54:07 -04:00
Jeremy Dahlgren
79297438ed
Avoid extra allocations in RestGetAliasesAction (#126177)
When no explicit aliases are provided in the call there is no need
to collect the index names or aliases into HashSets if they won't be
used. Also fixed where the index name was being added for each
loop of the alias list.
2025-04-07 15:02:05 -04:00
Brian Seeders
5888e903e4
[docs] Re-generate and fix 9.0.0 release notes in markdown (#126425) 2025-04-07 14:17:06 -04:00
David Turner
f6c1965101
Forward port changes from backport of #125562 (#126413)
The backport to `8.x` needed some changes to pass through CI; this
commit forward-ports the relevant bits of those changes back into `main`
to keep the branches aligned.
2025-04-07 19:05:06 +01:00
David Turner
fbbbdd7eec
Allow overriding blob container path in tests (#126391)
Some `AbstractBlobContainerRetriesTestCase#createBlobContainer`
implementations choose a path for the container randomly, but we have a
need for a test which re-creates the same container against a different
`S3Service` and `BlobStore` and must therefore specify the same path
each time. This commit exposes a parameter that lets callers specify a
container path.
2025-04-08 03:54:37 +10:00
David Turner
5dc7ab77b3
Remove usages of TransportMessage (#126375)
This base class is kinda pointless: everywhere it's used we can either
be more specific (e.g. choosing between `TransportRequest` or
`TransportResponse`) or more general (e.g. choosing `Writeable`). This
commit removes all the usages apart from the `extends` clauses of its
direct descendants.
2025-04-08 03:50:28 +10:00
István Zoltán Szabó
212971a435
[DOCS] Adds ML-CPP release notes. (#126420) 2025-04-07 19:15:20 +02:00
Nhat Nguyen
894d92af85
Skip rerank tests in mixed clusters (#126415)
If the clusters don't support inference test services, skip tests that require inference services. 
Hence, we should check for rerank tests.

Relates #123074
2025-04-07 09:43:55 -07:00
Oleksandr Kolomiiets
21ff72bef4
Use FallbackSyntheticSourceBlockLoader for text fields (#126237) 2025-04-07 09:32:35 -07:00
David Kyle
20eb59080c
[ML] Move request managers into service package (#126114) 2025-04-07 16:01:49 +02:00
Moritz Mack
0360db2cd0
Improved reproduction of scaling EsExecutors bug #124667 to work with max pool size > 1. (#125045)
Relates to #124867, ES-10640
2025-04-07 15:40:25 +02:00
Nik Everett
7e1e45eaa4
ESQL: Speed up TO_IP (#126338)
Speed up the TO_IP method by converting directly from utf-8 encoded
strings to the ip encoding. Previously we did:
```
utf-8 -> String -> INetAddress -> ip encoding
```

In a step towards solving #125460 this creates three IP parsing
functions, one the rejects leading zeros, one that interprets leading
zeros as decimal numbers, and one the interprets leading zeros as octal
numbers. IPs have historically been parsed in all three of those ways.

This plugs the "rejects leading zeros" parser into `TO_IP` because
that's the behavior it had before.

Here is the performance:
```
Benchmark               Score    Error  Units
leadingZerosAreDecimal  14.007 ± 0.093  ns/op
leadingZerosAreOctal    15.020 ± 0.373  ns/op
leadingZerosRejected    14.176 ± 3.861  ns/op
original                32.950 ± 1.062  ns/op
```

So this is roughly 45% faster than what we had.
2025-04-07 09:34:53 -04:00
Mike Pellegrini
72066ea49f
Update InferenceException to retain top-level message (#126345) 2025-04-07 09:05:00 -04:00
Pete Gillin
549fddb348
ES-10037 Tweak wording of autosharding logs (#126339)
ES-10037 #comment Tweaked wording of logging in https://github.com/elastic/elasticsearch/pull/126339
2025-04-07 13:15:52 +01:00
David Turner
527d2a203b
Improve handling of empty response (#125562)
Today `ActionResponse$Empty` implements `ToXContentObject`, but yields
no bytes of content when serialized which creates an invalid JSON
response. This commit removes the bogus interface and adjusts the
affected REST APIs to send a `text/plain` response instead.
2025-04-07 12:10:07 +01:00
Alexander Spies
a152b4e29b
ESQL: Fail with 500 not 400 for ValueExtractor bugs (#126296)
In case of wrong layouts of ESQL's operators, it can happen that
ValueExtractor.extractorFor encounters a data type mismatch. Currently,
this throws IllegalArgumentException, which is treated like a user
exception and triggers a 400 response.

We need to return a 500 status code for such errors; this is also
important for observability of ES clusters, which can normally use 500
responses as an indicator of a bug.

Throw IllegalStateException instead, it's close enough.
2025-04-07 11:21:57 +02:00
Christoph Büscher
f8b99258f4
Re-enable SearchProgressActionListenerIT testSearchProgressWithQuery (#124302) 2025-04-07 11:13:47 +02:00
Craig Taverner
1f6518f371
Document special behaviour of ignore_malformed for geo_point mappings (#125692)
With `geo_point` fields, here is the special case of values that have a syntactically valid format, but the numerical values for `latitude` and `longitude` are out of range.

If `ignore_malformed` is `false`, an exception will be thrown as usual. But if it is `true`, the document will be indexed correctly, by normalizing the latitude and longitude values into the valid range. The special `_ignored` field will not be set. The original source document will remain as before, but indexed values, doc-values and stored fields will all be normalized.
2025-04-07 11:05:51 +02:00
Jan Kuipers
24909ca9df
Adaptive allocations improvements (#126307)
* Adaptive allocations: don't update deployments that aren't started.

* AssignmentPlanner: don't plan deployments with zero allocations

* Update JavaDoc
2025-04-07 08:46:32 +02:00
Slobodan Adamović
0b09506b54
Improve error handling during index expressions resolving (#126018)
The `InvalidIndexNameException` exception was wrapped in a `ElasticsearchSecurityException`, which returns HTTP `403` status. 

This exception (along with newly introduced `InvalidSelectorException` and `UnsupportedSelectorException`) can be raised during index expression resolving due to an invalid user input and should result in HTTP `400` response instead.

This PR changes exception handling to avoid wrapping them in the `ElasticsearchSecurityException`.
2025-04-06 07:09:24 +02:00
elasticsearchmachine
1bace0b70d Mute org.elasticsearch.packaging.test.DockerTests test022InstallPluginsFromLocalArchive #116866 2025-04-06 09:42:11 +10:00
elasticsearchmachine
d7095c360c Mute org.elasticsearch.smoketest.MlWithSecurityIT test {yaml=ml/start_data_frame_analytics/Test start classification analysis when the dependent variable cardinality is too low} #123200 2025-04-06 09:39:23 +10:00
elasticsearchmachine
2ccf0c3513 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker using multiple fields ASYNC} #126371 2025-04-06 09:12:19 +10:00
elasticsearchmachine
ea440aa0b8 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker using a single field SYNC} #126370 2025-04-06 09:12:10 +10:00
elasticsearchmachine
947d33bf0f Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker using a single field ASYNC} #126369 2025-04-06 09:12:02 +10:00
elasticsearchmachine
e808bc2c33 Mute org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT test {p0=search.vectors/42_knn_search_bbq_flat/Vector rescoring has same scoring as exact search for kNN section} #126368 2025-04-06 08:06:58 +10:00
elasticsearchmachine
354a56dc91 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker add the _score column when missing SYNC} #126367 2025-04-06 07:43:02 +10:00
elasticsearchmachine
a53ae214f3 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker using another sort order SYNC} #126363 2025-04-06 02:22:17 +11:00
elasticsearchmachine
83418244ed Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker using another sort order ASYNC} #126362 2025-04-06 02:22:10 +11:00
elasticsearchmachine
0255c10866 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker before a limit ASYNC} #126361 2025-04-06 02:22:02 +11:00
elasticsearchmachine
81033069dd Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {rerank.Reranker add the _score column when missing ASYNC} #126360 2025-04-06 02:21:55 +11:00
Nhat Nguyen
fbfc707d95
Support serialization of aggregate metric double literal (#126352)
To backport the newly introduced AggregateMetricDoubleLiteral to 8.x, we need
to override the supportsVersion method instead of getMinimalSupportedVersion.
2025-04-04 20:50:10 -07:00
elasticsearchmachine
a6aaccdb7f Mute org.elasticsearch.xpack.ilm.actions.SearchableSnapshotActionIT testSearchableSnapshotTotalShardsPerNode #126354 2025-04-05 11:59:20 +11:00
Lisa Cawley
1d1feb6010
[DOCS] Migrate search profile API examples (#126347) 2025-04-04 22:42:09 +01:00
elasticsearchmachine
67fca830df Mute org.elasticsearch.xpack.ilm.ClusterStateWaitThresholdBreachTests testWaitInShrunkShardsAllocatedExceedsThreshold #126348 2025-04-05 08:21:50 +11:00
Tim Brooks
68bc2b8600
Adjust method to transition split state (#126179)
Make the method to transition to handoff be generic and support multiple
states.
2025-04-04 15:20:07 -06:00