Commit graph

82736 commits

Author SHA1 Message Date
Mark Vieira
4f4527b039
Fix enterprise search yaml rest compatibility test
(cherry picked from commit de119e4a55)
2025-03-28 15:43:56 -07:00
Mark Vieira
b0ef9bc79c
[8.18] Convert enterprise search module to new testing framework (#125807) (#125879)
* Convert enterprise search module to new testing framework (#125807)

(cherry picked from commit 3ac0c136aa)

# Conflicts:
#	x-pack/plugin/ent-search/qa/rest/build.gradle

* Fix task name
2025-03-29 05:52:42 +11:00
Benjamin Trent
23ea73d36f
Adjusting 41_knn_search_bbq_hnsw tests to have explicit refresh (#125255) (#125876)
(cherry picked from commit e9c4b267c2)
2025-03-29 05:14:01 +11:00
Charlotte Hoblik
b7a238af60
8.18.0 Connectors release notes (#125761) (#125853) 2025-03-29 01:54:32 +11:00
Craig Taverner
fa0db81652
[8.x] Add integration tests that verify ST_EXTENT_AGG needs a license (#125584) (#125712) (#125719) (#125834)
* Add integration tests that verify ST_EXTENT_AGG needs a license (#125584) (#125712)

* Fix java version problem on backport

* Use PLATINUM as license level for spatial features
2025-03-28 22:27:54 +11:00
Carlos Delgado
51b715f2bc
[8.18] Non existing synonyms sets do not fail shard recovery (#125659) (#125796) 2025-03-28 07:39:43 +01:00
Mark Vieira
dc62cd476c
Migrate x-pack core module to new testing framework (#125740) (#125799) 2025-03-28 05:24:43 +11:00
Alexander Spies
a2e2337ad1
[8.x] ESQL: Improve LOOKUP JOIN page (#125688) (#125797)
Some minor edits, formatting improvements etc.

(cherry picked from commit 3536c6d110)
2025-03-28 04:19:53 +11:00
Mark Vieira
44b928b206
[8.18] Migrate legacy QA projects to new test clusters framework (#125545) (#125718) 2025-03-27 10:07:37 -07:00
shainaraskas
a99453cce2
anchor addition for xlink (#125787) 2025-03-27 17:50:48 +01:00
Mark Vieira
980d6509a3
[9.0] Convert :test projects to new testing framework (#125724) (#125742) (#125791)
* Convert `:test` projects to new testing framework (#125724)

(cherry picked from commit e149a3e10d)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/RestrictedBuildApiService.java
#	test/external-modules/error-query/build.gradle
#	test/external-modules/error-query/src/javaRestTest/java/org/elasticsearch/test/esql/EsqlPartialResultsIT.java

* Bump timeout

(cherry picked from commit 533d38d863)

# Conflicts:
#	test/external-modules/error-query/build.gradle
2025-03-28 03:50:31 +11:00
David Kyle
592e681775
[ML] Check if the anomaly results index has been rolled over (#125404)
If the v7 index has already been rolled over don't try again
2025-03-27 17:15:48 +02:00
Benjamin Trent
4ee84ddc5f
Update bbq test data to better distinguish docs (#125705) (#125778)
Adjust the test data. I verified that the scores are now more
distinguishable when:

 - each doc has its own segment
 - when 1 & 2 are in the same segment but 3 is alone
 - 2 & 3 in the same segment but 1 alone
 - 1 & 3 in the same segment but 2 alone
 - all three in the same segment

closes: https://github.com/elastic/elasticsearch/issues/123727 closes:
https://github.com/elastic/elasticsearch/issues/124848
(cherry picked from commit d84eb1f53f)
2025-03-28 01:39:31 +11:00
Liam Thompson
cc676a0b3c
[DOCS][8.x] ESQL for search updates (#125759) (#125772)
* [DOCS][8.x] ESQL for search updates

(cherry picked from commit b964982cd4)

# Conflicts:
#	docs/reference/search/search-your-data/semantic-text-hybrid-search
2025-03-28 00:49:55 +11:00
Mary Gouseti
e2b7bdb049
Fix p0=deprecation/10_basic/Test Deprecations (#125687) (#125768)
In #120505 we introduced a capabilities check in the yaml test `deprecation/10_basic/Test Deprecations` but we forgot to add them in the `RestDeprecationInfoAction`.

In this PR we add the capabilities which will enable the test and we make the test resilient to the warning that occurs when the `.security-7` index is present.
2025-03-28 00:10:36 +11:00
Benjamin Trent
96def2843f
Return appropriate error on null dims update instead of npe (#125716) (#125767)
Calling `Object::toString` was trying to call `null.toString()`, really
it should have been `Objects::toString`, which accepts `null`.

closes: https://github.com/elastic/elasticsearch/issues/125713
(cherry picked from commit dd58b0b6fa)
2025-03-27 23:48:36 +11:00
Armin Braun
0296f7f935
Load FieldInfos from store if not yet initialised through a refresh on IndexShard (#125650) (#125762)
Load field caps from store if they haven't been initialised through a refresh yet.
Keep the plain reads to not mess with performance characteristics too much on the good path but protect against confusing races when loading field infos now (that probably should have been ordered stores in the first place but this was safe due to other locks/volatiles on the refresh path).

Closes #125483
2025-03-27 13:20:41 +01:00
Yang Wang
7bf6161c55
Format custom authz engine code (#125546) (#125753)
No compilation issue in this branch Relates: #125469
2025-03-27 21:10:59 +11:00
Mark Vieira
f236efc1d5
Convert remaining plugin projects to new test clusters framework (#125626) (#125726)
(cherry picked from commit 930b4ab995)

# Conflicts:
#	plugins/discovery-azure-classic/build.gradle
#	plugins/discovery-gce/qa/gce/build.gradle
2025-03-27 09:41:37 +11:00
Fang Xing
5c1e17c51b
[ES|QL] Make numberOfChannels consistent with layout map by removing duplicated ChannelSet (#125636) (#125721)
* make numberOfChannels consistent with layout

(cherry picked from commit 80125a4bac)

# Conflicts:
#	x-pack/plugin/esql/qa/server/src/main/java/org/elasticsearch/xpack/esql/qa/rest/RestEsqlTestCase.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
2025-03-27 08:53:29 +11:00
Mark Vieira
eeea3d7a83
Convert ingest-geoip file based update tests to new testing framework (#125632) (#125700) 2025-03-27 07:56:23 +11:00
Mark Vieira
3dc0f25b0f
[Test] Allow configuring configDir for the Java test cluster (#125094) (#125708)
For creating and deleting projects in multi-project tests, we need
create and delete settings and secrets files on the fly. This PR adds
such feature to the Java test cluster with an option to specify the
config directory.

(cherry picked from commit a1b0ed104b)

Co-authored-by: Yang Wang <yang.wang@elastic.co>
2025-03-27 06:23:37 +11:00
Stanislav Malyshev
a368b41ca7
[8.18] Handle long overflow in dates (#124048) (#125697)
* Handle long overflow in dates (#124048)

* Handle long overflow in dates

(cherry picked from commit 07921a78a6)

# Conflicts:
#	rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/search/500_date_range.yml
#	server/src/main/java/org/elasticsearch/index/mapper/DateFieldMapper.java
#	server/src/main/java/org/elasticsearch/index/mapper/MapperFeatures.java

* Add import
2025-03-27 05:34:59 +11:00
Mark Vieira
ca2fa59bc6
Convert remaining module projects to new test clusters framework (#125613) (#125676) 2025-03-27 03:53:02 +11:00
Liam Thompson
69115b5758
[DOCS][8.x] Add Using ESQL for search landing page (#125580) (#125653) 2025-03-26 22:17:36 +11:00
elasticsearchmachine
bb43a7d192 Mute org.elasticsearch.qa.verify_version_constants.VerifyVersionConstantsIT testLuceneVersionConstant #125638 2025-03-26 15:57:35 +11:00
Mark Vieira
fb3fc17777
[8.18] Re-enable VerifyVersionConstantsIT (#125605) (#125616)
* Re-enable VerifyVersionConstantsIT (#125605)

(cherry picked from commit 65751062f7)

# Conflicts:
#	qa/verify-version-constants/build.gradle

* Fix test
2025-03-26 13:43:07 +11:00
Keith Massey
7c6e1303c4
Using a consistent index template name to avoid undefined behavior (#125624) (#125629) 2025-03-26 10:58:29 +11:00
Mark Tozzi
25d9c20cbc
Esql - Fix lucene push down behavior when a range contains nanos and millis (#125595) (#125618)
Follow up to #125345. If the query contained both a nanos and a millis comparison, we were formatting the dates incorrectly for the lucene push down. This PR adds a test and a fix for that case.
---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-03-26 07:34:32 +11:00
shainaraskas
bb39781db7
[8.18] fix anchors (#125589) 2025-03-25 12:47:04 -04:00
elasticsearchmachine
dd6f622064 Prune changelogs after 8.17.4 release 2025-03-25 15:25:01 +00:00
elasticsearchmachine
b4f07e2837 Bump versions after 8.17.4 release 2025-03-25 15:23:21 +00:00
elasticsearchmachine
23477c22d0 Prune changelogs after 8.16.6 release 2025-03-25 12:25:50 +00:00
elasticsearchmachine
65f15e5e82 Bump versions after 8.16.6 release 2025-03-25 12:23:43 +00:00
kosabogi
2d41927b8d
Updates Watsonx inference integration with rerank feature (#125565) 2025-03-25 12:39:08 +01:00
Charlotte Hoblik
274b59a2f5
Update ccs compatibility matrix with 8.18 (#125309) (#125578)
* update ccs matrix with 8.18

* fix table size

* fix compatibility matrix
2025-03-25 22:38:09 +11:00
Valeriy Khakhutskyy
55476abc9a
[ML] Increasing timeout for DFA integration tests #125510 (#125571)
Increase waiting time to prevent DFA integration tests from timing out.
2025-03-25 21:56:45 +11:00
Alexander Spies
b36a5ffc66
[8.x] ESQL: Add more details on ENRICH vs. LOOKUP JOIN #125487 (#125528) (#125564)
Manual backport of docs-PR #125487
2025-03-25 20:07:12 +11:00
Mark Tozzi
f7d6a78726
ESQL - date nanos range bug? (#125345) (#125540)
Fixes #125439

We were incorrectly formatting nanosecond dates when building lucene queries. We had missed this in our testing because none of the CSV tests were running against Lucene. This happened because the date nanos test data includes multivalue fields. Our warning behavior for multivalue fields is inconsistent between queries run in Lucene and queries run in pure ES|QL without pushdown. Our warning tests, however, require that the specified warnings be present in all execution paths. When we first built the date nanos CSV tests, we worked around this by always using an MV function to unpack the multivalue fields. But we forgot that using an MV function prevents the entire query from being pushed down to Lucene, and thus that path wasn't being tested.

In this PR, I've duplicated many of the tests to have a version that doesn't use the MV function, and uses warningRegex instead of warning. The regex version does not fail if the header is absent, so it's safe to use in both modes. Rewriting the tests this way revealed several situations in which this bug can manifest, all of which are fixed in this PR. I cannot be confidant that there aren't more paths that can trigger this bug and aren't covered by these tests, but I haven't found any yet.

I've left some trace level logging that I found helpful while debugging this.

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-03-25 10:38:28 +11:00
Patrick Doyle
a55471d6ea
Unmute entitlements unit tests (#125217) (#125530) 2025-03-25 08:01:47 +11:00
Ryan Ernst
eee3e1d686
Add --with-plugins to run task (#125468) (#125522)
This commit adds a new command line option to the run task to include
plugins. The option takes in a comma separated list of plugins from the
plugins directory. For example:
```
./gradlew run --with-plugins=analysis-icu,analysis-nori
```
2025-03-25 05:53:27 +11:00
Alexander Spies
4956c2607d
ESQL: Add ENRICH vs LOOKUP JOIN to ENRICH docs (#125489) (#125505)
This was initially part of
https://github.com/elastic/elasticsearch/pull/124531 but got lost during
the manual backport.
2025-03-25 02:54:25 +11:00
Dan Rubinstein
fde490fe44
Fix AlibabaCloudSearchCompletionAction not accepting ChatCompletionInputs (#125023) (#125416)
* Fix AlibabaCloudSearchCompletionAction not accepting ChatCompletionInputs

* Update docs/changelog/125023.yaml

* Fix unit tests

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2025-03-24 11:06:08 -04:00
István Zoltán Szabó
f9a0509301
[8.18] [DOCS] Update service-openai.asciidoc (#125490)
* Update service-openai.asciidoc (#125419)

Many customers want to use our OpenAI Inference endpoint against OpenAI compatible API's they have written, or Ollama, or Nvidia Triton OpenAI API front end. I had heard that was the intent of this OpenAI inference endpoint, but we do not state it directly. Can we validate this is OK with Search PM and include this?

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/inference/service-openai.asciidoc

---------

Co-authored-by: Brad Quarry <38725582+bradquarry@users.noreply.github.com>
2025-03-24 14:52:48 +01:00
kosabogi
5bf7249739
Document delay subparameter in transform checkpoints and usage guide (#125280) (#125496)
* Adds explanation on the delay parameter

* Attribute fixes

* Update docs/reference/transform/usage.asciidoc



---------

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
2025-03-25 00:43:07 +11:00
Simon Cooper
4a5494d5fb
Re-enable SIMD operations on JDK 24 (#125484) (#125492) 2025-03-25 00:18:11 +11:00
Lorenzo Dematté
33944f61b0
[Entitlements] Print a warning during plugin installation if a legacy policy file is found (#125294) (#125473)
This PR adds a check and a warning message if the user is trying to install a plugin containing the legacy plugin-security.policy used by SecurityManager.

Relates to ES-10846
2025-03-24 19:41:37 +11:00
Rene Groeschke
61d97c3185
[Gradle] Wire java toolchain plugin into integration tests (#125347) (#125463)
* [Gradle] Wire java toolchain plugin into integration tests

This addresses transient issues we see in the integration tests where previously resolved jdk toolchain
is not available. See https://gradle-enterprise.elastic.co/s/66msrp7n2nsdy/tests/task/:build-tools-internal:integTest/details/org.elasticsearch.gradle.internal.test.rest.LegacyYamlRestTestPluginFuncTest/yamlRestTest%20executes%20and%20copies%20api%20and%20tests%20to%20correct%20source%20set\?top-execution\=1 for example

* Fix fix
2025-03-24 06:22:05 +11:00
Niels Bauman
5a945d6699
Fix NPE in rolling over unknown target and return 404 (#125352) (#125455)
Since #122905 we were throwing NPEs (i.e. 5xxs) when a rollover request has an unknown/non-existent target. Before that, we returned a 400 - illegal argument exception. We now return a 404 which matches "missing target" better. Additionally, to avoid this from happening again, we add a YAML test that asserts the correct exception behavior.

(cherry picked from commit fdd453734d)

# Conflicts:
#	server/src/main/java/org/elasticsearch/action/admin/indices/rollover/MetadataRolloverService.java
2025-03-23 01:13:12 +11:00
Ryan Ernst
33dc75fede
Move remaining check methods in RestEntitlementsCheckAction (#125351) (#125443)
* Move remaining check methods in RestEntitlementsCheckAction (#125351)

All future check methods should be found reflectively with the
EntitlementTest annotation.

* Fix compilation after #125442
2025-03-22 09:03:37 +11:00