Commit graph

87430 commits

Author SHA1 Message Date
Luigi Dell'Aquila
b0bd718b96
ES|QL: add generative tests for CHANGE_POINT command (#129758) 2025-06-20 15:28:42 +02:00
Luigi Dell'Aquila
539d8d4a8e
ES|QL: Add multi-node generative tests (#129765) 2025-06-20 15:28:30 +02:00
Ievgen Degtiarenko
c1046b0704
Remove resolved mutes (#129750) 2025-06-20 15:13:52 +02:00
Iraklis Psaroudakis
2940f9a011
Accommodate hollow engine changes (#129535)
* Field infos calculation method inside Engine
* buildSeqNoStats as static public method

So it can be overriden in stateless if/as needed.

Relates ES-11457
2025-06-20 16:01:55 +03:00
Luke Whiting
a952245178
Fix missing feature flag on Streams YAML tests (#129747)
* Fix missing feature flag on YAML tests

* Unmute tests
2025-06-20 21:12:51 +10:00
Ioana Tagirta
0c0bf6bd93
Resolve Keep plan added to FORK branches (#129754) 2025-06-20 13:07:18 +02:00
Luke Whiting
18c1e55eb3
Reserve transport version for streams endpoint backport (#129753) 2025-06-20 20:57:35 +10:00
Moritz Mack
9b2ca99ca4
Tolerate incompatible versions with different build hash (#128589)
Tolerate incompatible versions with different build hash.
I'm keeping the serverless feature flag to not create warnings there.

Relates to ES-11869
2025-06-20 12:16:32 +02:00
Jan Kuipers
586c3b28ca
fix esql categorize csv test for old JDKs (#129746) 2025-06-20 19:57:43 +10:00
elasticsearchmachine
b2acda9a9f Mute org.elasticsearch.snapshots.SnapshotShutdownIT testSnapshotShutdownProgressTracker #129752 2025-06-20 19:12:39 +10:00
Ievgen Degtiarenko
f2703ce461
Fix toIpInAgg & toIpInSort (#129743) 2025-06-20 11:01:27 +02:00
Antonio
39a3e80b51
[ResponseOps] Granting kibana_system role access to the cases analytics indices (#129414)
* Update kibana system user permissions for cases analytics index.

* [CI] Auto commit changes from spotless

* Add permissions for aliases.
Create constants for indexes and aliases.
Update tests.

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2025-06-20 11:00:13 +02:00
Julia Bardi
4aa83e262a
[Fleet] add index privileges to support reroute processor (#129692)
* add index privileges to support reroute processor

* [CI] Auto commit changes from spotless

* add more indices

* update

* fix test

* remove unnecessary indices

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-06-20 10:41:51 +02:00
Dimitris Rempapis
bd4f5fee8d
ShardSearchStatsTests - add missing metrics and methods (#129311)
ShardSearchStatsTests - add missing metrics and methods - complete coverage
2025-06-20 11:32:52 +03:00
Nick Tindall
f715f63137
Fix and unmute GetSnapshotsIT (#129741)
Closes: 129740
2025-06-20 18:29:47 +10:00
Ankita Kumar
9e19b85783
Metrics to account for time spent waiting for next chunk (#129469)
This PR addresses ES-12071.

We want to collect metrics for the time that is spent waiting for the next chunk of a bulk request. This can help with diagnosing high bulk latency in case the latency is attributable to external factors such as network connection.

Co-authored-by: Francisco Fernández Castaño <francisco.fernandez.castano@gmail.com>
2025-06-20 08:21:18 +02:00
elasticsearchmachine
73a5ce2071 Mute org.elasticsearch.snapshots.GetSnapshotsIT testFilterByState #129740 2025-06-20 15:27:03 +10:00
Yang Wang
0e83b19de7
Remove obsolete Metadata BWC for repositories (#129685)
When migrating RepositoriesMetadata from cluster custom to project
custom (#125398), we needed temporary BWC handling for clusters running
on a version that is before this change but after the initial MP change.
Such a cluster can only exist in the serverless environment which has
progressed way past any applicable versions. Therefore we no longer need
the BWC handling and this PR removes it.

Relates: #125398
2025-06-20 11:34:48 +10:00
Brendan Cully
4ce06d1aa2
Add deleteByQuery to InternalEngine (#129679) 2025-06-19 15:40:38 -07:00
elasticsearchmachine
0f2806a9c2 Mute org.elasticsearch.streams.StreamsYamlTestSuiteIT test {yaml=streams/logs/10_basic/Check for repeated toggle to same state} #129735 2025-06-20 07:33:59 +10:00
elasticsearchmachine
2e13fca5ee Mute org.elasticsearch.streams.StreamsYamlTestSuiteIT test {yaml=streams/logs/10_basic/Basic toggle of logs state enable to disable and back} #129733 2025-06-20 07:33:50 +10:00
Rene Groeschke
29db3f3464
[Build] Extract logsdb rolling-upgrade tests (#129673)
- introduce separate subproject for testing logsdb rolling-upgrade tests
- should reduce :qa:rolling-upgrade test task durations
2025-06-19 22:04:36 +02:00
Iván Cea Fontenla
2b8d9df0ef
[ESQL] Fix match phrase tests not being deterministic (#129724)
Closes https://github.com/elastic/elasticsearch/issues/129676
2025-06-20 03:13:17 +10:00
Luigi Dell'Aquila
0dba16384e
ES|QL: Fix generative tests (#129717) 2025-06-20 02:13:26 +10:00
Mikhail Berezovskiy
eeca493860
Move HTTP content aggregation from Netty into RestController (#129302) 2025-06-19 09:05:17 -07:00
Albert Zaharovits
083326e658
Threadpool merge executor does not block aborted merges (#129613)
This PR addresses a bug where aborted merges are blocked if there's
insufficient disk space.

Previously, the merge disk space estimation did not consider if the
operation has been aborted when/while it was enqueued for execution.
Consequently, aborted merges, for e.g. when closing a shard, were
blocked if their disk space estimation was exceeding the available disk
space threshold. In this case, the shard close operation would itself
block.

This fix estimates a disk space budget of `0` for aborted merges, and it
periodically checks if any enqueued merge tasks have been aborted (more
generally, it checks if the budget estimate for any merge tasks has
changed, and reorders the queue if so). This way aborted merges are
prioritized and are never blocked.

Closes https://github.com/elastic/elasticsearch/issues/129335
2025-06-20 00:51:13 +10:00
Mary Gouseti
ee5d652411
Increase node up timeout in AbstractLocalClusterFactory (#129639)
In the last two months a lot of tests were converted to use the newer rest test framework. Some tests start 1 node, other start 3 nodes, others even more, the framework runs tests in parallel but it doesn't know how many nodes its tests needs meaning that running 3 tests in parallel, for example, can be very different when they are single node clusters or 3 node clusters etc. During this execution we saw the 3x more CPU load than what we would want to have ideally.

Currently there is no good solution for this because if dial down the concurrency we will use the nodes inefficiently, but if we keep the concurrency to where it is we risk longer start up times. Considering that the starting time of elasticsearch is not related to this test, we choose to increase the timeout to reduce the noise.
2025-06-19 17:37:37 +03:00
Aurélien FOUCRET
34ccaba56d
[ES|QL] RERANK command - Updating the syntax and behavior (#129488) 2025-06-19 15:46:33 +02:00
elasticsearchmachine
a0109bb0fe Mute org.elasticsearch.search.query.RescoreKnnVectorQueryIT testKnnSearchRescore #129713 2025-06-19 23:31:05 +10:00
Ioana Tagirta
c173522a71
Fix for FORK branches with mixed outputs and unsupported field types (#129636) 2025-06-19 14:20:22 +02:00
Jan Kuipers
7f12f8009b
fix MachineDependentHeapTests (#129699) 2025-06-19 14:07:39 +02:00
Ioana Tagirta
04231e9798
ES|QL: Make fork available in release builds (#129606) 2025-06-19 13:14:06 +02:00
Ignacio Vera
22eb035a27
Clone IndexInput when creating MemorySegmentPostingsVisitor (#129690) 2025-06-19 13:13:00 +02:00
Ioana Tagirta
0e538bdd61
Capability checks for tests using dense_vector (#129694) 2025-06-19 13:09:40 +02:00
Luke Whiting
1ccf1c6806
Streams - Log's Enable, Disable and Status endpoints (#129474)
* Enable And Disable Endpoint

* Status Endpoint

* Integration Tests

* REST Spec

* REST Spec tests

* Some documentation

* Update docs/changelog/129474.yaml

* Fix failing security test

* PR Fixes

* PR Fixes - Add missing feature flag name to YAML spec

* PR Fixes - Fix support for timeout and master_timeout parameters

* PR Fixes - Make the REST handler validation happy with the new params

* Delete docs/changelog/129474.yaml

* PR Fixes - Switch to local metadata action type and improve request handling

* PR Fixes - Make enable / disable endpoint cancellable

* PR Fixes - Switch timeout param name for status endpoint

* PR Fixes - Switch timeout param name for status endpoint in spec

* PR Fixes - Enforce local only use for status action

* PR Fixes - Refactor StreamsMetadata into server

* PR Fixes - Add streams module to multi project YAML test suite

* PR Fixes - Add streams cluster module to multi project YAML test suite
2025-06-19 11:48:44 +01:00
Yang Wang
6858c32529
[Test] Allow allocation in mixed cluster (#129680)
The RunningSnapshotIT upgrade test adds shutdown markers to all nodes
and removes them once all nodes are upgraded. If an index gets created
in a mixed cluster, for example by ILM or deprecation messages, the
index cannot be allocated because all nodes are shutting down. Since the
cluster ready check between node upgrades expects a yellow cluster, the
unassigned index prevents the ready check to succeed and eventually
timeout. This PR fixes it by removing shutdown marker for the 1st
upgrade node to allow it hosting new indices.

Resolves: #129644 Resolves: #129645 Resolves: #129646
2025-06-19 20:13:11 +10:00
Jan Kuipers
5449d95dcd
ES|QL categorize tech preview -> GA (#129398) 2025-06-19 12:06:28 +02:00
Pete Gillin
8555c16290
Fix testPurgeCacheEntriesForDatabase on Windows (#129648)
Our hypothesis is that the path is not round-tripping on Windows. This
fixes the test by always using the canonical string for the path, when
inserting and when purging.

Fixes #129635
2025-06-19 09:06:09 +01:00
Jan Kuipers
1bcd9b4170
Use standard tokenizer in ES|QL categorize (#129642)
* Use standard tokenizer in ES|QL categorize

* rename capability

* remove unused param

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-06-19 08:54:25 +02:00
Aurélien FOUCRET
b02519b24e
Fix size of the bulks used in BulkInferenceExecutorTests (#129640) 2025-06-19 16:45:27 +10:00
Yang Wang
91ee26f690
[Test] Use default project s3 repo tests (#129683)
The stateful side tests generally do not exercise multi-project.
Therefore we need to use the default project-id instead of random one.

Relates: #127631
2025-06-19 16:42:19 +10:00
Yang Wang
0932beb1f8
Remove obsolete Metadata.FORMAT field and usages (#129519)
The only production usage is for cleaning up all global state files. It
is replaced by directly calling the relevant method without creating the
FORAMT instance. Test only usages are either replaced by equivalent
method calls or dropped.

Relates: #114698
2025-06-19 15:38:33 +10:00
Yang Wang
92b32b535b
Passing in project-id when creating s3 client (#129301)
Enables creating different s3 clients for different projects

Relates: #127631
2025-06-19 12:34:11 +10:00
Niels Bauman
a0b8737380
Make ILM AsyncActionStep project-aware (#129632)
Updates all ILM async actions to operate on `ProjectState` and use a
project client.
2025-06-19 09:00:09 +10:00
Keith Massey
92e4244f8e
Putting the ingest otel processor behind the logs stream feature flag (#129667) 2025-06-18 17:40:12 -05:00
elasticsearchmachine
32827386c2 Mute org.elasticsearch.xpack.esql.qa.multi_node.EsqlSpecIT test {scoring.TestMatchPhraseWithScoreBoost ASYNC} #129676 2025-06-19 08:08:39 +10:00
Jonathan Buttner
2e566c9506
[ML] CustomService adding template validation prior to request flow (#129591)
* Adding template validation prior to request flow

* Fixing tests

* Narrowing exception
2025-06-19 07:02:46 +10:00
Nik Everett
0f986769e0
ESQL: Fix PushQueryIT#testEqualityOrTooBig (#129657)
Fixes a test about query pushing for `==` to lucene.

Closes #129545
2025-06-19 06:40:42 +10:00
Brian Seeders
e53592e460
[CI] Move default CI OS to ubuntu-2404 and drop ubuntu-2004 (#129008) 2025-06-18 15:37:00 -04:00
Julian Kiryakov
42047ac0d3
Fix PushQueriesIT.testLike() fails (#129647)
Fix an issue where PushQueriesIT.testLike() fails if we randomly generate the empty string as a test value
2025-06-18 15:36:08 -04:00