* Use String.replace() instead of replaceAll() for non-regexp replacements
When arguments do not make use of regexp features replace() is a more efficient option, especially the char-variant.
This PR changes SAML transport actions to use `GENERIC` executor
in order to avoid executing potentially slow and blocking IO/HTTP
operations on the `transport_worker` threads.
Fixes#104962
Loads of code here that is only used in tests and one duplicate unused
class that was only used as an indirection to parsing the
`AsyncSearchResponse`. Moved what I could easily move via automated
refactoring to `SearchResponseUtils` in tests and removed the duplicate
now unused class from the client codebase.
To improve cross-cluster search user experience, Kibana needs an endpoint that is accessible
by arbitrary Kibana dashboard search users and provides:
1. a listing of clusters in scope for a CCS query (based on the index expression and whether
there are any indices on each cluster that the Kibana user has access to query).
2. whether that cluster is currently connected to the querying cluster (will it come back as
skipped or failed in a CCS search)
3. showing the skip_unavailable setting for those clusters (so you can know whether it will
return skipped or failed in a CCS search)
4. the ES version of the cluster
Since no single Elasticsearch endpoint provides all of these features, this PR creates a new endpoint `_resolve/cluster` that works along side the existing `_resolve/index` endpoint
(and leverages some of its features).
Example usage against a cluster with 2 remote clusters configured:
GET /_resolve/cluster/*,remote*:bl*
Response:
{
"(local)": {
"connected": true,
"skip_unavailable": false,
"matching_indices": true,
"version": {
"number": "8.12.0-SNAPSHOT",
"build_flavor": "default",
"minimum_wire_compatibility_version": "7.17.0",
"minimum_index_compatibility_version": "7.0.0"
}
},
"remote2": {
"connected": true,
"skip_unavailable": true,
"matching_indices": true,
"version": {
"number": "8.12.0-SNAPSHOT",
"build_flavor": "default",
"minimum_wire_compatibility_version": "7.17.0",
"minimum_index_compatibility_version": "7.0.0"
}
},
"remote1": {
"connected": true,
"skip_unavailable": false,
"matching_indices": false,
"version": {
"number": "8.12.0-SNAPSHOT",
"build_flavor": "default",
"minimum_wire_compatibility_version": "7.17.0",
"minimum_index_compatibility_version": "7.0.0"
}
}
}
Almost all errors show up as "error" entries in the response.
Only the local SecurityException returns a 403 since that happens before the ResolveCluster
Transport code kicks in.
A predicate to check whether the cluster supports a feature is available
to rest handlers defined in server. This commit adds that predicate to
plugins defining rest handlers as well.
Closes#97032
Adding the ability to set `require_data_stream` parameter (boolean) on bulk and indexing APIs.
For document indexing, this flag requires the indexing operation to either be pointed at a data stream, or match a template that will create a data stream.
* Introduce Prerequisites criteria (Predicate + factory) for modular skip decisions
- Removed accessors to specific criteria from SkipSection (used only on tests), adjusted test assertions
- Moved Features check (YAML test runner features) to SkipSection build time
* Separated check for xpack/no_xpack
Check for xpack is cluster-configuration (modules installed) dependent, while Features are meant to be "static" test-runner capabilities. We separate them so checks on one (test-runner features) can be run before and separately from the other.
* Consolidate skip() methods
- Divide require and skip predicates
- Divide requires and skip parsing (distinct sections)
- Renaming SkipSection to PrerequisiteSection and related methods/fields (e.g. skip -> evaluate)
* Refactoring tests
- moving and adding VersionRange tests
- adding specific version and os skip tests
- modified parse/validate/build to make SkipSection more unit-testable
* Adding cluster feature-based skip criteria
* Updated javadoc + renaming + better skip reason message
This weakens an assertion in the ESQL tests rolling upgrade tests so
they'll pass against older versions of Elasticsearch. Apparently the
warning message changed. There isn't a good reason to be so strict about
the assertion anyway.
Closes#104101
It seems that this test failure was purely down to inconsistency of ml-cpp
dependencies, and not a real problem. Therefore the test can be unmuted.
Fixes#104193
This adds support for the type parameter to the Query API key API.
The type for an API Key can currently be either rest or cross_cluster.
Relates: #101691
This removes a replica from a "full cluster updrade" test for esql.
There are rare errors that look to be caused by this test and I think
this might have something to do with it. Either way, it doesn't hurt
anything.
Loading fields from `_source` is *super* slow because you have to
decompress the stored fields and then turn the stored field into a
map-of-maps. And then dig through the map-of-maps. This adds "does this
field exist" style checks before most loads from `_source`. Not all
fields can do it, but most fields can.
This really improves the performance of our
`esql_dissect_duration_and_stats` benchmark, mostly because it is
running `dissect` on a field that has to load from `_source` that isn't
in many of the documents. Here's the performance:
```
| 50th percentile service time | 867.667 | 100.491 | -767.176 | ms | -88.42% |
| 90th percentile service time | 886.042 | 102.434 | -783.608 | ms | -88.44% |
| 100th percentile service time | 893.035 | 104.598 | -788.437 | ms | -88.29% |
```
This pull request introduces support for cross-clusters query (CCQ) in
ESQL. The enrich in CCQ will be implemented in a separate pull request
due to its complex semantic nature. The primary change occurs in the
ComputeService class, where a cluster-compute-action is introduced.
The implementation in this PR is equivalent to CCS with
ccs_minimize_round_trips enabled. Currently, our plan is to support a
single mode in CCQ. At present, the coordinator on the remote cluster
collects pages from data nodes in the same cluster and provides them to
the coordinator of the main cluster. This is achieved using two exchange
buffers, although a single exchange buffer could suffice. However, the
use of two buffers allows for future execution of a plan on this
coordinator to perform partial reduce operations, such as limit, topN,
and partial-to-partial aggregation.
Security and backward compatibility tests have been added in the
multi-cluster-search-security and multi-clusters QA modules,
respectively.
It's in the title, fix a couple of tests here and there. Also, fix REST
tests in particular and add a utility to cleanly read a SearchResponse
from a REST response without leaking the parser.
for #102030
This commit adds a dummy IndexVersion constant for 8.13. It is needed
for CCR tests which expect a different index version between current and
previous releases.
Cleaning this up a little even though it's still quite horrible.
`.get()` in this API actually means `actionGet()` so to speak.
I think a good first step to cleaning this up is to at least reduce
the duplication though and save 1k lines.
