This computation involves parsing all the pipeline metadata on the
cluster applier thread. It's pretty expensive if there are lots of
pipelines, and seems mostly unnecessary because it's only needed for a
validation check when creating new processors.
* Reset job if existing reset fails (#106020)
* Try again to reset a job if waiting for completion of an existing reset task fails.
* Update docs/changelog/106020.yaml
* Update 106020.yaml
* Update docs/changelog/106020.yaml
* Improve code
* Trigger rebuild
These Java versions are EOL and no longer supported by Elasticsearch so
we can remove them from our CI testing. We only need to support LTS
versions >= 17 and the currently latest bundled JDK version.
This uses the dedicated index block API in the docs for the shrink, split, and clone APIs, rather than putting the block in as a setting directly. The specialized API will wait for ongoing operations to finish, which is better during indexing operations.
Resolves#105831
When using a pre-filter with nested kNN vectors, its treated like a
top-level filter. Meaning, it is applied over parent document fields.
However, there are times when a query filter is applied that may or may
not match internal nested or non-nested docs. We failed to handle this
case correctly and users would receive an error.
closes: https://github.com/elastic/elasticsearch/issues/105901
Previously the `categorize_text` aggregation could throw an
exception if nested as a sub-aggregation of another aggregation
that produced empty buckets at the end of its results. This
change avoids this possibility.
Fixes#105836
This change ensures that the matches implementation of the `SourceConfirmedTextQuery` only checks the current document instead of calling advance on the two phase iterator. The latter tries to find the first doc that matches the query instead of restricting the search to the current doc. This can lead to abnormally slow highlighting if the query is very restrictive and the highlight is done on a non-matching document.
Closes#103298
* ESQL: fix single valued query tests (#105986)
In some cases the tests for our lucene query that makes sure a field is
single-valued was asserting incorrect things about the stats that come
from the query. That was failing the test from time to time. This fixes
the assertion in those cases.
Closes#105918
* ESQL: Reenable svq tests
We fixed the test failure in #105986 but this snuck in.
Closes#105952
The GeoIP endpoint does not use the xpack http client. The GeoIP downloader uses the JDKs builtin cacerts.
If customer is using custom https endpoint they need to provide the cacert in the jdk, whether our jdk bundled in or their jdk. Otherwise they will see something like
```
...PKiX path building failed: sun.security.provier.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target...
```
(cherry picked from commit 30828a5680)
Co-authored-by: Jennie Soria <predogma@users.noreply.github.com>
* Update README.asciidoc
updating the readme with the latest blurb from PMM and a reference to RAG + a few links to search labs content.
* Tweak verbiage
---------
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
(cherry picked from commit 0b664dd4d4)
Co-authored-by: Serena Chou <serenachou@users.noreply.github.com>
First check whether the full cluster supports a specific indicator (feature) before we mark an indicator as "unknown" when (meta) data is missing from the cluster state.
We seem to have a couple of checks to make sure we delete the data
stream when the last index reaches the delete step however, these checks
seem a bit contradictory.
Namely, the first check makes use if `Index` equality (UUID included)
and the second just checks the index name. So if a data stream with just
one index (the write index) is restored from snapshot (different UUID)
we would've failed the first index equality check and go through the
second check `dataStream.getWriteIndex().getName().equals(indexName)`
and fail the delete step (in a non-retryable way :( ) because we don't
want to delete the write index of a data stream (but we really do if the
data stream has only one index)
This PR makes 2 changes: 1. use the index name equality everywhere in
the step (we already looked up the index abstraction and the parent data
stream, so we know for sure the managed index is part of the data
stream) 2. do not throw exception when we got here via a write index
that is NOT the last index in the data stream but report the exception
so we keep retrying this step (i.e. this enables our users to simply
execute a manual rollover and the index is deleted by ILM eventually on
retry)
The heap attack tests hit OOM where the circuit breaker was
under-accounted. This was because the ProjectOperator retained
references to released blocks. Consequently, the released block couldn't
be GCed although we have decreased memory usage in the circuit breaker.
Relates #10563
We only had a few mentions of 429 handling, now documenting our expectation generically.
Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
The build handles platform specific code which may be for arm or x86.
Yet there are multiple ways to describe 64bit x86, and the build
converts between the two in several places. This commit consolidates on
the x64 nomenclature in most places, except where necessary (eg ML still
uses x86_64).
relates #105715
* ESQL: Fix wrong attribute shadowing in pushdown rules (#105650)
Fix https://github.com/elastic/elasticsearch/issues/105434
Fixes accidental shadowing when pushing down `GROK`/`DISSECT`, `EVAL` or
`ENRICH` past a `SORT`.
Example for how this works:
```
...
| SORT x
| EVAL x = y
...
pushing this down just like that would be incorrect as x is used in the SORT, so we turn this essentially into
...
| EVAL $$x = x
| EVAL x = y
| SORT $$x
| DROP $$x
...
```
The same logic is applied to `GROK`/`DISSECT` and `ENRICH`.
This allows to re-enable the dependency checker (after fixing a small
bug in it when handling `ENRICH`).
* Make OptimizerRules compile again
ILM transitions to `wait-for-index-color` (a step that needs a cluster
state changed event to evaluate against) but misses the cluster state
event that notifies that `partial-index` is now `GREEN`. And then the
cluster is quiet and no more state changes occur and we timeout. Note
that the test is unblocked by the teardown of the IT that triggers some
cluster state changes.
This fixes the test by issueing some empty `reroute` request to cause
some cluster state traffic in the cluster and ILM notices an index is
assigned.
Note that a production cluster is busy and ILM would eventually notice
the new state and make progress.
```
2024-02-22T06:33:01,388][INFO ][o.e.x.i.IndexLifecycleTransition] [node_t0] moving
index [index] from [{"phase":"frozen","action":"searchable_snapshot","name":"mount-snapshot"}] to [{"phase":"froz
en","action":"searchable_snapshot","name":"wait-for-index-color"}] in policy [policy]
[2024-02-22T06:33:01,490][INFO ][o.e.c.r.a.AllocationService] [node_t0] current.health="GREEN"
message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started
[[partial-index][0]]])." previous.health="YELLOW" reason="shards started [[partial-index][0]]"
```
Fixes#102405
For Unattended Transforms, if we fail to create the destination index on
the first run, we will retry the transformation iteration, but we will
not retry the destination index creation on that next iteration.
This change stops the Unattended Transform from progressing beyond the
0th checkpoint, so all retries will include the destination index
creation.
Fix#105683
Relate #104146
