Currently when you try to index a document to a TSDS via an alias the
alias resolves to the latest backing index of the TSDS. This commit
delegates finding the write index to the original data stream the alias
points to.
Fixes#104189
There are two possibilities to retrieve flamegraph data:
* Via the native UI
* Via the APM integration
Depending on the scenario, different request parameters are set. While
we have improved the CO2 and cost calculation for the native UI, the
host id, which is required for an improved CO2 and cost calculation, is
not yet available for the APM integration.
So far we've not performed this calculation at all because there were
no associated host data for stacktraces. Consequently, we've returned
zero values in all cases. With this commit we associate "dummy" host
data so the CO2 and cost calculation falls back to default values. Once
a host id is available for that case as well, we will instead use the
improved calculations.
When hotspot encounters an error, it will emite a log file which can be
used for reproducing the error. This file is dumped to /tmp by default.
This commit configures the replay file to be alongside the hs_err file.
client.type existed from the days of the node client existing alongside
the java client. Since the node client no longer exists, it no longer
serves a purpose, and is already ignored. Yet the setting still exists.
This commit deprecates the client.type node setting.
This commit adjusts compile tasks to explicitly provision a Java
toolchain for the Java minimum runtime version. By doing so the Java
used by Gradle may be upgraded without the possibility of causing
spurious warnings from javac which could fail the build, such as when
new warnings are added in later JDK versions.
Previously only aggregate functions (max/sum/etc..) were allowed inside
the stats command. This PR allows expressions involving one or multiple
aggregates to be used, such as:
stats x = avg(salary % 3) + max(emp_no),
y = min(emp_no / 3) + 10 - median(salary)
by z = languages % 2
Improve verifier to not allow scalar functions over grouping for now
I have seen scenarios in which field-caps return information from
outdated shards. While this is probably acceptable for most cases, ESQL
query planning relies on field-caps, potentially leading to missing
data. The reason for this is that we don't check readAllowed when not
acquiring a searcher for cases without a filter. I don't expect too much
penalty in terms of performance with this change, but it helps avoid a
subtle issue for ESQL.
Closes#104809
x-content embeds its jackson implementation inside its jar. This commit
formalizes the setup for this embedding with a gradle plugin so that it
can be reused by other libs.
* Bounding queue capacity and allowing it to be adjusted
* Adding some deadlock tests
* Adding some more tests for the request executor and queue logic
* Adding debug message
* Retaining overflow items after capacity change
* Addressing feedback
This PR updates the ingest service to detect if a failed ingest document was bound for a data stream configured
with a failure store, and in that event, restores the document to its original state, transforms it with its failure
information, and redirects it to the failure store for the data stream it was originally targeting.
We can avoid building composite byte buf instances on the transport
layer (they have quite a bit of overhead and make heap dumps more
complicated to read). There's no need to add another round of references
to the BytesReference components here. Just write these out as they come
in. This would allow for some efficiency improving follow-ups where we
can essentially release the pages that have passed the write pipeline.
To avoid having this explode the size of the queue for writes per
channel, I moved that to a linked list. The slowdown from a linked list
is irrelevant I believe. Mostly the queue is empty so it doesn't matter
or if it isn't empty, operations other than dequeuing are much more
important to performance in this logic anyway (+ Netty internally uses a
LL down the line anyway).
I would regard this as step-1 in making the serialisation here more lazy
like on the REST layer to avoid copying bytes to the outbound buffer
that we already have as `byte[]`.
This adds two new vector index types: - flat - int8_flat
Both store the vectors in a flat space and search is brute-force over
the vectors in the index. For the regular `flat` index, this can be
considered syntactic sugar that allows `knn` queries without having to
put indices within HNSW.
For `int8_flat`, this allows float vectors to be stored in a flat
manner, but also automatically quantized.
Yaml tests executed in mixed clusters need to skip clusters that run 8.12.x or earlier versions. The yaml tests assume hashing based time series ids, but if a node in the test cluster is on 8.12.x or earlier, then it can happen pre hashing time series ids are used (depending on the version of the elected master node).
Tsdb yaml tests that assert the _id or _tsid should be skipped if there are 8.12.x nodes in the mixed test cluster.
Rolling upgrade or full upgrade tests are better for assertion the _id or _tsid in this case, because tests are setup prior to upgrade and pre 8.12.x logic can be asserted in a more controlled way.
Closes#105129
* Transform: return results in order
Currently, when Transform searches over aggregations, it stores the
results in an unordered HashMap. This potentially rearranges search
results.
For example, if a user specifies an order in a search request, the
search response is in that order. But if the search request is
embedded in a Transform request, then Transform response will not
preserve the order and the result will look different.
With this change, Transform will always preserve the order of the
search response. A search embedded in a Transform should behave as
an unembedded search.
Closes#104847
We've seen cases of OOM errors in the test runner process, which occur
when we convert a response to a JSON string and then parse it. We can
directly parse from its input stream to avoid these OOM errors.
* [DOCS] Add link to on-prem install tutorial
* Move link to bottom of packages section
* Rearrange things according to suggestions
* Add another link on the 'Install Elasticsearch with RPM' page
Created new Exception QueryPhaseTimeoutException, which returns RestStatus 504.
We considered the 408 status code, but decided that the official spec for that status doesn't
match this scenario, so 504 was considered the closest fit.
This change attempts to fix testListTasksWaitForCompletion by setting bariers to
verify task started on all nodes and had a chance to list all running tasks
before canceling the TEST_TASK
I tried to move this logic to use pooled SearchHit instances but it
turned out to be too complicated in one go, so simplifying obvious spots
here:
* ReversePayload is pointless, it just reverses the original payload.
* a number of listeners were unnecessary and could be expressed inline
much clearer
* moved some "unpooling" to later in the logic to make objects live for
shorter and have fewer references to them
Today the docs on balancing settings describe what the settings all do
but offer little guidance about how to configure them. This commit adds
some extra detail to avoid some common misunderstandings and reorders
the docs a little so that more commonly-adjusted settings are mentioned
earlier.
We can speed up the slice operation quite a bit by speeding up skip for
the common case and passing the delegete as the basis for the stream (this neatly avoids
a multi-morphic call to `length` on the bytes reference).
Also, while we're at it, we can speed up the common-case read operation
the same way.
This changes all of our ingest-related builders other than BulkRequestBuilder
(which was already changed in #104927) to inherit from ActionRequestLazyBuilder
(added in #104927) rather than ActionRequestBuilder. This means that the
requests will not be created until the builder's request() method is called, making
upcoming ref counting work much more feasible.
It's in the title. We already have the size here and allocating the
iterator isn't free. In fact it's 10G of allocations during http_logs
indexing that we can avoid with a simple condition.
This PR adds missing validation before reloading JWT shared secret settings.
The shared secret setting must always be configured when the client
authentication type is `shared_secret` and omitted when it's `none`.
The counted-terms aggregation is defined in its own plugin. When other
plugins (such as the profiling plugin) want to use this aggregation,
this leads to class loader issues, such as that the aggregation class is
not recognized. By moving just the aggregation code itself to the server
module but keeping everything else (including registration) in the
`mapper-counted-keyword` module, we can use the counted-terms
aggregation also from other plugins.
