* Reapply "[Build] Do not invalidate configuration cache when branch is switched (#118894)" (#119300) (#119325)
* Reapply "[Build] Do not invalidate configuration cache when branch is switched (#118894)" (#119300)
The original PR (#118894) has broken serverless.
* Fix gitinfo plugin for serverless usage
* Update buildscan git revision reference
(cherry picked from commit 5278159987)
# Conflicts:
# build-conventions/src/main/java/org/elasticsearch/gradle/internal/conventions/PublishPlugin.java
* Fix merge conflict
In 8.x we need to have bwc back to when before `element_type: byte`
existed. To prevent loss of coverage for past versions & telemetry, here
I move the index creation around so that we only create with `byte` when
we have the more recent telemetry changes (and thus also `byte`
elements).
closes: https://github.com/elastic/elasticsearch/issues/114556
* Adding new bbq index types behind a feature flag (#114439)
new index types of bbq_hnsw and bbq_flat which utilize the better binary quantization formats. A 32x reduction in memory, with nice recall properties.
(cherry picked from commit 6c752abc23)
* spotless
* Guard second doc parsing pass with index setting (#114649)
* Guard second doc parsing pass with index setting
* add test
* updates
* updates
* merge
(cherry picked from commit 98e0a4e953)
* Update 21_synthetic_source_stored.yml
**Description:**
This PR addresses the issue described in [#114402](https://github.com/elastic/elasticsearch/issues/114402), where the `synthetic_source` feature does not correctly handle the `bit` type in `dense_vector` fields when `index` is set to `false`. The root cause of the issue was that the `bit` type was not properly accounted for, leading to an array that is 8 times the size of the actual `dims` value of docvalue. This mismatch will causes an array out-of-bounds exception when reconstructing the document.
**Changes:**
- Adjusted the `synthetic_source` logic to correctly handle the `bit` type by ensuring the array size accounts for the 8x difference in dimensions.
- Added yaml test to cover the `bit` type scenario in `dense_vector` fields with `index` set to `false`.
**Related Issues:**
- Closes [#114402](https://github.com/elastic/elasticsearch/issues/114402)
- Introduced in [#110059](https://github.com/elastic/elasticsearch/pull/110059)
Co-authored-by: Rassyan <yjkhngds@gmail.com>
* Add a query rules tester API call
* Update docs/changelog/114168.yaml
* Wrap client call in async with origin
* Remove unused param
* PR feedback
* Remove redundant test
* CI workaround - add ent-search as ml dependency so it can find node features
API for `/_inference/{task_type}/{inference_id}/_stream` and `/_inference/{inference_id}/_stream`
Request is `application/json`
Response is `text/event-stream`
This adds some more counts for dense_vector field mapping stats. This
allows for seeing the number of mappings with a given element type,
similarity, or index type.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Add remote cluster stats to _cluster/stats
* Implement remote cluster stats polling
* Add docs for the include_remotes part
(cherry picked from commit b26d81c713)
Deprecate to, from, include_lower, include_upper range query params.
These params have been removed from our documentation in v. 0.90.4 (d6ecdecc19),
but did not got through deprecation cycle.
These params to be removed in v9.0.
Related to #81276Closes#48538
This implements the `parseBytesRef` method for the `_ts_routing_hash` field so we
can parse the values generated by the companion `format` method.
We parse the values when fetching them from the source when the field is used
as a `sort` paired with `search_after`.
Before this change a sort by and search_after `_ts_routing_hash` would yield
an `UnsupportedOperationException`
(cherry picked from commit 4e5e870370)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
Here we introduce a new index-level setting, `ignore_above`, similar to what we have
for `ignore_malformed`. The setting will apply to all `keyword`, `wildcard` and `flattened`
fields. Each field mapping will still be allowed to override the index-level setting using a
mapping-level `ignore_above` value.
(cherry picked from commit 208a1fe571)
* Add support for multi-value dimensions (#112645)
Closes https://github.com/elastic/elasticsearch/issues/110387
Having this in now affords us not having to introduce version checks in
the ES exporter later. We can simply use the same serialization logic
for metric attributes as we do for other signals. This also enables us
to properly map `*.ip` fields to the ip field type as ip fields
containing a list of IPs are not converted to a comma-separated list.
(cherry picked from commit 8d223cbf7a)
# Conflicts:
# server/src/main/java/org/elasticsearch/index/mapper/TimeSeriesIdFieldMapper.java
* Remove skip test for 8.x
This was just needed for 8.x to 9.0 compatibility tests
* Fix `ignore_above` for flattened fields (#112944)
Flattened fields do no actually use `ignore_above`. When retrieving source, `ignore_above` behaves
as expected. Anyway, it is ignored when fetching field values through the fields API. In that case
values are returned no matter the `ignore_above` setting. Here we implement the filtering logic
in the `ValueFecther` implementation of `RootFlattenedFieldType`.
(cherry picked from commit 91674f85a0)
* Fix
---------
Co-authored-by: Salvatore Campagna <93581129+salvatore-campagna@users.noreply.github.com>
The addition of the logger requires several updates to tests to deal with the possible warning, or muting if there is not way to specify an allowed (but not mandatory) warning
Introduces per-field param `synthetic_source_keep` that overrides the
behavior for keeping the field source in synthetic source mode: -
`none` : no source is stored - `arrays`: the incoming source is
recorded as-is for arrays of a given field - `all`: the incoming source
is recorded as is for both singleton and array values of a given field
Related to #112012
* Retrieve the source for objects and arrays within arrays in a separate parsing phase (#113027)
In synthetic source, storing array elements to `_ignored_source` may
hide other, regular elements from showing up during source synthesizing.
This is due to contents from `_ignored_source` taking precedence over
matching fields from regular source loading.
To avoid this, arrays are pre-emptively tracked and marked for source
storing, if any of their elements needs to store its source. A second
doc parsing phase is introduced that checks for fields missing values
and records their source, while skipping objects and arrays that don't
contain any such fields.
Fixes#112374
(cherry picked from commit 4ff4384550)
# Conflicts:
# muted-tests.yml
* merge
* fix compilation error, jdk-21 function
* Deprecate dot-prefixed indices and composable template index patterns (#112571)
This commit adds a module emitting a deprecation warning when a
dot-prefixed index is manually or automatically created, or when a
composable index template with an index pattern that uses a dot-prefix
is created. This pattern warns that in the future these indices will not
be allowed. In a future breaking change (10.0.0 maybe?) the deprecation
can then be changed to an exception.
These deprecations are only displayed when a non-operator user is using
the API (one that does not set the `X-elastic-product-origin` header).
* Attempt to fix build for V7 gradle stuff
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
