Commit graph

76923 commits

Author SHA1 Message Date
Liam Thompson
d0f4966431
[DOCS] Add local dev setup instructions (#107913)
* [DOCS] Add local dev setup instructions

- Replace existing Run ES in Docker locally page, with simpler no-security local dev setup
- Move this file into Quickstart folder, along with existing quickstart guide
- Update self-managed instructions in Quickstart guide to use local dev approach
2024-05-07 18:10:48 +02:00
Moritz Mack
6e7afa04b4
Introduce constants for special ReservedClusterStateMetadata versions (#107995) 2024-05-07 18:00:59 +02:00
Ignacio Vera
b0fb722cc5
Small refactor of ESQL lucene operator (#108350)
This refactor removes duplicated code on all implementations of LuceneOperator.Factory by moving the factory from 
an interface to an abstract class.

In addition it introduces a getCheckedOutput() to centralize the catching of IOExceptions thrown by the different 
implementations of getOutput().
2024-05-07 17:44:55 +02:00
Lisa Cawley
a079cdc17d
[DOCS] Update transform and anomaly detection rule creation steps (#107975) 2024-05-07 07:52:45 -07:00
Tim Rühsen
e8cb8c67d5
[Profiling] Decrease storage size by reordering fields for the events data stream (#108341)
* [Profiling] Reorder events field sorting

* Bump template version

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-05-07 16:35:54 +02:00
Luigi Dell'Aquila
c19b7089d4
Mute RestEsqlIT testDoLogWithDebug (#108368) 2024-05-07 16:27:46 +02:00
Moritz Mack
c5d960875a
Verify nodes are ready after restart despite bad file settings (#108024) 2024-05-07 16:27:26 +02:00
Jonathan Buttner
9623e522c3
[ML] Inference document configurable settings (#108273)
* Starting to document various inference settings

* Finish settings

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: Max Hniebergall <137079448+maxhniebergall@users.noreply.github.com>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

* Update docs/reference/settings/inference-settings.asciidoc

Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>

---------

Co-authored-by: Max Hniebergall <137079448+maxhniebergall@users.noreply.github.com>
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
2024-05-07 10:19:08 -04:00
Moritz Mack
4c263e2ab4
CLI launcher / terminal improvements (#106470)
This removes getWriter() and getErrorWriter() from Terminal in order to prepare for using log4j in server-cli.
Additionally, ensure IO failures on the pump thread are handled once completed (closed) and
catch throwables in Command#main to print them (formatted) via Terminal.
2024-05-07 16:09:52 +02:00
Jonathan Buttner
4574f2a434
Adding manage_ml (#108262) 2024-05-07 09:46:53 -04:00
Tim Grein
6f8f1286c6
[Inference API] Remove explicit CoreMatchers prefix (#108355) 2024-05-07 15:06:44 +02:00
Jonathan Buttner
1a7c46a197
Filtering out rate limit service settings field (#108261) 2024-05-07 08:23:40 -04:00
Liam Thompson
9b7e9b5d59
[DOCS] ESQL goes GA (#108342) 2024-05-07 14:12:50 +02:00
Bogdan Pintea
b6ded320f0
Reenable JdbcCsvSpecIT inWithCompatibleDateTypes (#108017)
Reenable org.elasticsearch.xpack.sql.qa.single_node.JdbcCsvSpecIT
"test {filter.testInWithCompatibleDateTypes}", no longer reproducible.
2024-05-07 13:06:59 +02:00
Bogdan Pintea
de725aef80
Add docs clarifications on DATE_DIFF args (#108301)
This adds some clarifications on the time unit strings the function
takes as arguments, noting the differences between these and the time
span literals, as well as the abbreviations' source.
2024-05-07 12:59:01 +02:00
David Turner
26db24317d
Revert "Cluster state role mapper file settings service (#107886)" (#108346)
This reverts commit 391136c089.
2024-05-07 10:52:51 +01:00
Moritz Mack
fa196f72c1
Extended feature validations in REST tests and improved wording. (#108292) 2024-05-07 10:42:52 +02:00
Yang Wang
eba6a848df
Unmute Azure 3rd party tests (#108336)
Unmute Azure 3rd party tests (again) after re-generating credentials
following the updated guide.

Relates: #107928
Fixes: #107720
Fixes: #107502
2024-05-07 10:09:16 +02:00
Andrew Wilkins
ee85f74e55
apm-data: increase version for templates (#108340)
* apm-data: bump version of resources

* Update docs/changelog/108340.yaml
2024-05-07 16:07:06 +08:00
Moritz Mack
b71fc0c561
Migrate remaining usage of skip version in YAML specs to cluster_features (#108055) 2024-05-07 09:42:17 +02:00
Yang Wang
029624000b
Rename snapshottedCommits to acquiredIndexCommits (#107934)
As title says. Rename to be more accurate.
2024-05-07 01:09:44 -04:00
Yang Wang
2ef9e1ab24 [Test mute] AwaitsFix #108321
See title
2024-05-07 14:51:30 +10:00
Volodymyr Krasnikov
846b74f165
Fix coordination request and document rejection metrics (#107915)
* Fix coordination request rejection ratio metric

* change test

* fix metrics

* minor tuning

* Publish plain counters, do aggregation in kibana
2024-05-06 17:32:42 -07:00
Nik Everett
0117ea158e
ESQL: Add min and max to IntVector (#108216)
This adds `min` and `max` methods to `IntVector` which returns the
minimum and maximum values in the vector. It'll calculate that min and
max and cache it if required. This is useful because we use `IntVector`
as ordinals for things - and it's especially useful to know that the
`max` is less than the size of thing we're looking up.
2024-05-06 12:08:15 -04:00
Nhat Nguyen
e6b43a1709
Fix BlockHash DirectEncoder (#108283)
The DirectEncoder currently returns the incorrect value for the
positionCount() method, which should be the number of positions ready in
the current batch. We need to keep track of whether a position is loaded
via encodeNextBatch() and consumed via the read() method. However, we
can always return 1 for positionCount(), indicating that one position is
already loaded. Our tests failed to catch this because mv_ordering
wasn't enabled when generating test blocks, effectively disabling the
DirectEncoders.

Closes #108268
2024-05-06 09:05:55 -07:00
Nik Everett
d8d25ebdd7
ESQL: Log queries at debug level (#108257)
Previously we were logging all ESQL queries. That's a lot! Plus maybe
there's PII in there or something. Let's not do that unless you ask for
it. This changes the query logging to the `debug` log level you can
still get at these if you want them, but you don't have them by default.
you have to turn it on.
2024-05-06 11:19:02 -04:00
Slobodan Adamović
0378a77b43
Fix role descriptor's hashCode, equals and isEmpty implementations (#108255)
Added missing `remoteClusterPermissions` to `hashCode`, `equals` and
`isEmpty` implementations.

Resolves https://github.com/elastic/elasticsearch/issues/108253,
https://github.com/elastic/elasticsearch/issues/108285

Note: Marking as `>non-issue` since the PR that introduced
`remoteClusterPermissions` is not released yet.
2024-05-06 11:18:52 -04:00
Nik Everett
089fd7d7da
ESQL: Rework integration-only csv testing (#108313)
This reworks the integration-test-only csv testing for `metadata` to use
the `required_feature:` syntax instead of the `-IT_tests_only`
extension. This is a little more flexible and way nicer on the eyes.
2024-05-06 11:06:50 -04:00
Nhat Nguyen
3913b6f720
Fix random limit in AsyncOperatorTests (#108289)
Adjust the lower bound to include the case where the number of positions is zero.

Closes #107847
2024-05-06 07:54:27 -07:00
Bogdan Pintea
8c49dd2020
Make ip.ImplictCastingEqual test reliable (#108314)
This adds a sort to a test to make the results reliable.
2024-05-06 10:52:27 -04:00
Nikolaj Volgushev
31afff92f8
Invalidate cross cluster API key docs (#108297)
This PR documents privilege requirements for cross-cluster API key
invalidation, which were updated in
https://github.com/elastic/elasticsearch/pull/107411.
2024-05-06 10:02:14 -04:00
Ignacio Vera
c89de11e57
Optimise frequent item sets aggregation for single value fields (#108130)
Similar to #107832, this commit optimize requent item sets aggregation for single value fields.
2024-05-06 16:01:21 +02:00
Moritz Mack
609f5f8015
Extend timeout of assertBusy in DeprecationHttpIT for remaining places (#108229) 2024-05-06 15:57:08 +02:00
Liam Thompson
1be1110740
[DOCS] Clarify retriever is not API (#108295) 2024-05-06 15:52:25 +02:00
Ignacio Vera
e604a055e7
Optimize FieldData#toString methods for single value fields (#108291)
This commit create singleton instances when applicable in the FieldData#toString methods.
2024-05-06 15:33:35 +02:00
shainaraskas
58729edc30
add gatekeeper workaround (#108265) 2024-05-06 09:18:51 -04:00
shainaraskas
b84bd458b6
[DOCS] clarify that the repo location setting accepts only one value (#108267) 2024-05-06 09:18:17 -04:00
Pat Whelan
92feb1e5b8
[Transform] Document PIT performance issue (#108149)
Relates #80187
Relates #107969

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-05-06 08:46:00 -04:00
Bogdan Pintea
b26d7d3e14
Introduce an IP functions group (#108304)
This takes the CIDR_MATCH out of the operators group and adds it to a
new `IP functions` group.
The change also re-aranges the groups, grouping together the
type-specific functions and ordering them alphabetically.
2024-05-06 13:43:30 +02:00
Artem Prigoda
a049496aff
Fix SnapshotStatusApisIT#testInfiniteTimeout (#108294)
Make sure that the create-snapshot task completes on the master node before
we start checking the status of the snapshot

Resolve #107405

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-05-06 13:32:06 +02:00
István Zoltán Szabó
d90c0af3a6
[DOCS] Documents param for Health API. (#108296) 2024-05-06 12:23:17 +02:00
Yang Wang
bcf4297e89
Ensure necessary security context for s3 bulk deletions (#108280)
This PR moves the doPrivileged wrapper closer to the actual deletion
request to ensure the necesary security context is established at all
times. Also added a new repository setting to configure max size for s3
deleteObjects request.

Fixes: #108049
2024-05-06 06:02:09 -04:00
Julia Bardi
1fea0d760d
[Fleet] Add unprivileged mapping to .fleet-agents (#108293) 2024-05-06 11:23:34 +02:00
Alexander Spies
acad73bd87
ESQL: Do not use ql aggregate function (#108060)
Decouple our aggregate functions from the ql project by introducing own AggregateFunction/SpatialAggregateFunction classes for ESQL.

Additionally, also migrate the Project logical plan, which indirectly depends on AggregateFunction.
2024-05-06 11:14:41 +02:00
Ievgen Degtiarenko
0d5b80af34
Make translog fsync configurable (#108242) 2024-05-06 10:04:17 +02:00
elasticsearchmachine
844979414b
Forward port release notes for v8.13.3 (#108256) 2024-05-06 07:44:48 +02:00
Ed Savage
5a6f8cdb97
Muting failing testSerializationForCurrentVersion test (#108286) 2024-05-06 09:55:58 +12:00
Nhat Nguyen
16884a3bcd
Fix tsdb codec when doc-values spread in two blocks (#108276)
Currently, loading ordinals multiple times (after advanceExact) for 
documents with values spread across multiple blocks in the TSDB codec
will fail due to the absence of re-seeking for the ordinals block.

Doc-values of a document can spread across multiple blocks in two cases:
when it has more than 128 values or when it exceeds the remaining space
in the current block.
2024-05-04 13:54:56 -07:00
Chris Hegarty
7f90a98ed5
Update native vector provider to use unsigned int7 values only (#108243)
This commit updates the native vector provider to reflect that Lucene's scalar quantization is unsigned int7, with a range of values from 0 to 127 inclusive. Stride has been pushed down into native, to allow other platforms to more easily select there own stride length.

Previously the implementation supports signed int8. We might want the more general signed int8 implementation in the future, but for now unsigned int7 is sufficient, and allows to provide more efficient implementations on x64.
2024-05-04 10:42:55 +01:00
Fang Xing
4daac77e3b
[ES|QL] Add/Modify annotations for operators for better doc generation (#108220)
* annotation for operators
2024-05-03 22:59:51 -04:00