Commit graph

12605 commits

Author SHA1 Message Date
Nik Everett
c6fe3c3efe
ESQL: Improve syntax for LOOKUP tables (#109489)
Replace the syntax for `tables` with something a little more natural.

Now it is:

```
$ curl -uelastic:password -HContent-Type:application/json -XPOST \
    'localhost:9200/_query?error_trace&pretty&format=txt' \
-d'{
    "query": "ROW a=1::LONG | LOOKUP t ON a",
    "tables": {
        "t": {
            "a": {"long":     [    1,     4,     2]},
            "v1": {"integer": [   10,    11,    12]},
            "v2": {"keyword": ["cat", "dog", "wow"]}
        }
    }
}'
      v1       |      v2       |       a
---------------+---------------+---------------
10             |cat            |1
```
2024-06-11 23:26:04 +10:00
Liam Thompson
d6fb5cfbe6
[DOCS] Expand context about xpack.security.enabled setting (#109575) 2024-06-11 14:59:40 +02:00
Benjamin Trent
29288d6590 Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-11 06:54:23 -04:00
Liam Thompson
0480c1acba
[DOCS] Remove ESQL demo env link from 8.14+ (#109562) 2024-06-11 11:24:52 +02:00
Carlos Delgado
d975997a3a
Add semantic-text warning about inference endpoints removal (#109561) 2024-06-11 18:33:25 +10:00
Oleksandr Kolomiiets
a9f31bd2aa
Support synthetic source for date fields when ignore_malformed is used (#109410) 2024-06-10 10:26:31 -07:00
Karen Metts
f4d87e0f25
[DOCS] Add note that Logstash sets up data streams (#109502) 2024-06-10 12:24:23 -04:00
Jean-Fabrice Bobo
a9bc30d66e
FIx misleading repository-s3 type (#109347)
in 8.x, `repository-s3` type has been replaced by `s3` type. Fixing
remaining reference to `repository-s3` in the documentation.
2024-06-11 01:51:55 +10:00
Alexander Reelsen
7cba6c8c16
Docs: Fix available update by query operations (#109486) 2024-06-10 15:57:56 +02:00
Oleksandr Kolomiiets
eedc2b9354
Fix typo in TSDB documentation (#109504) 2024-06-10 06:24:05 -07:00
Carlos Delgado
4d3f9f2fb9
Fix RRF example for semantic query (#109516)
Follow up to https://github.com/elastic/elasticsearch/pull/109433, fix
appropriately this time the semantic query example with RRF.
2024-06-10 17:59:13 +10:00
David Turner
683245e41e
Detect long-running tasks on network threads (#109204)
This commit introduces a watchdog timer to monitor for long-running
tasks on network threads. If a network thread is active and has not made
progress for two consecutive ticks of the timer then the watchdog logs a
warning and a thread dump.
2024-06-10 17:47:40 +10:00
Luigi Dell'Aquila
3d0c65d0c5
ES|QL: add tests for COALESCE() function on VERSION type (#109468) 2024-06-07 18:01:42 +02:00
Benjamin Trent
a5fbfe81b2 Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-07 07:24:43 -04:00
Panagiotis Bailis
1c3b3d8f11
Adding support for explain in rrf (#108682) 2024-06-07 11:09:06 +03:00
Nik Everett
7916e6a231
ESQL: Implement LOOKUP, an "inline" enrich (#107987)
This adds support for `LOOKUP`, a command that implements a sort of
inline `ENRICH`, using data that is passed in the request:

```
$ curl -uelastic:password -HContent-Type:application/json -XPOST \
    'localhost:9200/_query?error_trace&pretty&format=txt' \
-d'{
    "query": "ROW a=1::LONG | LOOKUP t ON a",
    "tables": {
        "t": {
            "a:long":     [    1,     4,     2],
            "v1:integer": [   10,    11,    12],
            "v2:keyword": ["cat", "dog", "wow"]
        }
    },
    "version": "2024.04.01"
}'
      v1       |      v2       |       a       
---------------+---------------+---------------
10             |cat            |1
```

This required these PRs: * #107624 * #107634 * #107701 * #107762 *
#107923 * #107894 * #107982 * #108012 * #108020 * #108169 * #108191 *
#108334 * #108482 * #108696 * #109040 * #109045

Closes #107306
2024-06-07 11:38:51 +10:00
Benjamin Trent
d3561f9cf3 Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-06 18:22:08 -04:00
István Zoltán Szabó
d89dae2a32
[DOCS] Modifies semantic search-related docs to refer to the semantic_text workflow (#109418)
Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
2024-06-06 16:45:46 +02:00
Carlos Delgado
d4d5d9320c
Fix semantic_text retrievers docs example (#109433) 2024-06-06 16:31:12 +02:00
elasticsearchmachine
f8291f8e83
Forward port release notes for v8.14.0 (#109403) 2024-06-05 14:52:38 -04:00
Lorenzo Verardo
02a6c831e1
Limit the value in prefix query (#108537)
Reuse the setting index.max_regex_length for the max length in a prefix query.

Closes #108486
2024-06-05 14:51:07 -04:00
Benjamin Trent
ac53d6020b Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-05 12:38:23 -04:00
Mark J. Hoy
80a22ec046
[Inference API] Add Docs for Mistral Embedding Support for the Inference API (#109319)
* Initial docs for put-inference for Mistral

* adds mistral embeddings to tutorial; add changelog

* update mistral text and dimensions

* fix mistral spelling error

* fix azure AI studio; fix Mistral label

* fix auto-formatted items

* change pipeline button back to azure openai

* put proper Azure AI Studio include in

* fix missing azure-openai; fix huggingface hidden

* fix mistral tab for reindex

* re-add Mistral service settings to put inference
2024-06-05 11:23:29 -04:00
Tim Grein
dc13b75656
[Inference API] Add text_embedding task type to Google AI Studio docs (#109307) 2024-06-05 15:35:31 +02:00
Jonathan Buttner
fdb5058b13
[ML] Inference API rate limit queuing logic refactor (#107706)
* Adding new executor

* Adding in queuing logic

* working tests

* Added cleanup task

* Update docs/changelog/107706.yaml

* Updating yml

* deregistering callbacks for settings changes

* Cleaning up code

* Update docs/changelog/107706.yaml

* Fixing rate limit settings bug and only sleeping least amount

* Removing debug logging

* Removing commented code

* Renaming feedback

* fixing tests

* Updating docs and validation

* Fixing source blocks

* Adjusting cancel logic

* Reformatting ascii

* Addressing feedback

* adding rate limiting for google embeddings and mistral

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-06-05 08:25:25 -04:00
Andrei Verdes
fbe2b1ab24
update package-security.asciidoc to include link to reset password (#109314) 2024-06-05 11:38:52 +02:00
Benjamin Trent
013e0c7cc6 Merge branch 'main' into lucene_snapshot_9_11 2024-06-04 18:08:29 -04:00
Parker Timmins
bb3ff8e924
ESQL: add REPEAT string function (#109220)
Add support for the string manipulation function REPEAT(string, number). This function concatenates the string argument with itself the specified number of times. If number is 0 an empty string is returned. If number is less than 0, null is returned and a warning is logged. If number is less than 0 and is a constant, the query will fail without executing.
2024-06-04 16:32:43 -05:00
Luigi Dell'Aquila
5f6e8f687b
ES|QL: add MV_APPEND function (#107001)
Adding `MV_APPEND(value1, value2)` function, that appends two values
creating a single multi-value. If one or both the inputs are
multi-values, the result is the concatenation of all the values, eg.

```
MV_APPEND([a, b], [c, d]) -> [a, b, c, d]
```

~I think for this specific case it makes sense to consider `null` values
as empty arrays, so that~ ~MV_APPEND(value, null) -> value~ ~It is
pretty uncommon for ESQL (all the other functions, apart from
`COALESCE`, short-circuit to `null` when one of the values is null), so
let's discuss this behavior.~

[EDIT] considering the feedback from Andrei, I changed this logic and
made it consistent with the other functions: now if one of the
parameters is null, the function returns null
2024-06-05 03:42:29 +10:00
Luigi Dell'Aquila
21952c7e36
ES|QL: add geo tests for mv_dedupe (#109342)
Adding more unit tests for MV_DEDUPE function, covering geo_point,
geo_shape, cartesian_point and cartesian_shape. This also adds docs for
Kibana.

Fixes https://github.com/elastic/elasticsearch/issues/108982
2024-06-05 03:33:14 +10:00
john-wagster
dd83b5b8d0 Multivalue Sparse Vector Support (#109007)
Updated LuceneDocument to take advantage of looking up feature values on existing features and selecting the max when parsing multi-value sparse vectors
2024-06-04 12:50:58 -04:00
Benjamin Trent
cf84416fc5 Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-04 12:50:52 -04:00
Joe Gallo
d9941f6285
Ingest geoip new databases release highlight (#109355) 2024-06-04 12:48:19 -04:00
Jedr Blaszyk
04124165ec
[Docs][Connector API] Add documentation for update featueres endpoint (#109346) 2024-06-04 16:38:18 +02:00
Nhat Nguyen
3a4dfb0066
Add remove index setting command (#109276)
The new subcommand elasticsearch-node remove-index-settings can be used 
to remove index settings from the cluster state in case where it
contains incompatible index settings that prevent the cluster from
forming. This tool can cause data loss and its use should be your last
resort.

Relates #96075
2024-06-03 18:23:13 -07:00
Liam Thompson
2268e383e8
[DOCS][ESQL][8.14] Add API key based security model info for ESQL CCS (#109155)
Co-authored-by: Jake Landis <jake.landis@elastic.co>
2024-06-03 18:44:33 +02:00
István Zoltán Szabó
d1da412a3e
[DOCS] Expands DELETE inference API docs (#109282)
* [DOCS] Expands DELETE inference API docs.

* [DOCS] Adds discrete flag.
2024-06-03 17:32:31 +02:00
shainaraskas
d12a318304
change from string to array in example (#109263) 2024-06-03 11:29:00 -04:00
Benjamin Trent
9cd123d6cc Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-02 16:46:19 -04:00
István Zoltán Szabó
95ce898436
[DOCS] Adds docs to semantic text (#108311)
Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
Co-authored-by: Mike Pellegrini <mike.pellegrini@elastic.co>
Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>
2024-05-31 16:56:07 +02:00
Martijn van Groningen
29ccca3ddf
Adjust index.mapping.dimension_fields.limit index setting default in docs. (#109091) 2024-05-31 10:52:31 +02:00
Jake Landis
28588e0847
update release notes (#109127) 2024-05-30 11:37:54 -05:00
Salvatore Campagna
f11ef44084
Introduce logs index mode as Tech Preview (#108896)
This PR introduces a new index mode, `logs`, which enables usage of LogsDB in Elasticsearch.
As a result of adopting the `logs` index mode, default index sorting is applied using the hostname
and @timestamp fields. Users are allowed, anyway, to override index sort settings.
By default, it will also use synthetic source and the same codecs used by TSDB.

Note: the logs index mode is a Tech Preview feature.
2024-05-30 14:26:48 +02:00
elasticsearchmachine
7b5925f4b6 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-30 10:01:52 +00:00
George Wallace
ceb941f48c
Removing server from the version compatibility table (#109168)
* Removing server from the version compatibility table

* fixed typo

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-05-29 11:57:46 -06:00
István Zoltán Szabó
1413c67d99
[DOCS] Amends inference reference docs and tutorials (#109159)
* [DOCS] Fixes inference tutorial widgets.

* [DOCS] Adds link to notebooks, rearranges sections in PUT inference API docs.
2024-05-29 17:43:10 +02:00
elasticsearchmachine
0f8c375bc8
Forward port release notes for v8.13.4 (#108527) 2024-05-29 11:22:04 -04:00
Iván Cea Fontenla
f16f71e2a2
ESQL: Add ip_prefix function (#109070)
Added ESQL function to get the prefix of an IP. It works now with both
IPv4 and IPv6. For users planning to use it with mixed IPs, we may need
to add a function like "is_ipv4()" first.

**About the skipped test:** There's currently a "bug" in the
evaluators//functions that return null. Evaluators can't handle them.
We'll work on support for that in another PR. It affects other
functions, like `substring()`. In this function, however, it only
affects in "wrong" cases (Like an invalid prefix), so it has no impact.

Fixes https://github.com/elastic/elasticsearch/issues/99064
2024-05-29 10:23:45 -04:00
István Zoltán Szabó
1e58f3a485
[DOCS] Fixes sparse vector query docs. (#109153) 2024-05-29 14:56:59 +02:00
Liam Thompson
b6241711ef
[DOCS] Update CCS matrix for 8.14 (#109142) 2024-05-29 14:00:19 +02:00