Commit graph

337 commits

Author SHA1 Message Date
Albert Zaharovits
f4aaa20f28
Add support for the type parameter to the Query API Key API (#103695)
This adds support for the type parameter to the Query API key API.
The type for an API Key can currently be either rest or cross_cluster.

Relates: #101691
2024-01-11 10:53:50 +02:00
Chris Hegarty
f14d87be58
Add ES|QL async query api docs (#104054)
This commit adds detailed API docs for the ES|QL async apis, as well as moving the narrative to the ES|QL specific REST docs. Additionally, a few top-level lists are reflowed to reflect that there are now multiple APIs.
2024-01-09 09:17:02 +00:00
Johannes Fredén
f6a305afd7
Add expiration time to update api key api (#103453)
* Add expiration time to update api key api
2024-01-04 10:31:42 +01:00
Benjamin Trent
7fde357f3a
Improve docs around knn similarity search (#103158)
Adding equations to the docs around how to best calculate similarity & score. The similarity parameter for search was added in 8.8.

The max-inner-product mentions will be removed for all versions before 8.11 when backporting.

closes: https://github.com/elastic/elasticsearch/issues/102924
2023-12-11 14:56:16 -05:00
Jedr Blaszyk
73644bd650
[Connector API] Add docs for create connector API (#103177) 2023-12-11 13:03:06 +01:00
Andrei Stefan
6bba0efa78
ESQL: New telemetry for commands (#102937)
* Add enrich,mv_expand,show,row,from,drop,keep,rename commands to telemetry
2023-12-06 02:24:00 +02:00
Jan Kuipers
a67d5b8986
Inference telemetry (#102877)
* Empty infenrece usage wiring.

* Add fake data

* Fix NPE for secretSettings == null

* Real inference model stats

* New transport version

* Code polish

* Lint fixes

* Update docs/changelog/102877.yaml

* Update 102877.yaml

* Add inference to yamlRestTest

* Declare inference usage action as non-operator

* TransportInferenceUsageActionTests

* Lint fixes

* Replace map by ToXContentObject/Writeable

* Polish code

* AbstractWireSerializingTestCase<InferenceFeatureSetUsage.ModelStats>

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-12-04 14:27:54 +01:00
Johannes Fredén
6aad7f4e9f
Add support for invalidation timestamp in QueryApiKey (#102590)
This is a follow up PR from
https://github.com/elastic/elasticsearch/pull/102472. This adds the
ability to use `invalidation` timestamp as a valid [query
value](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-query-api-key.html#security-api-query-api-key-request-body)
in the QueryApiKey API.
2023-11-27 04:10:51 -05:00
Fabio Busatto
9f1875ee2c
[DOCS] Fix docs for user profiles (#102452)
* [DOCS] Fix docs for user profiles
2023-11-23 10:38:17 +01:00
Albert Zaharovits
bd10775b02
Grant API Key API with JWTs (#101904)
Introduces support for JWTs to the grant API Key API.
Callers can now pass-in a JWT in the request, like:
POST /_security/api_key/grant
{
  "grant_type": "access_token",
  "access_token" : "some.signed.JWT",
  "client_authentication": { // optional
    "scheme": "SharedSecret",
    "value": "ES-Client-Authentication header value after scheme"
  }
}
The JWT will be authenticated by a backing JWT realm and
a new API Key will be returned for the authenticated user.
2023-11-21 14:11:08 +02:00
Jan Kuipers
2e95b992b2
Add stats by model to the machine learning usage stats. (#101915)
* Add inference counts by NLP model to the machine learning usage stats.

* Update docs/changelog/101915.yaml

* Add inference_counts_by_model to yamlRestTest.

* Strip leading dot from internal model IDs.

* Add last access and task type to the stats by model.

* Change stats_by_model for map to list

* Simplify code.

* Fix style
2023-11-13 08:50:15 +01:00
Abdon Pijpelink
70128f5b74
[DOCS] Mark 'ignore_throttled' deprecated in all docs (#101838) 2023-11-07 13:03:49 +01:00
Daniel Mitterdorfer
a579504e11
Remove auto_configure privilege for profiling (#101026)
With this commit we remove the `auto_configure` privilege for the Fleet
service account that targets profiling-related indices. This privilege
was needed to automatically create indices and data streams in the past
but as this managed by the Elasticsearch plugin, there is no need to
grant this privilege to Fleet-managed components.
2023-10-23 13:14:14 +02:00
Abdon Pijpelink
8ac4ba751e
Restructure ES|QL docs (#100806)
* Break out 'Limitations' into separate page

* Add REST API docs

* Restructure commands, functions, and operators refs

* Add placeholder for getting started guide

* Group 'Syntax', 'Metafields', and 'MV fields' under 'Language'

* Add placeholder for Kibana page

* Add link from landing page

* Apply uniform formatting to ACOS, CASE, and DATE_PARSE function refs

* Reword default LIMIT

* Add support for COUNT(*)

* Move 'Commands' and 'Functions and operators' to individual pages

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-10-17 17:36:14 +02:00
Keith Massey
92ec9d6605
Add executed pipelines to bulk api response (#100031)
This change allows users to pass a new list_executed_pipelines parameter
to the bulk API, which results in an executed_pipelines list being returned.
2023-10-17 09:39:09 -05:00
Daniel Mitterdorfer
dfaec0dbf0
Enable Universal Profiling as Enterprise feature (#100333)
With this commit we ensure that Universal Profiling can only be used
with an Enterprise license.
2023-10-09 07:58:04 +02:00
István Zoltán Szabó
c2072fc4d3
[DOCS] Adds reference documentation for inference API (#99658)
* [DOCS] Creates documentation structure.

* [DOCS] Adds PUT inference API docs and part of GET inference API docs.

* [DOCS] Fixes complaining CI.

* [DOCS] Adds GET and DELETE API docs for inference API.

* [DOCS] Adds POST inference API docs.

* Apply suggestions from code review
2023-09-29 10:12:07 +02:00
Nhat Nguyen
ae17505557
Introduce authorization for enrich in ESQL (#99646)
This change introduces a new privilege monitor_enrich. Users are 
required to have this privilege in order to use the enrich functionality
in ESQL. Additionally, it eliminates the need to use the enrich_origin
when executing enrich lookups. The enrich_origin will only be used when
resolving enrich policies to prevent warnings when accessing system
indices directly.

Closes #98482
2023-09-27 12:45:39 -07:00
James Rodewig
daddca75d1
[DOCS] Fix Usage API snippet test (#99497) 2023-09-12 15:53:55 -04:00
James Rodewig
255c9a7f95
[DOCS] Move x-pack docs to docs/reference dir (#99209)
**Problem:**
For historical reasons, source files for the Elasticsearch Guide's security, watcher, and Logstash API docs are housed in the `x-pack/docs` directory. This can confuse new contributors who expect Elasticsearch Guide docs to be located in `docs/reference`. 

**Solution:**
- Move the security, watcher, and Logstash API doc source files to the `docs/reference` directory
- Update doc snippet tests to use security

Rel: https://github.com/elastic/platform-docs-team/issues/208
2023-09-12 14:53:41 -04:00
ChrisHegarty
2e4440ff1c Remove space in dissect feature list 2023-08-14 14:54:37 +01:00
ChrisHegarty
7cb310b9d0 Merge upstream 2023-08-14 09:55:02 +01:00
Yang Wang
d0f64941f0
Remove RCS 2.0 feature flag for beta release (#98307)
This PR removes the RCS 2.0 feature flag so that it is ready for beta
release.
2023-08-14 08:33:37 +10:00
elasticsearchmachine
9d1e4bcc2a Merge pull request ESQL-1546 from elastic/main
🤖 ESQL: Merge upstream
2023-08-07 13:27:50 -04:00
Kathleen DeRusso
4367c3f31c
Add query rulesets counts to enterprise search telemetry (#98071) 2023-08-07 10:38:12 -04:00
elasticsearchmachine
593849aea3 Merge pull request ESQL-1517 from elastic/main
🤖 ESQL: Merge upstream
2023-08-01 13:17:51 -04:00
Carlos Delgado
970685321f
Synonyms APIs - Synonyms Sets documentation (#98015) 2023-08-01 11:17:50 +02:00
elasticsearchmachine
f24b47af46 Merge pull request ESQL-1470 from elastic/main
🤖 ESQL: Merge upstream
2023-07-24 13:18:04 -04:00
Marantidis Kiriakos
ea42c2e076
boxplot support for transform 52189 (#96515) 2023-07-24 10:11:26 +02:00
elasticsearchmachine
121ec848e6 Merge pull request ESQL-1421 from elastic/main
🤖 ESQL: Merge upstream
2023-07-12 01:20:30 -04:00
Kathleen DeRusso
9ee32c4cc8
Add documentation for query rules CRUD APIs (#97524)
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-07-11 14:49:40 -04:00
elasticsearchmachine
a10223cc99 Merge pull request ESQL-1337 from elastic/main
🤖 ESQL: Merge upstream
2023-06-28 13:16:49 -04:00
Mary Gouseti
1abd51b167
Start with data stream lifecycle documentation (#95326) 2023-06-28 16:18:05 +03:00
elasticsearchmachine
2834c015cb Merge pull request ESQL-1219 from elastic/main
🤖 ESQL: Merge upstream
2023-06-01 13:16:52 -04:00
Mary Gouseti
8bcdaa9d62
Test fix: Update to the latest form the data_lifecycle telemetry section (#96484) 2023-06-01 15:33:26 +03:00
elasticsearchmachine
06004e2e9d Merge pull request ESQL-1207 from elastic/main
🤖 ESQL: Merge upstream
2023-05-30 13:18:56 -04:00
Keith Massey
17ff9f0fb1
Adding data_lifecycle to the _xpack/usage API (#96177)
This adds a `data_lifecycle` section to the _xpack/usage API, giving basic information about data lifecycles in the cluster. The data looks something like:
```
    "data_lifecycle": {
        "available": true,
        "enabled": true,
        "lifecycle": {
            "count": 1,
            "default_rollover_used": true,
            "retention": {
                "minimum_millis": 360000,
                "maximum_millis": 360000,
                "average_millis": 360000.0
            }
        }
    }
```
2023-05-30 13:48:03 +03:00
elasticsearchmachine
0ebbf0f60e Merge pull request ESQL-1188 from elastic/main
🤖 ESQL: Merge upstream
2023-05-26 13:18:02 -04:00
Sloane Perrault
9649ee0536
add analytics_collections to ent search usage test (#96385) 2023-05-26 12:10:54 -04:00
Andrei Stefan
9d4571a39e Docs fix 2023-05-25 02:17:02 +03:00
debadair
777598d602
[DOCS] Remove redirect pages (#88738)
* [DOCS] Remove manual redirects

* [DOCS] Removed refs to modules-discovery-hosts-providers

* [DOCS] Fixed broken internal refs

* Fixing bad cross links in ES book, and adding redirects.asciidoc[] back into docs/reference/index.asciidoc.

* Update docs/reference/search/point-in-time-api.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/setup/restart-cluster.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/sql/endpoints/translate.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/snapshot-restore/restore-snapshot.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update repository-azure.asciidoc

* Update node-tool.asciidoc

* Update repository-azure.asciidoc

---------

Co-authored-by: amyjtechwriter <61687663+amyjtechwriter@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Amy Jonsson <amy.jonsson@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2023-05-24 12:32:46 +01:00
Jim Ferenczi
154176b607
Fix _xpack/usage for enterprise_search in docs tests (#95845)
The previous fix (#95565) didn't work since the section was misplaced.
Note that this test runs only on snapshot build so I tested manually and the failure is now related to remote_clusters section missing.

Closes #95603
2023-05-05 10:11:13 +01:00
Jean-Pierre Matsumoto
87b8f1cf73
Bad ref to 'node_id' parameter in Task Mgt doc (#90380)
* Bad ref to 'node_id' parameter in Task Mgt doc

Here in documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/8.4/tasks.html#tasks-api-query-params

Name of parameter to filter node is `nodes`.

* Change 'node_id' into 'nodes' and add 'nodes' as a common param

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-05-03 11:55:29 +02:00
Albert Zaharovits
b90060538a
Fix _xpack_usage for enterprise_search (#95565)
Fixes #95564
2023-04-26 19:23:36 +03:00
Sloane Perrault
b57286aa0f
Add Search Application telemetry to xpack/usage (#95306)
* search application telemetry via xpack.usage

* impl EnterpriseSearchInfoTransportAction, fix test

* fix docs yaml tests and security operator tests

* set enabled & available with settings & license

* test count increases and decreases

* include xpack instead of xpack.usage specifically

* rename enterprise search usage serializing tests

* call action directly

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>

* handle `enabled = false` for entsearch usage

* return empty stats on error

* use an OriginSettingClient

---------

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>
2023-04-25 09:56:09 -04:00
Przemysław Witek
2b70165ffd
[Transform] Allow specifying destination index aliases in the Transform's dest config (#94943) 2023-04-17 15:08:43 +02:00
Mary Gouseti
99145bbe9c
Add new endpoints to configure data lifecycle on a data stream level. (#94590)
With PR we introduce CRUD endpoints which update/delete the data lifecycle on the data stream level. When this is updated it will apply at the next DLM run to all the backing indices that are managed by DLM.
2023-04-04 18:37:38 +02:00
Carlos Delgado
7a782031c4
Add Enterprise Search Module (#94381)
Create base module for ent-search with CRUD APIs to manage behavioral analytics and Search Application.

---------

Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
Co-authored-by: Aurélien FOUCRET <aurelien.foucret@gmail.com>
Co-authored-by: Kathleen DeRusso <63422879+kderusso@users.noreply.github.com>
Co-authored-by: Ioana Tagirta <ioanatia@users.noreply.github.com>
Co-authored-by: Joseph McElroy <joseph.mcelroy@elastic.co>
Co-authored-by: Aurelien FOUCRET <aurelien.foucret@elastic.co>
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2023-04-04 10:36:13 +01:00
Benjamin Trent
f23b906891
Add new similarity field to knn clause in _search (#94828)
This adds a new parameter to `knn` that allows filtering nearest neighbor results that are outside a given similarity.

`num_candidates` and `k` are still required as this controls the nearest-neighbor vector search accuracy and exploration. For each shard the query will search `num_candidates` and only keep those that are within the provided `similarity` boundary, and then finally reduce to only the global top `k` as normal.

For example, when using the `l2_norm` indexed similarity value, this could be considered a `radius` post-filter on `knn`.

relates to: https://github.com/elastic/elasticsearch/issues/84929 && https://github.com/elastic/elasticsearch/pull/93574
2023-03-28 15:29:01 -04:00
Andrei Dan
223385f887
Introduce a _lifecycle/explain API for data stream backing indices (#94621)
This adds an {index}/_lifecycle/explain API to retrieve information
about an index's status within its lifecycle.

The response looks like so:
```
"indices" : {
    ".ds-metrics-foo-2023.03.22-000001" : {
      "index" : ".ds-metrics-foo-2023.03.22-000001",
      "managed_by_dlm" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "rollover_date_millis" : 1679475564293,
      "time_since_rollover" : "121ms",
      "lifecycle" : { },
      "generation_time" : "121ms"
    },
    ".ds-metrics-foo-2023.03.22-000002" : {
      "index" : ".ds-metrics-foo-2023.03.22-000002",
      "managed_by_dlm" : true,
      "index_creation_date_millis" : 1679475564351,
      "time_since_index_creation" : "63ms",
      "lifecycle" : { }
    }
  }
}
```
2023-03-27 08:44:40 +01:00