Commit graph

325 commits

Author SHA1 Message Date
Daniel Mitterdorfer
a579504e11
Remove auto_configure privilege for profiling (#101026)
With this commit we remove the `auto_configure` privilege for the Fleet
service account that targets profiling-related indices. This privilege
was needed to automatically create indices and data streams in the past
but as this managed by the Elasticsearch plugin, there is no need to
grant this privilege to Fleet-managed components.
2023-10-23 13:14:14 +02:00
Abdon Pijpelink
8ac4ba751e
Restructure ES|QL docs (#100806)
* Break out 'Limitations' into separate page

* Add REST API docs

* Restructure commands, functions, and operators refs

* Add placeholder for getting started guide

* Group 'Syntax', 'Metafields', and 'MV fields' under 'Language'

* Add placeholder for Kibana page

* Add link from landing page

* Apply uniform formatting to ACOS, CASE, and DATE_PARSE function refs

* Reword default LIMIT

* Add support for COUNT(*)

* Move 'Commands' and 'Functions and operators' to individual pages

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-10-17 17:36:14 +02:00
Keith Massey
92ec9d6605
Add executed pipelines to bulk api response (#100031)
This change allows users to pass a new list_executed_pipelines parameter
to the bulk API, which results in an executed_pipelines list being returned.
2023-10-17 09:39:09 -05:00
Daniel Mitterdorfer
dfaec0dbf0
Enable Universal Profiling as Enterprise feature (#100333)
With this commit we ensure that Universal Profiling can only be used
with an Enterprise license.
2023-10-09 07:58:04 +02:00
István Zoltán Szabó
c2072fc4d3
[DOCS] Adds reference documentation for inference API (#99658)
* [DOCS] Creates documentation structure.

* [DOCS] Adds PUT inference API docs and part of GET inference API docs.

* [DOCS] Fixes complaining CI.

* [DOCS] Adds GET and DELETE API docs for inference API.

* [DOCS] Adds POST inference API docs.

* Apply suggestions from code review
2023-09-29 10:12:07 +02:00
Nhat Nguyen
ae17505557
Introduce authorization for enrich in ESQL (#99646)
This change introduces a new privilege monitor_enrich. Users are 
required to have this privilege in order to use the enrich functionality
in ESQL. Additionally, it eliminates the need to use the enrich_origin
when executing enrich lookups. The enrich_origin will only be used when
resolving enrich policies to prevent warnings when accessing system
indices directly.

Closes #98482
2023-09-27 12:45:39 -07:00
James Rodewig
daddca75d1
[DOCS] Fix Usage API snippet test (#99497) 2023-09-12 15:53:55 -04:00
James Rodewig
255c9a7f95
[DOCS] Move x-pack docs to docs/reference dir (#99209)
**Problem:**
For historical reasons, source files for the Elasticsearch Guide's security, watcher, and Logstash API docs are housed in the `x-pack/docs` directory. This can confuse new contributors who expect Elasticsearch Guide docs to be located in `docs/reference`. 

**Solution:**
- Move the security, watcher, and Logstash API doc source files to the `docs/reference` directory
- Update doc snippet tests to use security

Rel: https://github.com/elastic/platform-docs-team/issues/208
2023-09-12 14:53:41 -04:00
ChrisHegarty
2e4440ff1c Remove space in dissect feature list 2023-08-14 14:54:37 +01:00
ChrisHegarty
7cb310b9d0 Merge upstream 2023-08-14 09:55:02 +01:00
Yang Wang
d0f64941f0
Remove RCS 2.0 feature flag for beta release (#98307)
This PR removes the RCS 2.0 feature flag so that it is ready for beta
release.
2023-08-14 08:33:37 +10:00
elasticsearchmachine
9d1e4bcc2a Merge pull request ESQL-1546 from elastic/main
🤖 ESQL: Merge upstream
2023-08-07 13:27:50 -04:00
Kathleen DeRusso
4367c3f31c
Add query rulesets counts to enterprise search telemetry (#98071) 2023-08-07 10:38:12 -04:00
elasticsearchmachine
593849aea3 Merge pull request ESQL-1517 from elastic/main
🤖 ESQL: Merge upstream
2023-08-01 13:17:51 -04:00
Carlos Delgado
970685321f
Synonyms APIs - Synonyms Sets documentation (#98015) 2023-08-01 11:17:50 +02:00
elasticsearchmachine
f24b47af46 Merge pull request ESQL-1470 from elastic/main
🤖 ESQL: Merge upstream
2023-07-24 13:18:04 -04:00
Marantidis Kiriakos
ea42c2e076
boxplot support for transform 52189 (#96515) 2023-07-24 10:11:26 +02:00
elasticsearchmachine
121ec848e6 Merge pull request ESQL-1421 from elastic/main
🤖 ESQL: Merge upstream
2023-07-12 01:20:30 -04:00
Kathleen DeRusso
9ee32c4cc8
Add documentation for query rules CRUD APIs (#97524)
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-07-11 14:49:40 -04:00
elasticsearchmachine
a10223cc99 Merge pull request ESQL-1337 from elastic/main
🤖 ESQL: Merge upstream
2023-06-28 13:16:49 -04:00
Mary Gouseti
1abd51b167
Start with data stream lifecycle documentation (#95326) 2023-06-28 16:18:05 +03:00
elasticsearchmachine
2834c015cb Merge pull request ESQL-1219 from elastic/main
🤖 ESQL: Merge upstream
2023-06-01 13:16:52 -04:00
Mary Gouseti
8bcdaa9d62
Test fix: Update to the latest form the data_lifecycle telemetry section (#96484) 2023-06-01 15:33:26 +03:00
elasticsearchmachine
06004e2e9d Merge pull request ESQL-1207 from elastic/main
🤖 ESQL: Merge upstream
2023-05-30 13:18:56 -04:00
Keith Massey
17ff9f0fb1
Adding data_lifecycle to the _xpack/usage API (#96177)
This adds a `data_lifecycle` section to the _xpack/usage API, giving basic information about data lifecycles in the cluster. The data looks something like:
```
    "data_lifecycle": {
        "available": true,
        "enabled": true,
        "lifecycle": {
            "count": 1,
            "default_rollover_used": true,
            "retention": {
                "minimum_millis": 360000,
                "maximum_millis": 360000,
                "average_millis": 360000.0
            }
        }
    }
```
2023-05-30 13:48:03 +03:00
elasticsearchmachine
0ebbf0f60e Merge pull request ESQL-1188 from elastic/main
🤖 ESQL: Merge upstream
2023-05-26 13:18:02 -04:00
Sloane Perrault
9649ee0536
add analytics_collections to ent search usage test (#96385) 2023-05-26 12:10:54 -04:00
Andrei Stefan
9d4571a39e Docs fix 2023-05-25 02:17:02 +03:00
debadair
777598d602
[DOCS] Remove redirect pages (#88738)
* [DOCS] Remove manual redirects

* [DOCS] Removed refs to modules-discovery-hosts-providers

* [DOCS] Fixed broken internal refs

* Fixing bad cross links in ES book, and adding redirects.asciidoc[] back into docs/reference/index.asciidoc.

* Update docs/reference/search/point-in-time-api.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/setup/restart-cluster.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/sql/endpoints/translate.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/snapshot-restore/restore-snapshot.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update repository-azure.asciidoc

* Update node-tool.asciidoc

* Update repository-azure.asciidoc

---------

Co-authored-by: amyjtechwriter <61687663+amyjtechwriter@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Amy Jonsson <amy.jonsson@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2023-05-24 12:32:46 +01:00
Jim Ferenczi
154176b607
Fix _xpack/usage for enterprise_search in docs tests (#95845)
The previous fix (#95565) didn't work since the section was misplaced.
Note that this test runs only on snapshot build so I tested manually and the failure is now related to remote_clusters section missing.

Closes #95603
2023-05-05 10:11:13 +01:00
Jean-Pierre Matsumoto
87b8f1cf73
Bad ref to 'node_id' parameter in Task Mgt doc (#90380)
* Bad ref to 'node_id' parameter in Task Mgt doc

Here in documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/8.4/tasks.html#tasks-api-query-params

Name of parameter to filter node is `nodes`.

* Change 'node_id' into 'nodes' and add 'nodes' as a common param

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-05-03 11:55:29 +02:00
Albert Zaharovits
b90060538a
Fix _xpack_usage for enterprise_search (#95565)
Fixes #95564
2023-04-26 19:23:36 +03:00
Sloane Perrault
b57286aa0f
Add Search Application telemetry to xpack/usage (#95306)
* search application telemetry via xpack.usage

* impl EnterpriseSearchInfoTransportAction, fix test

* fix docs yaml tests and security operator tests

* set enabled & available with settings & license

* test count increases and decreases

* include xpack instead of xpack.usage specifically

* rename enterprise search usage serializing tests

* call action directly

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>

* handle `enabled = false` for entsearch usage

* return empty stats on error

* use an OriginSettingClient

---------

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>
2023-04-25 09:56:09 -04:00
Przemysław Witek
2b70165ffd
[Transform] Allow specifying destination index aliases in the Transform's dest config (#94943) 2023-04-17 15:08:43 +02:00
Mary Gouseti
99145bbe9c
Add new endpoints to configure data lifecycle on a data stream level. (#94590)
With PR we introduce CRUD endpoints which update/delete the data lifecycle on the data stream level. When this is updated it will apply at the next DLM run to all the backing indices that are managed by DLM.
2023-04-04 18:37:38 +02:00
Carlos Delgado
7a782031c4
Add Enterprise Search Module (#94381)
Create base module for ent-search with CRUD APIs to manage behavioral analytics and Search Application.

---------

Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
Co-authored-by: Aurélien FOUCRET <aurelien.foucret@gmail.com>
Co-authored-by: Kathleen DeRusso <63422879+kderusso@users.noreply.github.com>
Co-authored-by: Ioana Tagirta <ioanatia@users.noreply.github.com>
Co-authored-by: Joseph McElroy <joseph.mcelroy@elastic.co>
Co-authored-by: Aurelien FOUCRET <aurelien.foucret@elastic.co>
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2023-04-04 10:36:13 +01:00
Benjamin Trent
f23b906891
Add new similarity field to knn clause in _search (#94828)
This adds a new parameter to `knn` that allows filtering nearest neighbor results that are outside a given similarity.

`num_candidates` and `k` are still required as this controls the nearest-neighbor vector search accuracy and exploration. For each shard the query will search `num_candidates` and only keep those that are within the provided `similarity` boundary, and then finally reduce to only the global top `k` as normal.

For example, when using the `l2_norm` indexed similarity value, this could be considered a `radius` post-filter on `knn`.

relates to: https://github.com/elastic/elasticsearch/issues/84929 && https://github.com/elastic/elasticsearch/pull/93574
2023-03-28 15:29:01 -04:00
Andrei Dan
223385f887
Introduce a _lifecycle/explain API for data stream backing indices (#94621)
This adds an {index}/_lifecycle/explain API to retrieve information
about an index's status within its lifecycle.

The response looks like so:
```
"indices" : {
    ".ds-metrics-foo-2023.03.22-000001" : {
      "index" : ".ds-metrics-foo-2023.03.22-000001",
      "managed_by_dlm" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "rollover_date_millis" : 1679475564293,
      "time_since_rollover" : "121ms",
      "lifecycle" : { },
      "generation_time" : "121ms"
    },
    ".ds-metrics-foo-2023.03.22-000002" : {
      "index" : ".ds-metrics-foo-2023.03.22-000002",
      "managed_by_dlm" : true,
      "index_creation_date_millis" : 1679475564351,
      "time_since_index_creation" : "63ms",
      "lifecycle" : { }
    }
  }
}
```
2023-03-27 08:44:40 +01:00
Keith Massey
ebb860d1af
Adding to the documentation and tests for the _none pipeline (#93057) 2023-01-23 14:09:57 -06:00
Mary Gouseti
2192a982ac
Expose Health Api telemetry via xpack (#91708)
Every node (post `8.7`) collects stats from every health-api request it receives. We extend the `_xpack/usage` endpoint to expose these stats. When a node receives the request it will fan out to collect data from all other nodes, merge them and expose them. If the cluster is not fully upgraded, it will signal it with the `available` flag set to`false`.
2023-01-04 14:34:29 +01:00
István Zoltán Szabó
e16bee0e72
[DOCS] Adds bullet points to the statuses of the health object in transform stats API docs (#91790) 2022-11-22 15:18:08 +01:00
Craig Taverner
81d5859f61
Added documentation for cartesian-bounds aggregation (#91623)
* Added documentation for cartesian-bounds aggregation

* Fixed rounding errors in docs tests
2022-11-18 11:00:41 +01:00
István Zoltán Szabó
e715f3c737
[DOCS] Adds KNN object sub-properties individually to common params (#91503) 2022-11-10 17:23:55 +01:00
István Zoltán Szabó
ed452fb53d
[DOCS] Adds knn object to common parameters (#91464) 2022-11-10 11:21:01 +01:00
Przemysław Witek
95f484c4fd
[Transform] Expand the docs section regarding mappings deduction in transform's dest index (#91077) 2022-10-24 13:43:22 +02:00
Craig Taverner
4c5d24610f
Centroid aggregation for cartesian points and shapes (#89216)
Added Cartesian support for centroid aggregation

* First draft of cartesian-centroid docs
  However, this is largely a duplicate of geo-centroid docs since they are essentially identical behaviour. We should consider merging them.
* Work on isAggregatable caused a minor logic conflict. When that work was done, Point and Shape were not aggregatable, but now they are.
2022-09-28 17:14:30 +02:00
István Zoltán Szabó
45646b78e2
[DOCS] Adds missing_bucket setting to transform APIs (#90111) 2022-09-19 15:22:48 +02:00
István Zoltán Szabó
9a71d1fa78
[DOCS] Clarifies retention policy for transforms (#89685) 2022-08-29 13:17:15 +02:00
István Zoltán Szabó
accf737145
[DOCS] Adds unattended setting to transforms API docs. (#89335) 2022-08-29 11:46:52 +02:00
István Zoltán Szabó
226b8a260e
[DOCS] Modifies the description of frequency. (#89128) 2022-08-08 15:44:00 +02:00