Commit graph

292 commits

Author SHA1 Message Date
Przemysław Witek
2b70165ffd
[Transform] Allow specifying destination index aliases in the Transform's dest config (#94943) 2023-04-17 15:08:43 +02:00
Mary Gouseti
99145bbe9c
Add new endpoints to configure data lifecycle on a data stream level. (#94590)
With PR we introduce CRUD endpoints which update/delete the data lifecycle on the data stream level. When this is updated it will apply at the next DLM run to all the backing indices that are managed by DLM.
2023-04-04 18:37:38 +02:00
Carlos Delgado
7a782031c4
Add Enterprise Search Module (#94381)
Create base module for ent-search with CRUD APIs to manage behavioral analytics and Search Application.

---------

Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
Co-authored-by: Aurélien FOUCRET <aurelien.foucret@gmail.com>
Co-authored-by: Kathleen DeRusso <63422879+kderusso@users.noreply.github.com>
Co-authored-by: Ioana Tagirta <ioanatia@users.noreply.github.com>
Co-authored-by: Joseph McElroy <joseph.mcelroy@elastic.co>
Co-authored-by: Aurelien FOUCRET <aurelien.foucret@elastic.co>
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2023-04-04 10:36:13 +01:00
Benjamin Trent
f23b906891
Add new similarity field to knn clause in _search (#94828)
This adds a new parameter to `knn` that allows filtering nearest neighbor results that are outside a given similarity.

`num_candidates` and `k` are still required as this controls the nearest-neighbor vector search accuracy and exploration. For each shard the query will search `num_candidates` and only keep those that are within the provided `similarity` boundary, and then finally reduce to only the global top `k` as normal.

For example, when using the `l2_norm` indexed similarity value, this could be considered a `radius` post-filter on `knn`.

relates to: https://github.com/elastic/elasticsearch/issues/84929 && https://github.com/elastic/elasticsearch/pull/93574
2023-03-28 15:29:01 -04:00
Andrei Dan
223385f887
Introduce a _lifecycle/explain API for data stream backing indices (#94621)
This adds an {index}/_lifecycle/explain API to retrieve information
about an index's status within its lifecycle.

The response looks like so:
```
"indices" : {
    ".ds-metrics-foo-2023.03.22-000001" : {
      "index" : ".ds-metrics-foo-2023.03.22-000001",
      "managed_by_dlm" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "rollover_date_millis" : 1679475564293,
      "time_since_rollover" : "121ms",
      "lifecycle" : { },
      "generation_time" : "121ms"
    },
    ".ds-metrics-foo-2023.03.22-000002" : {
      "index" : ".ds-metrics-foo-2023.03.22-000002",
      "managed_by_dlm" : true,
      "index_creation_date_millis" : 1679475564351,
      "time_since_index_creation" : "63ms",
      "lifecycle" : { }
    }
  }
}
```
2023-03-27 08:44:40 +01:00
Keith Massey
ebb860d1af
Adding to the documentation and tests for the _none pipeline (#93057) 2023-01-23 14:09:57 -06:00
Mary Gouseti
2192a982ac
Expose Health Api telemetry via xpack (#91708)
Every node (post `8.7`) collects stats from every health-api request it receives. We extend the `_xpack/usage` endpoint to expose these stats. When a node receives the request it will fan out to collect data from all other nodes, merge them and expose them. If the cluster is not fully upgraded, it will signal it with the `available` flag set to`false`.
2023-01-04 14:34:29 +01:00
István Zoltán Szabó
e16bee0e72
[DOCS] Adds bullet points to the statuses of the health object in transform stats API docs (#91790) 2022-11-22 15:18:08 +01:00
Craig Taverner
81d5859f61
Added documentation for cartesian-bounds aggregation (#91623)
* Added documentation for cartesian-bounds aggregation

* Fixed rounding errors in docs tests
2022-11-18 11:00:41 +01:00
István Zoltán Szabó
e715f3c737
[DOCS] Adds KNN object sub-properties individually to common params (#91503) 2022-11-10 17:23:55 +01:00
István Zoltán Szabó
ed452fb53d
[DOCS] Adds knn object to common parameters (#91464) 2022-11-10 11:21:01 +01:00
Przemysław Witek
95f484c4fd
[Transform] Expand the docs section regarding mappings deduction in transform's dest index (#91077) 2022-10-24 13:43:22 +02:00
Craig Taverner
4c5d24610f
Centroid aggregation for cartesian points and shapes (#89216)
Added Cartesian support for centroid aggregation

* First draft of cartesian-centroid docs
  However, this is largely a duplicate of geo-centroid docs since they are essentially identical behaviour. We should consider merging them.
* Work on isAggregatable caused a minor logic conflict. When that work was done, Point and Shape were not aggregatable, but now they are.
2022-09-28 17:14:30 +02:00
István Zoltán Szabó
45646b78e2
[DOCS] Adds missing_bucket setting to transform APIs (#90111) 2022-09-19 15:22:48 +02:00
István Zoltán Szabó
9a71d1fa78
[DOCS] Clarifies retention policy for transforms (#89685) 2022-08-29 13:17:15 +02:00
István Zoltán Szabó
accf737145
[DOCS] Adds unattended setting to transforms API docs. (#89335) 2022-08-29 11:46:52 +02:00
István Zoltán Szabó
226b8a260e
[DOCS] Modifies the description of frequency. (#89128) 2022-08-08 15:44:00 +02:00
Andrey
ebde65d2a2
Remove suggest flag from index stats docs (#85479) 2022-07-14 12:50:53 -04:00
Przemysław Witek
8656a29675
[Transform] Implement per-transform num_failure_retries setting. (#87361) 2022-06-09 15:22:06 +02:00
Nik Everett
86effae55c
Docs: Data streams only support create (#87263)
This removes "data streams" from the docs for the `index`, `delete`,
and `update` actions because data streams only support the `update`
action.

Closes #87231
2022-06-08 13:41:42 -04:00
Przemysław Witek
70e37ae7c6
[Transform] Support range aggregation in transform (#86501) 2022-05-16 15:21:00 +02:00
James Rodewig
8322809607
Typo in example of the filter_path option (#84551) (#84661)
(cherry picked from commit b637d8bc70)

Co-authored-by: Stéphane Campinas <stephane.campinas@gmail.com>
2022-03-04 09:10:55 -05:00
James Rodewig
f9a64b2e86
[DOCS] Fix ignore_unavailable parameter definition (#84071)
The current `ignore_unavailable` definition is a bit misleading. The parameter primarily determines if a request that targets a missing or closed index returns an error.
2022-02-17 08:24:06 -05:00
Lisa Cawley
458ef91066
[DOCS] Move ML info and upgrade APIs (#84005) 2022-02-16 11:23:00 -08:00
Yannick Welsch
74b5bfdb73
Feature usage actions for archive (#83931)
Relates #81210
2022-02-16 08:31:41 +01:00
James Rodewig
3aee6171e7
[DOCS] Link to the 7.17 upgrade assistant docs (#83613) (#83615)
The 8.0 upgrade assistant is only available in 7.17. This updates our docs to link to the 7.17 version of the related Kibana docs page.

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
(cherry picked from commit 3787c13619)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2022-02-07 18:29:26 -05:00
Jake Landis
5c73ad1da2
REST API compatiblity documentation (#83487)
Co-authored-by: debadair <debadair@elastic.co>
2022-02-04 16:32:22 -06:00
David Kyle
04d520c212
[ML][DOCS] Add Trained model APIs to the REST APIs index (#82791) 2022-01-25 15:14:48 +00:00
Lisa Cawley
0a16177f40
[DOCS] Fix formatting in cat transforms API (#82899) 2022-01-20 17:45:43 -08:00
Przemysław Witek
7be74a8046
Introduce deduce_mappings transform setting (#82256) 2022-01-18 09:01:23 +01:00
Ievgen Degtiarenko
11b52619c5
do not scroll if max docs is less than scroll size (update/delete by query) (#81654)
This change allows to not open scroll while reindex/delete_by_query/update_by_query
if configured max_docs if less then or equal to the number of documents returned by the scroll batch.
2021-12-21 15:26:51 +01:00
Lisa Cawley
429bdd9afc
[DOCS] Move trained model APIs out of dataframe analytics (#81315) 2021-12-03 09:21:09 -08:00
Lisa Cawley
4ed6e8ad3c
[DOCS] Adds missing timeout parameter to transform APIs (#81129) 2021-12-02 13:28:28 -08:00
James Rodewig
cf818edcde
[DOCS] Fix syntax error in bulk dynamic_templates docs (#81264) 2021-12-02 14:05:21 -05:00
Lisa Cawley
8ab03d021c
[DOCS] Edits reset transforms API (#81027) 2021-11-25 08:40:50 -08:00
Benjamin Trent
5627dc66e1
[ML] deprecate estimated_heap_memory_usage_bytes and replace with model_size_bytes (#80554)
This deprecates estimated_heap_memory_usage_bytes on model put and replaces it with model_size_bytes.

On GET, only model_size_bytes is returned unless v7 rest-api compatibility is requested.

For the ml/info API, only model_size_bytes is returned

A forward-port of: #80545
2021-11-10 10:23:25 -05:00
James Rodewig
f56a0f4b66
[DOCS] Remove testenv annotations from doc snippet tests (#80023)
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.

Relates to #79309, #31619
2021-11-05 18:38:50 -04:00
Benjamin Trent
4557d5f797
[ML] adding new model types and deployments to xpack usage (#80282)
This adds new model types + deployment information to xpack/usage under ml.inference

closes: #80200
2021-11-04 11:26:37 -04:00
James Rodewig
7940e0777c
[DOCS] Re-add several query params to search API docs (#79716)
PR #55884 removed documentation for several query parameters from the search API
docs. During tests, I failed to notice that these are valid parameters but require other parameters to use.

Changes:

* Notes the following search API parameters require the `q` query string parameter:

  * `analyzer`
  * `analyze_wildcard`
  * `default_operator`
  * `df`
  * `lenient`

* Notes the following search API parameters require the `suggest_field` and `suggest_text` query parameters:

  * `suggest_mode`
  * `suggest_size`

* Re-adds the above parameters to the search API docs.

These changes also affect API documentation that reuses the search API parameters:

* Delete by query API
* Update by query API
* Count API
* Explain API
* Validate API

Closes #79674
2021-10-25 11:58:54 -04:00
Przemysław Witek
1595d3a20f
[Transform] Add _meta field to TransformConfig (#79003) 2021-10-15 08:12:03 +02:00
edh-oss
62a471aefe
Update JSON parser and snippets (#77983)
Related to issue  #77823

This does the following:

- Updates several asciidoc files that contained code snippets with
  invalid JSON, most involving unnecessary trailing commas.

- Makes the switch from the Groovy JSON parser to the Jackson parser,
  pursuant to the general goal of eliminating Groovy dependence.

- Makes testing of JSON validity at build time more strict.

Note that this update still allows backslash escaping for any
character. Currently that matters because of the file
"docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc",
specifically this part:

    "attributes" : {
      "ml.machine_memory" :
        "$body.datafeeds.0.node.attributes.ml\.machine_memory",
      "ml.max_open_jobs" : "512"
    }

It's not clear to me what change, if any, is appropriate there. So,
I've left in the escaped period and configured the parser to ignore
it for the time being.
2021-09-20 11:08:26 +01:00
István Zoltán Szabó
81851684d1
[DOCS] Fixes description of index_total property for GET Transforms stats API docs. (#77354) 2021-09-08 08:53:15 +02:00
James Rodewig
cfae69717a
[DOCS] Update anchor and add redirect for aliases (#77349)
PRs #73062 and #73043 repurposed the `alias` anchor for a new guide for index
and data stream aliases. Previously, this anchor was used for our field alias
documentation.

Repurposing the anchor has caused continuity errors for users selecting
different versions of the ES docs. It could also cause confusion for users with
a `/current/` link to the `alias` page.

This updates the anchor for the alias guide and adds a redirect page to
disambiguate the `alias` anchor.

It also fixes a bread crumb issue for redirects following the 'Modifying your
Data' redirect page.

Closes #77034.
2021-09-07 09:42:42 -04:00
Gordon Brown
4944959acf
Node shutdown API docs (#74505)
* Put Shutdown API docs

* Get Shutdown API docs

* Properly escape regex

* Doc tests build now! (but don't pass)

* Doc tests pass now! (with a code fix that will go in a different PR)

* DELETE shutdown docs

* Edit for language & consistency

* Fix doctest

* Add cloud-only banner

* Add allocation_delay docs

* Restore file that got deleted somehow?

* Use `restart` type in example to demonstrate `allocation_delay` param

* Fix typo per review

Co-authored-by: debadair <debadair@elastic.co>

* Vastly improve wording per review

Co-authored-by: debadair <debadair@elastic.co>

* Adjust test request & response so it passes

Co-authored-by: Deb Adair <debadair@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-08-19 12:42:27 -04:00
Przemysław Witek
ec07e4213e
[Transform] Rename interim_results to align_checkpoints (#76609) 2021-08-18 13:58:50 +02:00
Mayya Sharipova
23781bb555
Remove xpack actions for vector field (#75017)
We have already decided not to have xpack usage for field mappers
(see #53076). As mappings stats of all fields is already tracked
in cluster stats.
Moreover xpack usage for vector field is a quite expensive operation
(see #74974).

This removes xpack actions for vector field.
2021-07-13 09:52:50 -04:00
James Rodewig
a9ed7dc1eb
[DOCS] Document async SQL APIs (#75078)
Adds formal API docs and JSON specs for the following APIs:

* Get async SQL search
* Get async SQL search status
* Delete async SQL search

Closes #74845
2021-07-12 10:12:02 -04:00
Esduard
b3e171b600
changing hard coded 10k page size limit to 65k (#74651)
changes hard code 10k page size limit to 65k(default for `search.max_bucket`)

Closes #57719

Co-authored-by: Hendrik Muhs <hendrik.muhs@elastic.co>
2021-07-05 15:21:36 +02:00
James Rodewig
70764de4b6
[DOCS] Move EQL APIs to separate page (#74846)
In preparation for #74845, we need to create formal API reference documentation for our SQL APIs.

Due to the number of SQL APIs, we'll likely need to create a separate nested page for them. For parity, this PR moves
our EQL APIs to a separate page as well. Previously, they were listed under our search APIs.
2021-07-01 13:59:35 -04:00
debadair
29bb8fd67e
[DOCS] Add info about GET/POST to the API conventions. (#74529)
* [DOCS] Add info about GET/POST to the API conventions.

* Fixed anchor

* Added redirect

* Incorporated review suggestions
2021-06-30 09:24:42 -07:00