Commit graph

9020 commits

Author SHA1 Message Date
Benjamin Trent
a68c6acdb3
[ML] adding new PUT trained model vocabulary endpoint (#77387)
This commit removes the ability to set the vocabulary location in the model config.
This opts instead for sane defaults to be set and used. Wrapping this up in an
API.

The index is now always the internally managed .ml-inference-native index
and the document ID is always <model_id>_vocabulary

This API only works for pytorch/nlp type models.
2021-09-08 10:21:45 -04:00
Przemyslaw Gomulka
9d3b39ccb7
[doc] Remove external herokuapp website link (#77405)
The website is no longer live and is external to elastic domain. It was used as help for testing migration
2021-09-08 14:10:08 +02:00
István Zoltán Szabó
81851684d1
[DOCS] Fixes description of index_total property for GET Transforms stats API docs. (#77354) 2021-09-08 08:53:15 +02:00
David Turner
1045abe71f
Limit count of HTTP channels with tracked stats (#77303)
Today we expire the client stats for HTTP channels 5 minutes after they
close. It's possible to open a very large number of HTTP channels in 5
minutes, possibly inadvertently, and the stats for those channels can be
overwhelming.

This commit introduces a limit on the number of channels tracked by each
node which applies in addition to the age limit, and makes these limits
configurable via static settings. It drops the pruning of old stats when
starting to track a new channel and instead uses a queue to expire the
oldest stats when each channel closes if necessary to respect the count
limit; it only performs age-based expiry when retrieving the stats,
since the count limit now bounds the memory needed. Finally, it
tightents up some missing synchronization and makes sure that we expose
only immutable objects to the stats subsystem.
2021-09-08 07:25:57 +01:00
Benjamin Trent
708491d0d3
[ML] add allocation state reason and support for partial model allocations (#76925)
Previously, if a model failed to be allocated on any node, the deployment failed.

This commit allows for an allocation to be partially_started and indicates its
current state via a new state value in the deployment stats API.

Additionally, when starting a deployment, the user may specify to wait_for
starting, partially_started, started and the API will block (as long as timeout doesn't expire) until that state is reached.
2021-09-07 15:23:13 -04:00
Armin Braun
7508720fe5
Add Filtering by SLM Policy to Get Snapshots API (#77321)
It's in the title, add new `slm_policy_filter` param as a filter to the get snapshots API.
2021-09-07 18:24:09 +02:00
James Rodewig
f170f6c000
[DOCS] Fix default for is_write_index (#77006) (#77362)
This updates the default for the `is_write_index` parameter of the aliases API and create alias API.
The default behavior for `is_write_index` can vary based on:

1. Whether the alias is used for indices or data streams.
2. If an index alias, whether the alias points to multiple indices.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: 诗心客 <ishixinke@qq.com>
2021-09-07 11:34:53 -04:00
James Rodewig
adcc05c798 [DOCS] Add anchor text to field alias redirect 2021-09-07 10:10:52 -04:00
James Rodewig
cfae69717a
[DOCS] Update anchor and add redirect for aliases (#77349)
PRs #73062 and #73043 repurposed the `alias` anchor for a new guide for index
and data stream aliases. Previously, this anchor was used for our field alias
documentation.

Repurposing the anchor has caused continuity errors for users selecting
different versions of the ES docs. It could also cause confusion for users with
a `/current/` link to the `alias` page.

This updates the anchor for the alias guide and adds a redirect page to
disambiguate the `alias` anchor.

It also fixes a bread crumb issue for redirects following the 'Modifying your
Data' redirect page.

Closes #77034.
2021-09-07 09:42:42 -04:00
Benjamin Trent
de49ff22a4
[ML] creating new PUT model definition part API (#76987)
This commit simplifies the interactions for uploading chunked model definitions and model vocabulary.
2021-09-07 08:22:52 -04:00
David Turner
0534fbada8
Clarify 'runs in the foreground' (#76709)
Users following the installation guide sometimes get stuck, reporting
that Elasticsearch has "hung" after running `./bin/elasticsearch`
because it doesn't exit, it just sits there in the foreground waiting
for something to happen. We do say that it "runs in the foreground" in
the docs but that's not clear enough, maybe folks expect it to declare
it started up successfully or to open a browser window for them or
something.

This commit expands the docs to clarify what users should expect from
running Elasticsearch from the command line and what they should do
next.
2021-09-06 12:29:18 +01:00
debadair
5b41c03a24
[DOCS] Adds information about version compatibility headers (#77096)
* [DOCS] Adds information about version compatibility

* Clarified that the behavior is not guaranteed to be the same

* Update docs/reference/api-conventions.asciidoc

* Update docs/reference/api-conventions.asciidoc

* Update docs/reference/api-conventions.asciidoc

Co-authored-by: Adam Locke <adam.locke@elastic.co>

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-09-03 14:33:23 -07:00
Gordon Brown
7c7f312d76
Move index.hidden from Static to Dynamic settings (#77218)
The setting `index.hidden` was originally implemented as a static
setting, but when it was converted to be a dynamic setting, the docs
were never updated to reflect that.

This commit moves the docs for `index.hidden` from the section for
static settings to the section for dynamic settings.
2021-09-03 09:15:52 -06:00
Benjamin Trent
02e17c3442
[ML] adding new defer_definition_decompression parameter to put trained model API (#77189)
This new parameter is a boolean parameter that allows
users to put in a compressed model without it having
to be inflated on the master node during the put
request

This is useful for system/module set up and then later
having the model validated and fully parsed when it
is being loaded on a node for usage
2021-09-03 09:07:54 -04:00
István Zoltán Szabó
cdec5228e8
[DOCS] Fixes line breaks. (#77248) 2021-09-03 14:40:43 +02:00
István Zoltán Szabó
70a012b0c7
[DOCS] Fixes section IDs in start/stop trained model deployment APIs. (#77247) 2021-09-03 14:24:37 +02:00
Martijn van Groningen
1ae4f3c937
Add enrich node cache (#76800)
Introduce a LRU cache to avoid searches that occur frequently
from the enrich processor.

Relates to #48988
2021-09-03 09:33:44 +02:00
Lisa Cawley
007469af63
[DOCS] Replaces index pattern in ML docs (#77041) 2021-09-01 10:26:06 -07:00
Adam Locke
32e364d394
[DOCS] Clarify indexing a runtime field (#77117)
* [DOCS] Clarify indexing a runtime field

* Clarify wording based on reviewer feedback
2021-09-01 11:59:11 -04:00
Benjamin Trent
0e1efa6533
[ML] generalize pytorch sentiment analysis to text classification (#77084)
* [ML] generalize pytorch sentiment analysis to text classification

* Update x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/inference/trainedmodel/TextClassificationConfig.java
2021-09-01 08:45:13 -04:00
Benjamin Trent
100f222650
Adds support for the rate aggregation under a composite agg (#76992)
rate aggregation should support being a sub-aggregation
of a composite agg.

The catch is that the composite aggregation source
must be a date histogram. Other sources can be present
but their must be exactly one date histogram source
otherwise the rate aggregation does not know which
interval to compare its unit rate to.

closes https://github.com/elastic/elasticsearch/issues/76988
2021-09-01 07:29:13 -04:00
Armin Braun
0920e21445
Implement Sort By Repository Name in Get Snapshots API (#77049)
This one is the last sort column not yet implemented but used by Kibana.
2021-09-01 13:01:58 +02:00
Francois-Clement Brossard
ec11f9f931
Execute enrich policy wait_for_completion docfix (#77046) 2021-08-31 14:02:24 -04:00
Adam Locke
b9ad9860af
[DOCS] Add ES security principles (#76850)
* [DOCS] Add ES security principles

* Incorporating review feedback

* More changes from review feedback

* Fix cross-link to Painless guide

* Clarify callout text

* Add information about elasticsearch user

* Minor wording edits

* Consolidate Java Security Manager description, plus other edits

* Clarify not running as root

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-08-31 12:37:22 -04:00
Lukas Wegmann
627c0ee9c6
SQL: Fix disjunctions (and IN) with multiple date math expressions (#76424)
* SQL: Fix disjunctions with multiple date math expressions

* review comments

* remove redundant FieldAttribute parameter

* address comments and add more specs

* fix typo
2021-08-31 17:30:49 +02:00
István Zoltán Szabó
ea007902ef
[DOCS] Adds anomaly job health alert type docs (#76659)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2021-08-30 16:11:34 +02:00
Armin Braun
48f3784a6d
Add Sort By Shard Count and Failed Shard Count to Get Snapshots API (#77011)
It's in the title. As requested by the Kibana team, adding these two additional sort columns.

relates #74350
2021-08-30 13:39:51 +02:00
Dan Hermann
c4aad2965f
[DOCS] Map iteration support in ForEach processor (#76972) 2021-08-27 07:35:11 -05:00
Dan Hermann
4886753dec
[DOCS] Final pipelines may not change target index (#76997) 2021-08-27 07:32:02 -05:00
debadair
12dda6fb36
[DOCS] Add ILM error/troubleshooting info. Closes #75849 (#76957)
* [DOCS] Add ILM error/troubleshooting info. Closes #75849

* Apply suggestions from code review

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

* Updated xref & fixed whitespace issues

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-08-26 15:44:50 -04:00
Joe Gallo
a127154042
[DOCS] Fix the terms enum API docs for search_after (#76991) 2021-08-26 14:07:06 -04:00
Adam Locke
1056c857ee
[DOCS] Update combined fields wording (#76893)
* [DOCS] Update combined fields wording

* Clarifications from review feedback
2021-08-26 13:16:55 -04:00
Howard
4432b39112
[DOCS] Fix formatting for snapshot_meta thread pool (#76973) 2021-08-26 10:36:26 -04:00
Lisa Cawley
d36f24fbc3
[DOCS] Update datafeed details in ML docs (#76854) 2021-08-25 11:35:21 -07:00
István Zoltán Szabó
789368b38f
[DOCS] Fixes a syntax error in datafeed runtime field example. (#76917) 2021-08-25 12:04:32 +02:00
James Rodewig
bb9587351e
[DOCS] Fix typos (#76875) 2021-08-24 08:47:12 -04:00
Przemysław Witek
676d4de3de
[Transform] Implement the ability to preview the existing transform (#76697) 2021-08-24 14:41:49 +02:00
David Turner
4a17847b85
Add timing stats to publication process (#76771)
This commit introduces into the node stats API various statistics to
track the time that the elected master spends in various phases of the
cluster state publication process.

Relates #76625
2021-08-23 17:38:32 +01:00
James Rodewig
31153e0002
[DOCS] Fix multi-value search preference docs (#76822)
You can't combine a `_shards` search `preference` argument with `<custom-string>`.
2021-08-23 08:33:12 -04:00
Martijn van Groningen
8a1deff75a
Improve fault-detection.asciidoc (#76821)
Add section to fault-detection.asciidoc about nodes being removed from cluster
due to slow cluster state applying.
2021-08-23 14:31:06 +02:00
István Zoltán Szabó
8aed99fc02
[DOCS] Adds links that point to loss function to ML API docs. (#76438) 2021-08-23 13:09:37 +02:00
James Rodewig
38fe33a870 [DOCS] Fix whitespace to hide attribute 2021-08-22 21:18:51 -04:00
Keith Massey
0aab3c01c9
Ensuring that the ShrinkAction does not hang if total shards per node is too low (#76732)
We added configuration to AllocateAction to set the total shards per node property on the index. This makes it possible that a user could set this to a value lower than the total number of shards in the index that is about to be shrunk, meaning that all of the shards could not be moved to a single node in the ShrinkAction. This commit unsets the total shards per node property so that we fall back to the default value (-1, unlimited) in the ShrinkAction to avoid this.
Relates to #44070
2021-08-20 09:41:19 -05:00
Peter Dyson
cad55c8393
[DOCS] Clarify usage of optional human readable jvm uptime metric in Nodes Stats API (#76545)
To return the JVM `uptime` metric, the `human` query parameter must be `true`.

Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-08-20 08:55:23 -04:00
James Rodewig
e246e1ce53
[DOCS] Remove 'step' from headings (#76753) 2021-08-20 08:52:04 -04:00
James Rodewig
22a6c1f0d3
[DOCS] Add search xref tip to query_string docs (#76728)
Adds a tip containing a cross-reference to the "Search your data" docs.
This is the preferred starting point for ES search.
2021-08-20 08:44:20 -04:00
James Rodewig
1304e648c8
[DOCS] Fix upgrade version logic for -alpha and -beta releases (#76727) 2021-08-20 08:29:35 -04:00
James Rodewig
1c355bfda9
[DOCS] Reuse snapshot config in put SLM policy API docs (#76712)
Updates the put SLM policy API's `config` parameter to reuse the create snapshot API's request body parameters.
Previously, the `config` parameter was missing the `feature_states` parameter. This change should keep the two docs in sync.
2021-08-20 08:29:16 -04:00
Gordon Brown
4944959acf
Node shutdown API docs (#74505)
* Put Shutdown API docs

* Get Shutdown API docs

* Properly escape regex

* Doc tests build now! (but don't pass)

* Doc tests pass now! (with a code fix that will go in a different PR)

* DELETE shutdown docs

* Edit for language & consistency

* Fix doctest

* Add cloud-only banner

* Add allocation_delay docs

* Restore file that got deleted somehow?

* Use `restart` type in example to demonstrate `allocation_delay` param

* Fix typo per review

Co-authored-by: debadair <debadair@elastic.co>

* Vastly improve wording per review

Co-authored-by: debadair <debadair@elastic.co>

* Adjust test request & response so it passes

Co-authored-by: Deb Adair <debadair@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-08-19 12:42:27 -04:00
István Zoltán Szabó
7faec52a1e
[DOCS] Fixes model_prune_window property description. (#76711) 2021-08-19 16:16:37 +02:00