github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-04-25 07:37:19 -04:00
Code Issues Projects Releases Packages Wiki Activity

Consolidated Circuit Breaker documentation to include EQL and ML infer (#90809)

Browse source 
Fixes #85851 

Co-authored-by: Iraklis Psaroudakis <kingherc@gmail.com>
This commit is contained in:
Paramdeep Singh 2022-10-14 17:03:52 +05:30 • committed by GitHub
parent bd97e2bca0
commit 34ff7a9d98
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 63 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

35
docs/reference/eql/eql.asciidoc
View file

@ -849,40 +849,7 @@ GET /cluster_one:my-data-stream,cluster_two:my-data-stream/_eql/search
[[eql-circuit-breaker]]
=== EQL circuit breaker settings
When a <<eql-sequences, sequence>> query is executed, the node handling the query
needs to keep some structures in memory, which are needed by the algorithm
implementing the sequence matching. When large amounts of data need to be processed,
and/or a large amount of matched sequences is requested by the user (by setting the
<<eql-search-api-params-size, size>> query param), the memory occupied by those
structures could potentially exceed the available memory of the JVM. This would cause
an `OutOfMemory` exception which would bring down the node.
To prevent this from happening, a special <<circuit-breaker, circuit breaker>> is used,
which limits the memory allocation during the execution of a <<eql-sequences, sequence>>
query. When the breaker is triggered, an `org.elasticsearch.common.breaker.CircuitBreakingException`
is thrown and a descriptive error message is returned to the user.
This <<circuit-breaker, circuit breaker>> can be configured using the following settings:
`breaker.eql_sequence.limit`::
(<<cluster-update-settings,Dynamic>>) The limit for circuit breaker used to restrict
the memory utilisation during the execution of an EQL sequence query. This value is
defined as a percentage of the JVM heap. Defaults to `50%`. If the
<<parent-circuit-breaker,parent circuit breaker>> is set to a value less than `50%`,
this setting uses that value as its default instead.
`breaker.eql_sequence.overhead`::
(<<cluster-update-settings,Dynamic>>) A constant that sequence query memory
estimates are multiplied by to determine a final estimate. Defaults to `1`.
`breaker.eql_sequence.type`::
(<<static-cluster-setting,Static>>) Circuit breaker type. Valid values are:
`memory` (Default):::
The breaker limits memory usage for EQL sequence queries.
`noop`:::
Disables the breaker.
The relevant circuit breaker settings can be found in the <<circuit-breakers-page-eql, Circuit Breakers page>>.
include::syntax.asciidoc[]
include::functions.asciidoc[]

61
docs/reference/modules/indices/circuit_breaker.asciidoc
View file

@ -177,3 +177,64 @@ it triggers the regex circuit breaker and returns an error.
+
{es} only applies this limit if
<<script-painless-regex-enabled,`script.painless.regex.enabled`>> is `limited`.
[[circuit-breakers-page-eql]]
[discrete]
=== EQL circuit breaker
When a <<eql-sequences, sequence>> query is executed, the node handling the query
needs to keep some structures in memory, which are needed by the algorithm
implementing the sequence matching. When large amounts of data need to be processed,
and/or a large amount of matched sequences is requested by the user (by setting the
<<eql-search-api-params-size, size>> query param), the memory occupied by those
structures could potentially exceed the available memory of the JVM. This would cause
an `OutOfMemory` exception which would bring down the node.
To prevent this from happening, a special <<circuit-breaker, circuit breaker>> is used,
which limits the memory allocation during the execution of a <<eql-sequences, sequence>>
query. When the breaker is triggered, an `org.elasticsearch.common.breaker.CircuitBreakingException`
is thrown and a descriptive error message is returned to the user.
This <<circuit-breaker, circuit breaker>> can be configured using the following settings:
`breaker.eql_sequence.limit`::
(<<cluster-update-settings,Dynamic>>) The limit for circuit breaker used to restrict
the memory utilisation during the execution of an EQL sequence query. This value is
defined as a percentage of the JVM heap. Defaults to `50%`. If the
<<parent-circuit-breaker,parent circuit breaker>> is set to a value less than `50%`,
this setting uses that value as its default instead.
`breaker.eql_sequence.overhead`::
(<<cluster-update-settings,Dynamic>>) A constant that sequence query memory
estimates are multiplied by to determine a final estimate. Defaults to `1`.
`breaker.eql_sequence.type`::
(<<static-cluster-setting,Static>>) Circuit breaker type. Valid values are:
`memory` (Default):::
The breaker limits memory usage for EQL sequence queries.
`noop`:::
Disables the breaker.
[[circuit-breakers-page-model-inference]]
[discrete]
==== {ml-cap} circuit breaker
`breaker.model_inference.limit`::
(<<cluster-update-settings,Dynamic>>) The limit for the trained model circuit
breaker. This value is defined as a percentage of the JVM heap. Defaults to
`50%`. If the <<parent-circuit-breaker,parent circuit breaker>> is set to a
value less than `50%`, this setting uses that value as its default instead.
`breaker.model_inference.overhead`::
(<<cluster-update-settings,Dynamic>>) A constant that all trained model
estimations are multiplied by to determine a final estimation. See
<<circuit-breaker>>. Defaults to `1`.
`breaker.model_inference.type`::
(<<static-cluster-setting,Static>>) The underlying type of the circuit breaker.
There are two valid options: `noop` and `memory`. `noop` means the circuit
breaker does nothing to prevent too much memory usage. `memory` means the
circuit breaker tracks the memory used by trained models and can potentially
break and prevent `OutOfMemory` errors. The default value is `memory`.

17
docs/reference/settings/ml-settings.asciidoc
View file

@ -220,20 +220,5 @@ defaults to the largest size that could be assigned in the current cluster.
[[model-inference-circuit-breaker]]
==== {ml-cap} circuit breaker settings
`breaker.model_inference.limit`::
(<<cluster-update-settings,Dynamic>>) The limit for the trained model circuit
breaker. This value is defined as a percentage of the JVM heap. Defaults to
`50%`. If the <<parent-circuit-breaker,parent circuit breaker>> is set to a
value less than `50%`, this setting uses that value as its default instead.
The relevant circuit breaker settings can be found in the <<circuit-breakers-page-model-inference, Circuit Breakers page>>.
`breaker.model_inference.overhead`::
(<<cluster-update-settings,Dynamic>>) A constant that all trained model
estimations are multiplied by to determine a final estimation. See
<<circuit-breaker>>. Defaults to `1`.
`breaker.model_inference.type`::
(<<static-cluster-setting,Static>>) The underlying type of the circuit breaker.
There are two valid options: `noop` and `memory`. `noop` means the circuit
breaker does nothing to prevent too much memory usage. `memory` means the
circuit breaker tracks the memory used by trained models and can potentially
break and prevent `OutOfMemory` errors. The default value is `memory`.