mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-24 23:27:25 -04:00
93 lines
3.2 KiB
Text
93 lines
3.2 KiB
Text
[[rejected-requests]]
|
|
=== Rejected requests
|
|
|
|
When {es} rejects a request, it stops the operation and returns an error with a
|
|
`429` response code. Rejected requests are commonly caused by:
|
|
|
|
* A <<high-cpu-usage,depleted thread pool>>. A depleted `search` or `write`
|
|
thread pool returns a `TOO_MANY_REQUESTS` error message.
|
|
|
|
* A <<circuit-breaker-errors,circuit breaker error>>.
|
|
|
|
* High <<index-modules-indexing-pressure,indexing pressure>> that exceeds the
|
|
<<memory-limits,`indexing_pressure.memory.limit`>>.
|
|
|
|
****
|
|
If you're using Elastic Cloud Hosted, then you can use AutoOps to monitor your cluster. AutoOps significantly simplifies cluster management with performance recommendations, resource utilization visibility, real-time issue detection and resolution paths. For more information, refer to https://www.elastic.co/guide/en/cloud/current/ec-autoops.html[Monitor with AutoOps].
|
|
****
|
|
|
|
[discrete]
|
|
[[check-rejected-tasks]]
|
|
==== Check rejected tasks
|
|
|
|
To check the number of rejected tasks for each thread pool, use the
|
|
<<cat-thread-pool,cat thread pool API>>. A high ratio of `rejected` to
|
|
`completed` tasks, particularly in the `search` and `write` thread pools, means
|
|
{es} regularly rejects requests.
|
|
|
|
[source,console]
|
|
----
|
|
GET /_cat/thread_pool?v=true&h=id,name,queue,active,rejected,completed
|
|
----
|
|
|
|
`write` thread pool rejections frequently appear in the erring API and
|
|
correlating log as `EsRejectedExecutionException` with either
|
|
`QueueResizingEsThreadPoolExecutor` or `queue capacity`.
|
|
|
|
These errors are often related to <<task-queue-backlog,backlogged tasks>>.
|
|
|
|
See https://www.youtube.com/watch?v=auZJRXoAVpI[this video] for a walkthrough
|
|
of troubleshooting threadpool rejections.
|
|
|
|
[discrete]
|
|
[[check-circuit-breakers]]
|
|
==== Check circuit breakers
|
|
|
|
To check the number of tripped <<circuit-breaker,circuit breakers>>, use the
|
|
<<cluster-nodes-stats,node stats API>>.
|
|
|
|
[source,console]
|
|
----
|
|
GET /_nodes/stats/breaker
|
|
----
|
|
|
|
These statistics are cumulative from node startup. For more information, see
|
|
<<circuit-breaker,circuit breaker errors>>.
|
|
|
|
See https://www.youtube.com/watch?v=k3wYlRVbMSw[this video] for a walkthrough
|
|
of diagnosing circuit breaker errors.
|
|
|
|
[discrete]
|
|
[[check-indexing-pressure]]
|
|
==== Check indexing pressure
|
|
|
|
To check the number of <<index-modules-indexing-pressure,indexing pressure>>
|
|
rejections, use the <<cluster-nodes-stats,node stats API>>.
|
|
|
|
[source,console]
|
|
----
|
|
GET _nodes/stats?human&filter_path=nodes.*.indexing_pressure
|
|
----
|
|
|
|
These stats are cumulative from node startup.
|
|
|
|
Indexing pressure rejections appear as an
|
|
`EsRejectedExecutionException`, and indicate that they were rejected due
|
|
to `combined_coordinating_and_primary`, `coordinating`, `primary`, or `replica`.
|
|
|
|
These errors are often related to <<task-queue-backlog,backlogged tasks>>,
|
|
<<docs-bulk,bulk index>> sizing, or the ingest target's
|
|
<<index-modules,`refresh_interval` setting>>.
|
|
|
|
See https://www.youtube.com/watch?v=QuV8QqSfc0c[this video] for a walkthrough
|
|
of diagnosing indexing pressure rejections.
|
|
|
|
[discrete]
|
|
[[prevent-rejected-requests]]
|
|
==== Prevent rejected requests
|
|
|
|
**Fix high CPU and memory usage**
|
|
|
|
If {es} regularly rejects requests and other tasks, your cluster likely has high
|
|
CPU usage or high JVM memory pressure. For tips, see <<high-cpu-usage>> and
|
|
<<high-jvm-memory-pressure>>.
|