mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-06-28 17:34:17 -04:00
175 lines
6.4 KiB
Text
175 lines
6.4 KiB
Text
[[index-modules-slowlog]]
|
|
== Slow Log
|
|
|
|
[discrete]
|
|
[[search-slow-log]]
|
|
=== Search Slow Log
|
|
|
|
Shard level slow search log allows to log slow search (query and fetch
|
|
phases) into a dedicated log file.
|
|
|
|
Thresholds can be set for both the query phase of the execution, and
|
|
fetch phase, here is a sample:
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
index.search.slowlog.threshold.query.warn: 10s
|
|
index.search.slowlog.threshold.query.info: 5s
|
|
index.search.slowlog.threshold.query.debug: 2s
|
|
index.search.slowlog.threshold.query.trace: 500ms
|
|
|
|
index.search.slowlog.threshold.fetch.warn: 1s
|
|
index.search.slowlog.threshold.fetch.info: 800ms
|
|
index.search.slowlog.threshold.fetch.debug: 500ms
|
|
index.search.slowlog.threshold.fetch.trace: 200ms
|
|
--------------------------------------------------
|
|
|
|
All of the above settings are _dynamic_ and can be set for each index using the
|
|
<<indices-update-settings, update indices settings>> API. For example:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /my-index-000001/_settings
|
|
{
|
|
"index.search.slowlog.threshold.query.warn": "10s",
|
|
"index.search.slowlog.threshold.query.info": "5s",
|
|
"index.search.slowlog.threshold.query.debug": "2s",
|
|
"index.search.slowlog.threshold.query.trace": "500ms",
|
|
"index.search.slowlog.threshold.fetch.warn": "1s",
|
|
"index.search.slowlog.threshold.fetch.info": "800ms",
|
|
"index.search.slowlog.threshold.fetch.debug": "500ms",
|
|
"index.search.slowlog.threshold.fetch.trace": "200ms"
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[setup:my_index]
|
|
|
|
By default thresholds are disabled (set to `-1`).
|
|
|
|
The logging is done on the shard level scope, meaning the execution of a
|
|
search request within a specific shard. It does not encompass the whole
|
|
search request, which can be broadcast to several shards in order to
|
|
execute. Some of the benefits of shard level logging is the association
|
|
of the actual execution on the specific machine, compared with request
|
|
level.
|
|
|
|
|
|
The search slow log file is configured in the `log4j2.properties` file.
|
|
|
|
[discrete]
|
|
==== Identifying search slow log origin
|
|
|
|
It is often useful to identify what triggered a slow running query.
|
|
To include information about the user that triggered a slow search,
|
|
use the `index.search.slowlog.include.user` setting.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /my-index-000001/_settings
|
|
{
|
|
"index.search.slowlog.include.user": true
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[setup:my_index]
|
|
|
|
This will result in user information being included in the slow log.
|
|
|
|
[source,js]
|
|
---------------------------
|
|
{
|
|
"@timestamp": "2024-02-21T12:42:37.255Z",
|
|
"log.level": "WARN",
|
|
"auth.type": "REALM",
|
|
"elasticsearch.slowlog.id": "tomcat-123",
|
|
"elasticsearch.slowlog.message": "[index6][0]",
|
|
"elasticsearch.slowlog.search_type": "QUERY_THEN_FETCH",
|
|
"elasticsearch.slowlog.source": "{\"query\":{\"match_all\":{\"boost\":1.0}}}",
|
|
"elasticsearch.slowlog.stats": "[]",
|
|
"elasticsearch.slowlog.took": "747.3micros",
|
|
"elasticsearch.slowlog.took_millis": 0,
|
|
"elasticsearch.slowlog.total_hits": "1 hits",
|
|
"elasticsearch.slowlog.total_shards": 1,
|
|
"user.name": "elastic",
|
|
"user.realm": "reserved",
|
|
"ecs.version": "1.2.0",
|
|
"service.name": "ES_ECS",
|
|
"event.dataset": "elasticsearch.index_search_slowlog",
|
|
"process.thread.name": "elasticsearch[runTask-0][search][T#5]",
|
|
"log.logger": "index.search.slowlog.query",
|
|
"elasticsearch.cluster.uuid": "Ui23kfF1SHKJwu_hI1iPPQ",
|
|
"elasticsearch.node.id": "JK-jn-XpQ3OsDUsq5ZtfGg",
|
|
"elasticsearch.node.name": "node-0",
|
|
"elasticsearch.cluster.name": "distribution_run"
|
|
}
|
|
|
|
---------------------------
|
|
// NOTCONSOLE
|
|
|
|
If a call was initiated with an `X-Opaque-ID` header, then the ID is included
|
|
in Search Slow logs in the **elasticsearch.slowlog.id** field. See
|
|
<<x-opaque-id, X-Opaque-Id HTTP header>> for details and best practices.
|
|
|
|
[discrete]
|
|
[[index-slow-log]]
|
|
=== Index Slow log
|
|
|
|
The indexing slow log, similar in functionality to the search slow
|
|
log. The log file name ends with `_index_indexing_slowlog.json`. Log and
|
|
the thresholds are configured in the same way as the search slowlog.
|
|
Index slowlog sample:
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
index.indexing.slowlog.threshold.index.warn: 10s
|
|
index.indexing.slowlog.threshold.index.info: 5s
|
|
index.indexing.slowlog.threshold.index.debug: 2s
|
|
index.indexing.slowlog.threshold.index.trace: 500ms
|
|
index.indexing.slowlog.source: 1000
|
|
--------------------------------------------------
|
|
|
|
All of the above settings are _dynamic_ and can be set for each index using the
|
|
<<indices-update-settings, update indices settings>> API. For example:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /my-index-000001/_settings
|
|
{
|
|
"index.indexing.slowlog.threshold.index.warn": "10s",
|
|
"index.indexing.slowlog.threshold.index.info": "5s",
|
|
"index.indexing.slowlog.threshold.index.debug": "2s",
|
|
"index.indexing.slowlog.threshold.index.trace": "500ms",
|
|
"index.indexing.slowlog.source": "1000"
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[setup:my_index]
|
|
|
|
To include information about the user that triggered a slow indexing event,
|
|
use the `index.indexing.slowlog.include.user` setting.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /my-index-000001/_settings
|
|
{
|
|
"index.indexing.slowlog.include.user": true
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[setup:my_index]
|
|
|
|
By default Elasticsearch will log the first 1000 characters of the _source in
|
|
the slowlog. You can change that with `index.indexing.slowlog.source`. Setting
|
|
it to `false` or `0` will skip logging the source entirely, while setting it to
|
|
`true` will log the entire source regardless of size. The original `_source` is
|
|
reformatted by default to make sure that it fits on a single log line. If preserving
|
|
the original document format is important, you can turn off reformatting by setting
|
|
`index.indexing.slowlog.reformat` to `false`, which will cause the source to be
|
|
logged "as is" and can potentially span multiple log lines.
|
|
|
|
The index slow log file is configured in the `log4j2.properties` file.
|
|
|
|
[discrete]
|
|
=== Slow log levels
|
|
|
|
You can mimic the search or indexing slow log level by setting appropriate
|
|
threshold making "more verbose" loggers to be switched off.
|
|
If for instance we want to simulate `index.indexing.slowlog.level: INFO`
|
|
then all we need to do is to set
|
|
`index.indexing.slowlog.threshold.index.debug` and `index.indexing.slowlog.threshold.index.trace` to `-1`.
|