update screenshots and edit docs for search profiler (#36589)
|
@ -1,52 +1,22 @@
|
|||
[role="xpack"]
|
||||
[[profiler-getting-started]]
|
||||
|
||||
ifndef::gs-mini[]
|
||||
=== Getting Started
|
||||
endif::gs-mini[]
|
||||
|
||||
ifdef::gs-mini[]
|
||||
== Getting Started
|
||||
endif::gs-mini[]
|
||||
The {searchprofiler} is automatically enabled in {kib}. Go to *Dev Tools > Search Profiler*
|
||||
to get started.
|
||||
|
||||
The {searchprofiler} is automatically enabled in {kib}. It is located under the
|
||||
*Dev Tools* tab in {kib}.
|
||||
|
||||
[[first-profile]]
|
||||
To start profiling queries:
|
||||
|
||||
. Open Kibana in your web browser and log in. If you are running Kibana
|
||||
locally, go to `http://localhost:5601/`.
|
||||
|
||||
. Click **DevTools** in the side navigation to open the {searchprofiler}.
|
||||
Console is the default tool to open when first accessing DevTools.
|
||||
+
|
||||
image::dev-tools/searchprofiler/images/gs1.png["Opening DevTools"]
|
||||
+
|
||||
On the top navigation bar, click the second item: *Search Profiler*
|
||||
+
|
||||
image::dev-tools/searchprofiler/images/gs2.png["Opening the {searchprofiler}"]
|
||||
|
||||
. This opens the {searchprofiler} interface.
|
||||
+
|
||||
image::dev-tools/searchprofiler/images/gs3.png["{searchprofiler} Interface"]
|
||||
|
||||
. Replace the default `match_all` query with the query you want to profile and click *Profile*.
|
||||
+
|
||||
image::dev-tools/searchprofiler/images/gs4.png["Profiling the match_all query"]
|
||||
+
|
||||
{searchprofiler} displays the names of the indices searched, the shards in each index,
|
||||
and how long the query took. The following example shows the results of profiling
|
||||
the match_all query. Three indices were searched: `.monitoring-kibana-2-2016.11.30`,
|
||||
`.monitoring-data-2` and `test`.
|
||||
+
|
||||
If we take a closer look at the information for the test index, we can see from the
|
||||
Cumulative Time that the query took 0.132ms to execute. Of the five shards in the
|
||||
index (`DWZD0iosQNeJMTvb4q1JDw` 0 through 5), shard 3 was the slowest (0.053ms), followed by shard 2 (0.038ms). Shards are
|
||||
sorted by their time in descending order.
|
||||
+
|
||||
image::dev-tools/searchprofiler/images/gs5.png["Profile details for the test index"]
|
||||
+
|
||||
and how long it took for the query to complete. To try it out, replace the default `match_all` query
|
||||
with the query you want to profile and click *Profile*.
|
||||
|
||||
The following example shows the results of profiling the `match_all` query.
|
||||
If we take a closer look at the information for the `.kibana_1` sample index, the
|
||||
Cumulative Time field shows us that the query took 0.067ms to execute.
|
||||
|
||||
[role="screenshot"]
|
||||
image::dev-tools/searchprofiler/images/query.png["{searchprofiler} example"]
|
||||
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The Cumulative Time metric is the sum of individual shard times.
|
||||
|
@ -59,14 +29,21 @@ While the Cumulative Time metric is useful for comparing the performance of your
|
|||
indices and shards, it doesn't necessarily represent the actual physical query times.
|
||||
====
|
||||
|
||||
. To view more detailed profiling information for a shard, click the Expand button.
|
||||
This displays details about the query component(s) that ran on the shard.
|
||||
+
|
||||
In this example, there is a single `"MatchAllDocsQuery"` that ran on the shard.
|
||||
Since it was the only query run, it took 100% of the time. When you mouse over
|
||||
a row, the {searchprofiler} displays additional information about the query component."
|
||||
+
|
||||
image::dev-tools/searchprofiler/images/gs6.png["Profile details for the first shard"]
|
||||
+
|
||||
This panel shows the timing breakdown of low-level Lucene methods. For more information,
|
||||
see the reference docs for timing breakdowns in {ref}/search-profile-queries.html[Profiling queries].
|
||||
You can select the name of the shard and then click *View details* to see more profiling information,
|
||||
including details about the query component(s) that ran on the shard, as well as the timing
|
||||
breakdown of low-level Lucene methods. For more information, see {ref}/search-profile-queries.html[Profiling queries].
|
||||
|
||||
[float]
|
||||
=== Index and type filtering
|
||||
|
||||
By default, all queries executed by the {searchprofiler} are sent
|
||||
to `GET /_search`. It searches across your entire cluster (all indices, all types).
|
||||
|
||||
If you need to query a specific index or type (or several), you can use the Index
|
||||
and Type filters at the top left.
|
||||
|
||||
In the following example, the query is executed against the indices `test` and `kibana_1`
|
||||
and the type `my_type`. This is equivalent making a request to `GET /test,kibana_1/my_type/_search`.
|
||||
|
||||
[role="screenshot"]
|
||||
image::dev-tools/searchprofiler/images/filter.png["Filtering by index and type"]
|
|
@ -1,19 +1,19 @@
|
|||
[role="xpack"]
|
||||
[[xpack-profiler]]
|
||||
= Profiling your Queries and Aggregations
|
||||
= Profiling queries and aggregations
|
||||
|
||||
[partintro]
|
||||
--
|
||||
Elasticsearch has a powerful profiler API which can be used to inspect and analyze
|
||||
your search queries. The response, however, is a very large JSON blob which is
|
||||
difficult to analyze by hand.
|
||||
{es} has a powerful {ref}/search-profile.html[Profile API] which can be used to inspect and analyze
|
||||
your search queries. The response returns a large JSON blob, which can be
|
||||
difficult to analyze manually.
|
||||
|
||||
The {searchprofiler} tool can transform this JSON output
|
||||
into a visualization that is easy to navigate, allowing you to diagnose and debug
|
||||
poorly performing queries much faster.
|
||||
|
||||
|
||||
image::dev-tools/searchprofile/images/overview.png["{searchprofiler} Visualization"]
|
||||
[role="screenshot"]
|
||||
image::dev-tools/searchprofiler/images/overview.png["{searchprofiler} Visualization"]
|
||||
|
||||
--
|
||||
|
||||
|
|
Before Width: | Height: | Size: 9.5 KiB After Width: | Height: | Size: 274 KiB |
Before Width: | Height: | Size: 39 KiB |
Before Width: | Height: | Size: 114 KiB After Width: | Height: | Size: 368 KiB |
Before Width: | Height: | Size: 25 KiB |
Before Width: | Height: | Size: 31 KiB |
Before Width: | Height: | Size: 102 KiB |
Before Width: | Height: | Size: 30 KiB |
Before Width: | Height: | Size: 70 KiB |
Before Width: | Height: | Size: 6.8 KiB |
Before Width: | Height: | Size: 80 KiB After Width: | Height: | Size: 380 KiB |
Before Width: | Height: | Size: 32 KiB |
Before Width: | Height: | Size: 160 KiB After Width: | Height: | Size: 294 KiB |
Before Width: | Height: | Size: 143 KiB After Width: | Height: | Size: 364 KiB |
BIN
docs/dev-tools/searchprofiler/images/query.png
Normal file
After Width: | Height: | Size: 366 KiB |
|
@ -1,10 +1,10 @@
|
|||
[role="xpack"]
|
||||
[[xpack-profiler]]
|
||||
== Profiling your Queries and Aggregations
|
||||
== Profiling queries and aggregations
|
||||
|
||||
Elasticsearch has a powerful profiler API which can be used to inspect and analyze
|
||||
your search queries. The response, however, is a very large JSON blob which is
|
||||
difficult to analyze by hand.
|
||||
{es} has a powerful {ref}/search-profile.html[Profile API] which can be used to inspect and analyze
|
||||
your search queries. The response returns a large JSON blob, which can be
|
||||
difficult to analyze manually.
|
||||
|
||||
The {searchprofiler} tool can transform this JSON output
|
||||
into a visualization that is easy to navigate, allowing you to diagnose and debug
|
||||
|
@ -18,5 +18,3 @@ include::getting-started.asciidoc[]
|
|||
include::more-complicated.asciidoc[]
|
||||
|
||||
include::pasting.asciidoc[]
|
||||
|
||||
include::misc.asciidoc[]
|
||||
|
|
|
@ -1,16 +0,0 @@
|
|||
[role="xpack"]
|
||||
[[profiler-index]]
|
||||
=== Index and Type filtering
|
||||
|
||||
By default, all queries executed by the {searchprofiler} are sent
|
||||
to `GET /_search`. It searches across your entire cluster (all indices, all types).
|
||||
|
||||
If you need to query a specific index or type (or several), you can use the Index
|
||||
and Type filters at the top-left of the UI
|
||||
|
||||
In the following example, the query is executed against the indices `my_index` and `my_index1`
|
||||
and the type `my_type`:
|
||||
|
||||
image::dev-tools/searchprofiler/images/filter.png["Filtering by index and type"]
|
||||
|
||||
This is equivalent to `GET /my_index,my_index2/my_type/_search`.
|
|
@ -5,12 +5,12 @@
|
|||
To understand how the query trees are displayed inside the {searchprofiler},
|
||||
let's look at a more complicated query.
|
||||
|
||||
. Index the following data:
|
||||
. Index the following data via *Console*:
|
||||
+
|
||||
--
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST test/test/_bulk
|
||||
POST test/_bulk
|
||||
{"index":{}}
|
||||
{"name":"aaron","age":23,"hair":"brown"}
|
||||
{"index":{}}
|
||||
|
@ -25,16 +25,11 @@ POST test/test/_bulk
|
|||
// CONSOLE
|
||||
--
|
||||
|
||||
. Enter "test" in the Index filter above the query editor (the input box with a
|
||||
grayed-out `_all`). This restricts profiled queries to the `test` index.
|
||||
+
|
||||
--
|
||||
image::dev-tools/searchprofiler/images/gs7.png["Using the index filter"]
|
||||
--
|
||||
. From the {searchprofiler}, enter "test" in the Index field above the query editor to restrict profiled
|
||||
queries to the `test` index.
|
||||
|
||||
. Replace the default `match_all` query with a query that has two sub-query
|
||||
components and includes a simple aggregation. For example, copy and paste
|
||||
the following query into the query editor.
|
||||
. Replace the default `match_all` query in the query editor with a query that has two sub-query
|
||||
components and includes a simple aggregation, like the example below.
|
||||
+
|
||||
--
|
||||
[source,js]
|
||||
|
@ -71,53 +66,39 @@ the following query into the query editor.
|
|||
// NOTCONSOLE
|
||||
--
|
||||
|
||||
. Click *Profile* to profile the query and visualize the results.
|
||||
. Click *Profile* to profile the query and visualize the results.
|
||||
. Select the shard to view the query details.
|
||||
+
|
||||
--
|
||||
[role="screenshot"]
|
||||
image::dev-tools/searchprofiler/images/gs8.png["Profiling the more complicated query"]
|
||||
|
||||
As before, you'll see a list of shards appear in the center panel. You'll notice
|
||||
that the query was slightly slower (15ms vs 0.13ms) because it actually had to do
|
||||
a bit of work this time, unlike the `match_all` query.
|
||||
--
|
||||
|
||||
. Click the first shard's Expand button to view the query details.
|
||||
+
|
||||
--
|
||||
image::dev-tools/searchprofiler/images/gs9.png["Drilling into the first shard's details"]
|
||||
|
||||
--
|
||||
|
||||
You'll notice several interesting things in the results. The shard details
|
||||
contain a row for each query component:
|
||||
The detail view contains a row for each query component:
|
||||
|
||||
- The top-level `BooleanQuery` component corresponds to the bool in the query.
|
||||
- The second `BooleanQuery` corresponds to the terms query, which is internally
|
||||
converted to a `Boolean` of should clauses. It has two child queries that correspond
|
||||
to "sue" and "sally" from the terms query.
|
||||
- The `TermQuery` that's labeled with _"name:fred"_ corresponds to match: fred in the query.
|
||||
- The `TermQuery` that's labeled with "name:fred" corresponds to match: fred in the query.
|
||||
|
||||
If you look at the timings, you can see that "Self Time" and "Total Time" are no longer
|
||||
If you look at the time columns, you can see that "Self time" and "Total time" are no longer
|
||||
identical on all the rows. Self time represents how long the query component took to execute.
|
||||
Total time is the time a query component _and all its children_ took to execute.
|
||||
Therefore, queries like the Boolean queries often have larger Total than Self.
|
||||
|
||||
In particular, you can see that the `BooleanQuery` for "name:sue name:sally" took 3.8ms total,
|
||||
but 2.8ms of that was "self" time. That means 2.8ms was spent by the `BooleanQuery` itself, processing
|
||||
the should clauses
|
||||
Total time is the time a query component and all its children took to execute.
|
||||
Therefore, queries like the Boolean queries often have a larger total time than self time.
|
||||
|
||||
|
||||
==== Aggregations
|
||||
|
||||
This particular query also includes a aggregation (a `stats` agg on the `"age"` field).
|
||||
To view Aggregation profiling statistics, click the *Aggregation Profile* tab. This tab
|
||||
is only enabled if the query being profiled contains an aggregation.
|
||||
Click *Aggregation Profile* to view aggregation profiling statistics (this tab
|
||||
is only enabled if the query being profiled contains an aggregation).
|
||||
|
||||
|
||||
Select the name of the shard to view the aggregation details and timing breakdown.
|
||||
|
||||
[role="screenshot"]
|
||||
image::dev-tools/searchprofiler/images/gs10.png["Drilling into the first shard's details"]
|
||||
|
||||
Click a shard's Expand button to view the aggregation details. Hover over an
|
||||
aggregation row to view the timing breakdown.
|
||||
|
||||
For more information about how the {searchprofiler} works, how timings are calculated, and
|
||||
how to interpret various results, see
|
||||
{ref}/search-profile-queries.html[Profiling queries].
|
||||
|
|
|
@ -5,17 +5,18 @@
|
|||
The {searchprofiler} queries the cluster that the Kibana node is attached to.
|
||||
It does this by executing the query against the cluster and collecting the results.
|
||||
|
||||
This is convenient, but sometimes performance problems are temporal in nature. For example,
|
||||
a query might only be slow at certain time of day when many customers are using your system.
|
||||
But sometimes you may want to investigate performance problems that are temporal in nature.
|
||||
For example, a query might only be slow at certain time of day when many customers are using your system.
|
||||
You can setup a process to automatically profile slow queries when they occur and then
|
||||
save those profile responses for later analysis.
|
||||
|
||||
The {searchprofiler} supports this workflow by enabling you to paste the
|
||||
pre-captured JSON. The tool will detect that this is a profiler response JSON
|
||||
rather than a query, and render the visualization rather than querying the cluster.
|
||||
The {searchprofiler} supports this workflow by allowing you to paste the
|
||||
pre-captured JSON in the query editor. The {searchprofiler} will detect that you
|
||||
have entered a JSON response (rather than a query) and will just render the visualization,
|
||||
rather than querying the cluster.
|
||||
|
||||
To see how this works, copy and paste the following profile response into the
|
||||
query editor and click *Profile*:
|
||||
query editor and click *Profile*.
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
|
|