mirror of
https://github.com/elastic/kibana.git
synced 2025-06-28 11:05:39 -04:00
418281b2d1
# Backport This will backport the following commits from `main` to `8.x`: - [[Docs] Update nav instructions and sample data installation to accommodate for the solution views (#199163)](https://github.com/elastic/kibana/pull/199163) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"florent-leborgne","email":"florent.leborgne@elastic.co"},"sourceCommit":{"committedDate":"2024-11-08T10:10:32Z","message":"[Docs] Update nav instructions and sample data installation to accommodate for the solution views (#199163)\n\nThis PR:\r\n- updates navigation instructions to accommodate for the navigation\r\nchanges related to solution views.\r\n- updates instructions for adding sample data to rely on the\r\nintegrations page instead of the home page, that only exists with the\r\nclassic solution view\r\n- updates references to the home page to avoid confusing users using one\r\nof the new solution views\r\n\r\nCloses: https://github.com/elastic/platform-docs-team/issues/529\r\nCloses: https://github.com/elastic/platform-docs-team/issues/540","sha":"8e7fb7a77e2399e3e946bbba0462517fd7dad0f7","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Docs","release_note:skip","v9.0.0","docs","v8.16.0","backport:version","v8.17.0"],"title":"[Docs] Update nav instructions and sample data installation to accommodate for the solution views","number":199163,"url":"https://github.com/elastic/kibana/pull/199163","mergeCommit":{"message":"[Docs] Update nav instructions and sample data installation to accommodate for the solution views (#199163)\n\nThis PR:\r\n- updates navigation instructions to accommodate for the navigation\r\nchanges related to solution views.\r\n- updates instructions for adding sample data to rely on the\r\nintegrations page instead of the home page, that only exists with the\r\nclassic solution view\r\n- updates references to the home page to avoid confusing users using one\r\nof the new solution views\r\n\r\nCloses: https://github.com/elastic/platform-docs-team/issues/529\r\nCloses: https://github.com/elastic/platform-docs-team/issues/540","sha":"8e7fb7a77e2399e3e946bbba0462517fd7dad0f7"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199163","number":199163,"mergeCommit":{"message":"[Docs] Update nav instructions and sample data installation to accommodate for the solution views (#199163)\n\nThis PR:\r\n- updates navigation instructions to accommodate for the navigation\r\nchanges related to solution views.\r\n- updates instructions for adding sample data to rely on the\r\nintegrations page instead of the home page, that only exists with the\r\nclassic solution view\r\n- updates references to the home page to avoid confusing users using one\r\nof the new solution views\r\n\r\nCloses: https://github.com/elastic/platform-docs-team/issues/529\r\nCloses: https://github.com/elastic/platform-docs-team/issues/540","sha":"8e7fb7a77e2399e3e946bbba0462517fd7dad0f7"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
121 lines
4.6 KiB
Text
121 lines
4.6 KiB
Text
[role="xpack"]
|
|
[[xpack-grokdebugger]]
|
|
== Debug grok expressions
|
|
|
|
You can build and debug grok patterns in the {kib} *Grok Debugger*
|
|
before you use them in your data processing pipelines. Grok is a pattern
|
|
matching syntax that you can use to parse arbitrary text and
|
|
structure it. Grok is good for parsing syslog, apache, and other
|
|
webserver logs, mysql logs, and in general, any log format that is
|
|
written for human consumption.
|
|
|
|
Grok patterns are supported in {es} {ref}/runtime.html[runtime fields], the {es}
|
|
{ref}/grok-processor.html[grok ingest processor], and the {ls}
|
|
{logstash-ref}/plugins-filters-grok.html[grok filter]. For syntax, see
|
|
{ref}/grok.html[Grokking grok].
|
|
|
|
The {stack} ships with more than 120 reusable grok patterns. For a complete
|
|
list of patterns, see
|
|
https://github.com/elastic/elasticsearch/tree/master/libs/grok/src/main/resources/patterns[{es}
|
|
grok patterns] and
|
|
https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns[{ls}
|
|
grok patterns].
|
|
|
|
Because
|
|
{es} and {ls} share the same grok implementation and pattern
|
|
libraries, any grok pattern that you create in the *Grok Debugger* will work
|
|
in both {es} and {ls}.
|
|
|
|
[float]
|
|
[[grokdebugger-getting-started]]
|
|
=== Get started
|
|
|
|
This example walks you through using the *Grok Debugger*. This tool
|
|
is automatically enabled in {kib}.
|
|
|
|
NOTE: If you're using {stack-security-features}, you must have the `manage_pipeline`
|
|
permission to use the Grok Debugger.
|
|
|
|
. Find the *Grok Debugger* by navigating to the *Developer tools* page using the
|
|
navigation menu or the <<kibana-navigation-search,global search field>>.
|
|
. In *Sample Data*, enter a message that is representative of the data that you
|
|
want to parse. For example:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
55.3.244.1 GET /index.html 15824 0.043
|
|
-------------------------------------------------------------------------------
|
|
|
|
. In *Grok Pattern*, enter the grok pattern that you want to apply to the data.
|
|
+
|
|
To parse the log line in this example, use:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}
|
|
-------------------------------------------------------------------------------
|
|
|
|
. Click **Simulate**.
|
|
+
|
|
You'll see the simulated event that results from applying the grok
|
|
pattern.
|
|
+
|
|
[role="screenshot"]
|
|
image::dev-tools/grokdebugger/images/grok-debugger-overview.png["Grok Debugger"]
|
|
|
|
|
|
//TODO: Update LS and ingest node docs with pointers to the new grok debugger. Replace references to the Heroku app.
|
|
|
|
[float]
|
|
[[grokdebugger-custom-patterns]]
|
|
=== Test custom patterns
|
|
|
|
If the default grok pattern dictionary doesn't contain the patterns you need,
|
|
you can define, test, and debug custom patterns using the *Grok Debugger*.
|
|
|
|
Custom patterns that you enter in the *Grok Debugger* are not saved. Custom patterns
|
|
are only available for the current debugging session and have no side effects.
|
|
|
|
Follow this example to define a custom pattern.
|
|
|
|
. In *Sample Data*, enter the following sample message:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
Jan 1 06:25:43 mailserver14 postfix/cleanup[21403]: BEF25A72965: message-id=<20130101142543.5828399CCAF@mailserver14.example.com>
|
|
-------------------------------------------------------------------------------
|
|
|
|
. Enter this grok pattern:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{MSG:syslog_message}
|
|
-------------------------------------------------------------------------------
|
|
+
|
|
Notice that the grok pattern references custom patterns called `POSTFIX_QUEUEID`
|
|
and `MSG`.
|
|
|
|
. Expand **Custom Patterns** and enter pattern definitions for the custom
|
|
patterns that you want to use in the grok expression. You must specify each pattern definition
|
|
on its own line.
|
|
+
|
|
For this example, you must specify pattern definitions
|
|
for `POSTFIX_QUEUEID` and `MSG`:
|
|
+
|
|
[source,ruby]
|
|
-------------------------------------------------------------------------------
|
|
POSTFIX_QUEUEID [0-9A-F]{10,11}
|
|
MSG message-id=<%{GREEDYDATA}>
|
|
-------------------------------------------------------------------------------
|
|
|
|
. Click **Simulate**.
|
|
+
|
|
You'll see the simulated output event that results from applying
|
|
the grok pattern that contains the custom pattern:
|
|
+
|
|
[role="screenshot"]
|
|
image::dev-tools/grokdebugger/images/grok-debugger-custom-pattern.png["Debugging a custom pattern"]
|
|
+
|
|
If an error occurs, you can continue iterating over
|
|
the custom pattern until the output matches the event
|
|
that you expect.
|