github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 14:47:19 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Doc] update queue.max_bytes and queue.page_capacity (#13833) (#13910)

Browse source 
Fixed: #8650, #10718

Backport PR #13833 to 7.17 branch

Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
(cherry picked from commit d71553e5d7)

Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
This commit is contained in:
github-actions[bot] 2022-03-18 11:05:27 +00:00 • committed by GitHub
parent 3300df77b0
commit 59a30d8b39
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 6 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/static/persistent-queues.asciidoc vendored
View file

@ -42,7 +42,9 @@ TIP: If you want to define values for a specific pipeline, use <<multiple-pipeli
`queue.type`:: Specify `persisted` to enable persistent queues. By default, persistent queues are disabled (default: `queue.type: memory`).
`path.queue`:: The directory path where the data files will be stored. By default, the files are stored in `path.data/queue`.
`queue.page_capacity`:: The maximum size of a queue page in bytes. The queue data consists of append-only files called "pages". The default size is 64mb. Changing this value is unlikely to have performance benefits.
`queue.page_capacity`:: The queue data consists of append-only files called "pages." This value sets the maximum size of a queue page in bytes.
The default size of 64mb is a good value for most users, and changing this value is unlikely to have performance benefits.
If you change the page capacity of an existing queue, the new size applies only to the new page.
`queue.drain`:: Specify `true` if you want Logstash to wait until the persistent queue is drained before shutting down. The amount of time it takes to drain the queue depends on the number of events that have accumulated in the queue. Therefore, you should avoid using this setting unless the queue, even when full, is relatively small and can be drained quickly.
`queue.max_events`:: The maximum number of events not yet read by the pipeline worker. The default is 0 (unlimited).
We use this setting for internal testing.
@ -54,8 +56,10 @@ queue will be sized at the value of `queue.max_bytes` specified in
The default is 1024mb (1gb).
+
Be sure that your disk has sufficient capacity to handle the cumulative total of `queue.max_bytes` across all persistent queues.
The total of `queue.max.bytes` for _all_ queues should be
The total of `queue.max_bytes` for _all_ queues should be
lower than the capacity of your disk.
We do not check to see if the disk has enough capacity for `queue.max_bytes` for all queues.
The space check at startup only verifies that each queue has sufficient capacity to reach its `queue.max_bytes`.
+
TIP: If you are using persistent queues to protect against data loss, but don't
require much buffering, you can set `queue.max_bytes` to a smaller value.