Commit graph

1319 commits

Author SHA1 Message Date
Dan Hermann
19554259b8 add option for PQ checkpoint retry
Fixes #10234
2019-01-02 21:15:38 +00:00
Leon J
8af0cddeee Update pipeline.conf (#10235) 2019-01-02 12:55:51 +00:00
Nik Everett
5a0e9a1d2e Docs: Bump version to 7.0.0-alpha2 after release 2018-12-20 14:50:43 -05:00
Karen Metts
c53cc7324b
Add placeholder for alpha2 release notes (#10228) 2018-12-20 09:52:20 -05:00
Dan Hermann
47e1bd252f Increase default number of threads reported by hot_threads
Fixes #10218
2018-12-18 16:54:12 +00:00
Karen Metts
904494fbac Move new field ref content to deepdive
Fixes #10057
2018-11-30 20:54:38 +00:00
Ry Biesemeyer
ee21c5ae13 FieldReference: add formal grammar definition docs
Fixes #10057
2018-11-30 20:54:38 +00:00
debadair
628500fe0d
[DOCS] Updated release state for 7.0.0-alpha1 (#10146)
* [DOCS] Updated release state for 7.0.0-alpha1

* Update index.asciidoc
2018-11-15 08:22:32 -08:00
Karen Metts
5a7220f946 Incorporate review comments
Fixes #10064
2018-11-05 14:46:52 +00:00
Karen Metts
ca90a1f9dc Minor wording tweaks while waiting for review comments
Fixes #10064
2018-11-05 14:46:52 +00:00
Karen Metts
118c0d94fc Restructure logging topic to clarify and make it scannable
Fixes #10064
2018-11-05 14:46:52 +00:00
Karen Metts
095fa9aa95 Add conditional wrappers to codec table rows
Fixes #10078
2018-11-01 21:46:15 +00:00
Karen Metts
1e914ebc7a Add conditional directives to handle plugins with no codec
Fixes #10078
2018-11-01 21:46:14 +00:00
Karen Metts
fdef42897d Fixed indents
Fixes #10065
2018-11-01 13:56:08 +00:00
Guy Boertje
3fdee027d9
Add SC and EB input TLS support for the Logstash ArcSight module (#10056)
* Add SC and EB input TLS support for the Logstash ArcSight module
I added the ssl for the smart connector (tcp) and ssl and sasl for the
event broker.
Needs verification on a current stack.

* So this is the actual extent of changes (I thought it was to simple)
Fixed the docs omission
Fixed the load path issue
Fixed the ERB binding context problem
Added some basic happy path tests

* remove module settings in logstash.yml

* Overwrite my logstash.yml with the content on master

* Add comments to yaml fixtures
2018-10-23 09:12:12 +01:00
Karen Metts
4e4c390ac5 CPM note
Fixes #10028
2018-09-28 13:29:36 +00:00
Lisa Cawley
f087fcb9ca
[DOCS] Synchronize location of Breaking Changes (#9980) 2018-09-26 14:06:09 -07:00
Alex Scoble
0ecdc95e42 Quotes around pipeline names with dashes
Added quotes surrounding pipeline references that have dashes as logstash will throw an error otherwise

Fixes #9903
2018-09-04 20:57:51 +00:00
Karen Metts
03182208ad Incorporate review comments
Fixes #9949
2018-08-31 15:57:07 +00:00
Karen Metts
db73386510 Rework pipeline viewer doc to support tree view
Fixes #9949
2018-08-31 15:57:06 +00:00
Karen Metts
894f7600e1 Add text offering Elasticsearch Servicehoste hosted es Revise text
Fixes #9892
2018-08-16 19:56:45 +00:00
lcawl
227177522d [DOCS] Fixes broken link to Stack security 2018-08-15 15:22:57 -07:00
lcawl
ab2506416c [DOCS] Fixes broken link to Stack Overview 2018-08-15 15:18:05 -07:00
Karen Metts
66ed8334c2 Updates to azure module doc
Fixes #9869
2018-08-04 01:06:13 +00:00
Karen Metts
89ffcd14fb Add azure module doc
Fixes #9510
2018-07-27 18:14:14 +00:00
dedemorton
6f471fa665 Light edit to remove passive voice and fix misplaced modifier
Fixes #9856
2018-07-25 18:48:17 +00:00
Lisa Cawley
f8a81f870a
[DOCS] Clarify methods for stopping Logstash (#9828) 2018-07-11 09:14:39 -07:00
Lisa Cawley
e2ba8c29d8
[DOCS] Removes alternative docker pull example (#9831) 2018-07-11 08:52:34 -07:00
Josh Soref
c6cd247ec3 Multiple spelling corrections (#9782)
* spelling: actually
* spelling: already
* spelling: concurrency
* spelling: constructor
* spelling: destinations
* spelling: different
* spelling: elasticsearch
* spelling: encoding
* spelling: error
* spelling: explicitly
* spelling: failings
* spelling: falsey
* spelling: guarantees
* spelling: having
* spelling: implementation
* spelling: logstash
* spelling: module
* spelling: multiple
* spelling: omitted
* spelling: overridden
* spelling: pipeline
* spelling: raspberry
* spelling: receive
* spelling: recommended
* spelling: registered
* spelling: registering
* spelling: shutdown
* spelling: signal
* spelling: specified
* spelling: successful
* spelling: successfully
* spelling: valuefied
* spelling: vector
2018-07-04 10:41:10 +01:00
Karen Metts
19c83d8333 Add backward compatibility info to breaking changes section
Fixes #9812
2018-07-03 00:21:46 +00:00
Shaunak Kashyap
ccbfd8a9ae
Encourage central management UI users to add the monitoring_user role as well (#9759) 2018-06-22 07:12:04 -07:00
lcawl
643fa7df78 [DOCS] Adds link in breaking changes 2018-06-15 12:39:48 -07:00
Toby McLaughlin
208e480d97
[DOCS] Update config steps for open X-Pack (#9674) 2018-06-04 09:57:57 +10:00
Lisa Cawley
e75156ea54
[DOCS] Moves management folder to docs (#9680) 2018-05-30 09:23:53 -07:00
Lisa Cawley
604ad7259f
[DOCS] Move monitoring folder to docs (#9677) 2018-05-30 09:02:39 -07:00
Lisa Cawley
ad16230a38
[DOCS] Moves security folder to docs (#9678) 2018-05-30 08:54:45 -07:00
Lisa Cawley
865389fd35
[DOCS] Move setup folder to docs (#9681) 2018-05-30 08:13:32 -07:00
Lisa Cawley
e9a2029f2e
[DOCS] Moves settings folder to docs (#9679) 2018-05-30 07:57:20 -07:00
Andrew Cholakian
9311f8b8d0 The initial implementation of inter-pipeline comms doesn't handle inter-pipeline dependencies correctly.
It just blocks and doesn't handle the concurrency situation. One can think of the network of connected pipelines as a DAG (We explicitly ask users not to create cycles in our docs). In fact there are two different correct answers to the pipeline shutdown and reload problem.

When reloading pipelines we should assume the user understands whatever changes they're making to the topology. If a downstream pipeline is to be removed, we can assume that's intentional. We don't lose any data from upstream pipelines since those block until that downstream pipeline is restored. To accomplish this none of the `PipelineBus` methods block by default.

When shutting down Logstash we must: 1.) not lose in-flight data, and 2.) not deadlock. The only way to do both is to shutdown the pipelines in order. All state changes happen simultaneously on all piping via multithreading. As a result we don't need to implement a Topological sort or other algorithm to order dependencies, we simply issue a shutdown to all pipelines, and have ones that are dependent block waiting for upstream pipelines.

This patch also correctly handles the case where multiple signals cause pipeline actions to be created simultaneously. If we see two concurrent creates or stops, one of those actions becomes a noop.

Currently the Logstash plugin API has lifecycle methods for starting and stopping, but not reloading. We need to call a different method when a `pipeline` input is stopped due to a pipeline reload vs an agent shutdown. Ideally, we would enrich our plugin API. In the interest of expedience here, however, I added a flag to the `PipelineBus` that changes the shutdown mode of `pipeline` inputs, to be either blocking or non-blocking. It is switched to blocking if a shutdown is triggered.

This also reverts b78d32dede in favor of a better more concurrent approach without mutexes

This is a forward port of https://github.com/elastic/logstash/pull/9650 to master / 6.x

Fixes #9656
2018-05-24 13:59:39 +00:00
Karen Metts
e786ca1126 Fix breaking changes doc structure
Fixes #9606
2018-05-21 23:56:30 +00:00
Lisa Cawley
41e0a71e39
[DOCS] Removes redundant index.asciidoc files (#9607) 2018-05-18 12:11:05 -07:00
Lisa Cawley
abb09c5d1f
[DOCS] Moves X-Pack configuration pages in table of contents (#9604) 2018-05-17 16:37:34 -07:00
DeDe Morton
ec75b20082 Update life-of-an-event.asciidoc
Fixes #9558
2018-05-15 01:19:35 +00:00
Andrew Cholakian
2d5235af7a Update breaking-changes.asciidoc
Fixes #9550
2018-05-11 16:16:26 +00:00
Andrew Cholakian
6902935468 Fix formatting of breaking changes to not split across pages
The current formatting results in multiple pages, which is confusing, as seen on https://www.elastic.co/guide/en/logstash/current/breaking-changes.html

Fixes #9550
2018-05-11 16:16:26 +00:00
Andrew Cholakian
f49ffad20d Improve breaking PQ change doc for 6.3.0
Once 6.3.0 is the newest release the phrasing should reflect that.

Fixes #9548
2018-05-09 20:06:27 +00:00
Andrew Cholakian
0af9a91e25 Improve phrasing of version ranges in PQ upgrade advice
Fixes #9546
2018-05-09 19:47:12 +00:00
Andrew Cholakian
d2b78e698e Add greater context to PQ backward compat breakage
Fixes #9545
2018-05-09 19:32:27 +00:00
Andrew Cholakian
ddbb6ad839 Use @karenzone 's improved wording
Fixes #9542
2018-05-09 18:59:29 +00:00
Andrew Cholakian
e66225fd48 Mention PQ issues in breaking changes
Fixes #9542
2018-05-09 18:59:29 +00:00