Commit graph

1406 commits

Author SHA1 Message Date
Karen Metts
fdef42897d Fixed indents
Fixes #10065
2018-11-01 13:56:08 +00:00
Guy Boertje
3fdee027d9
Add SC and EB input TLS support for the Logstash ArcSight module (#10056)
* Add SC and EB input TLS support for the Logstash ArcSight module
I added the ssl for the smart connector (tcp) and ssl and sasl for the
event broker.
Needs verification on a current stack.

* So this is the actual extent of changes (I thought it was to simple)
Fixed the docs omission
Fixed the load path issue
Fixed the ERB binding context problem
Added some basic happy path tests

* remove module settings in logstash.yml

* Overwrite my logstash.yml with the content on master

* Add comments to yaml fixtures
2018-10-23 09:12:12 +01:00
Karen Metts
4e4c390ac5 CPM note
Fixes #10028
2018-09-28 13:29:36 +00:00
Lisa Cawley
f087fcb9ca
[DOCS] Synchronize location of Breaking Changes (#9980) 2018-09-26 14:06:09 -07:00
Alex Scoble
0ecdc95e42 Quotes around pipeline names with dashes
Added quotes surrounding pipeline references that have dashes as logstash will throw an error otherwise

Fixes #9903
2018-09-04 20:57:51 +00:00
Karen Metts
03182208ad Incorporate review comments
Fixes #9949
2018-08-31 15:57:07 +00:00
Karen Metts
db73386510 Rework pipeline viewer doc to support tree view
Fixes #9949
2018-08-31 15:57:06 +00:00
Karen Metts
894f7600e1 Add text offering Elasticsearch Servicehoste hosted es Revise text
Fixes #9892
2018-08-16 19:56:45 +00:00
lcawl
227177522d [DOCS] Fixes broken link to Stack security 2018-08-15 15:22:57 -07:00
lcawl
ab2506416c [DOCS] Fixes broken link to Stack Overview 2018-08-15 15:18:05 -07:00
Karen Metts
66ed8334c2 Updates to azure module doc
Fixes #9869
2018-08-04 01:06:13 +00:00
Karen Metts
89ffcd14fb Add azure module doc
Fixes #9510
2018-07-27 18:14:14 +00:00
dedemorton
6f471fa665 Light edit to remove passive voice and fix misplaced modifier
Fixes #9856
2018-07-25 18:48:17 +00:00
Lisa Cawley
f8a81f870a
[DOCS] Clarify methods for stopping Logstash (#9828) 2018-07-11 09:14:39 -07:00
Lisa Cawley
e2ba8c29d8
[DOCS] Removes alternative docker pull example (#9831) 2018-07-11 08:52:34 -07:00
Josh Soref
c6cd247ec3 Multiple spelling corrections (#9782)
* spelling: actually
* spelling: already
* spelling: concurrency
* spelling: constructor
* spelling: destinations
* spelling: different
* spelling: elasticsearch
* spelling: encoding
* spelling: error
* spelling: explicitly
* spelling: failings
* spelling: falsey
* spelling: guarantees
* spelling: having
* spelling: implementation
* spelling: logstash
* spelling: module
* spelling: multiple
* spelling: omitted
* spelling: overridden
* spelling: pipeline
* spelling: raspberry
* spelling: receive
* spelling: recommended
* spelling: registered
* spelling: registering
* spelling: shutdown
* spelling: signal
* spelling: specified
* spelling: successful
* spelling: successfully
* spelling: valuefied
* spelling: vector
2018-07-04 10:41:10 +01:00
Karen Metts
19c83d8333 Add backward compatibility info to breaking changes section
Fixes #9812
2018-07-03 00:21:46 +00:00
Shaunak Kashyap
ccbfd8a9ae
Encourage central management UI users to add the monitoring_user role as well (#9759) 2018-06-22 07:12:04 -07:00
lcawl
643fa7df78 [DOCS] Adds link in breaking changes 2018-06-15 12:39:48 -07:00
Toby McLaughlin
208e480d97
[DOCS] Update config steps for open X-Pack (#9674) 2018-06-04 09:57:57 +10:00
Lisa Cawley
e75156ea54
[DOCS] Moves management folder to docs (#9680) 2018-05-30 09:23:53 -07:00
Lisa Cawley
604ad7259f
[DOCS] Move monitoring folder to docs (#9677) 2018-05-30 09:02:39 -07:00
Lisa Cawley
ad16230a38
[DOCS] Moves security folder to docs (#9678) 2018-05-30 08:54:45 -07:00
Lisa Cawley
865389fd35
[DOCS] Move setup folder to docs (#9681) 2018-05-30 08:13:32 -07:00
Lisa Cawley
e9a2029f2e
[DOCS] Moves settings folder to docs (#9679) 2018-05-30 07:57:20 -07:00
Andrew Cholakian
9311f8b8d0 The initial implementation of inter-pipeline comms doesn't handle inter-pipeline dependencies correctly.
It just blocks and doesn't handle the concurrency situation. One can think of the network of connected pipelines as a DAG (We explicitly ask users not to create cycles in our docs). In fact there are two different correct answers to the pipeline shutdown and reload problem.

When reloading pipelines we should assume the user understands whatever changes they're making to the topology. If a downstream pipeline is to be removed, we can assume that's intentional. We don't lose any data from upstream pipelines since those block until that downstream pipeline is restored. To accomplish this none of the `PipelineBus` methods block by default.

When shutting down Logstash we must: 1.) not lose in-flight data, and 2.) not deadlock. The only way to do both is to shutdown the pipelines in order. All state changes happen simultaneously on all piping via multithreading. As a result we don't need to implement a Topological sort or other algorithm to order dependencies, we simply issue a shutdown to all pipelines, and have ones that are dependent block waiting for upstream pipelines.

This patch also correctly handles the case where multiple signals cause pipeline actions to be created simultaneously. If we see two concurrent creates or stops, one of those actions becomes a noop.

Currently the Logstash plugin API has lifecycle methods for starting and stopping, but not reloading. We need to call a different method when a `pipeline` input is stopped due to a pipeline reload vs an agent shutdown. Ideally, we would enrich our plugin API. In the interest of expedience here, however, I added a flag to the `PipelineBus` that changes the shutdown mode of `pipeline` inputs, to be either blocking or non-blocking. It is switched to blocking if a shutdown is triggered.

This also reverts b78d32dede in favor of a better more concurrent approach without mutexes

This is a forward port of https://github.com/elastic/logstash/pull/9650 to master / 6.x

Fixes #9656
2018-05-24 13:59:39 +00:00
Karen Metts
e786ca1126 Fix breaking changes doc structure
Fixes #9606
2018-05-21 23:56:30 +00:00
Lisa Cawley
41e0a71e39
[DOCS] Removes redundant index.asciidoc files (#9607) 2018-05-18 12:11:05 -07:00
Lisa Cawley
abb09c5d1f
[DOCS] Moves X-Pack configuration pages in table of contents (#9604) 2018-05-17 16:37:34 -07:00
DeDe Morton
ec75b20082 Update life-of-an-event.asciidoc
Fixes #9558
2018-05-15 01:19:35 +00:00
Andrew Cholakian
2d5235af7a Update breaking-changes.asciidoc
Fixes #9550
2018-05-11 16:16:26 +00:00
Andrew Cholakian
6902935468 Fix formatting of breaking changes to not split across pages
The current formatting results in multiple pages, which is confusing, as seen on https://www.elastic.co/guide/en/logstash/current/breaking-changes.html

Fixes #9550
2018-05-11 16:16:26 +00:00
Andrew Cholakian
f49ffad20d Improve breaking PQ change doc for 6.3.0
Once 6.3.0 is the newest release the phrasing should reflect that.

Fixes #9548
2018-05-09 20:06:27 +00:00
Andrew Cholakian
0af9a91e25 Improve phrasing of version ranges in PQ upgrade advice
Fixes #9546
2018-05-09 19:47:12 +00:00
Andrew Cholakian
d2b78e698e Add greater context to PQ backward compat breakage
Fixes #9545
2018-05-09 19:32:27 +00:00
Andrew Cholakian
ddbb6ad839 Use @karenzone 's improved wording
Fixes #9542
2018-05-09 18:59:29 +00:00
Andrew Cholakian
e66225fd48 Mention PQ issues in breaking changes
Fixes #9542
2018-05-09 18:59:29 +00:00
Mathieu Martin
ce0cc8c12b
Give an example of a single line Hash. (#9505)
Single kv hashes look like Ruby hashes, and people get confused because they don't actually support commas, to separate each pair. This adds a more explicit example under the preferred multi-line hash example.
2018-05-02 10:35:55 -04:00
Lisa Cawley
6723601217
[DOCS] Removes X-Pack release notes and breaking changes (#9509) 2018-05-01 16:36:10 -07:00
Lisa Cawley
24a2792bae
[DOCS] Adds new installation package details (#9404) 2018-04-27 08:56:25 -07:00
Karen Metts
f6eccf081b Fix forked path exp
Fixes #9419
2018-04-26 20:07:46 +00:00
Karen Metts
47fd3a519c Update doc for ArcSite module
Fixes #9406
2018-04-26 16:28:47 +00:00
Jenkins CI
93cad10da1 Migrate x-pack-logstash source to logstash 2018-04-24 10:46:15 -05:00
Lisa Cawley
7824302c2b
[DOCS] Removes X-Pack installation info (#9407) 2018-04-19 12:59:46 -07:00
Karen Metts
bb823a5c58
Doc for pipeline-to-pipeline communication (#9385)
* Doc for pipeline-to-pipeline

* Incorporate review comments

* Add link for collector pattern
2018-04-17 19:53:02 -04:00
Andrew Cholakian
a1c0e417e5 Support for inter-pipeline comms with a new pipeline input/output
This also makes the load / reload cycle of pipelines threadsafe
and concurrent in the Agent class.

Fixes #9225
2018-04-10 23:48:58 +00:00
Karen Metts
06f697daec Add pipelines.yml and log4j2.properties to list of config files Add anchor to log4j2 topic heading for linking
Fixes #9321
2018-04-09 23:47:29 +00:00
Karen Metts
8ede70de8c Add back trailing lines deleted by editor
Fixes #9335
2018-04-09 23:39:30 +00:00
Karen Metts
417b34c95c Lower case logstash.yml heading
Fixes #9335
2018-04-09 23:39:30 +00:00
Karen Metts
f5cf0cd6b2 Change LS_HEAP_SIZE to reference JVM.options
Fixes #9335
2018-04-09 23:39:30 +00:00