Commit graph

864 commits

Author SHA1 Message Date
DeDe Morton
dd8e6b6d41 Update monitoring api docs for multiple pipelines
Fixes #7354
2017-06-09 00:07:58 +00:00
Pier-Hugues Pellerin
42e9c0d758 Extraction of the document from ruby to actual asciidoc file
This PR is based on the proposal from #6846, it does the following

- Change the format of the plugin help page
- Create an external file for each plugin type that will get included by the plugin
- Add a script to generate the PR on each plugin repo with the static doc
- Add a script `ci/ci_docs.sh` that will be used to generate the list of plugins that can be installed in logstash from all the available plugins from the community organization

This PR is the building block for change in https://github.com/elastic/docs-tools/issues/3

Fixes #6941
2017-06-08 20:14:36 +00:00
DeDe Morton
8e491652e5 Edit multiple pipelines doc
Fixes #7337
2017-06-07 19:57:12 +00:00
Suyog Rao
dcd79d2f57 bump master version to 6.0.0-alpha3 (#7343)
* bump master version to 6.0.0-alpha3
2017-06-06 11:51:02 -07:00
Suyog Rao
9d19809436 add release notes for 6.0.0-alpha2 (#7322)
* add release notes for 6.0.0-alpha2

* Call it alpha2 in link

* Re-arrange breaking notes
2017-06-06 08:00:46 -07:00
Aaron Mildenstein
f443dae5fe Merge feature/modules into master (#7284)
* My changes (#7218)

* First upstream PR commit (#7172)

No tests yet.  Just for code review for now

* move all inner classes to their own folder + client and importer

* Fixes and tests (#7228)

Add tests for the `LogStash::Modules:CLIParser` class in `cli_parser.rb`
Fix a typo in `cli_parser.rb` (`uparsed` vs `unparsed`)
Fix a bad variable name found by testing in `cli_parser.rb` and update the error message accordingly in `en.yml`

* Remove fb_modules (#7280)

* fixes to import index-pattern & var updates & savedsearch capability (#7283)

* fixes to import index-pattern & var updates & savedsearch capability

fixes to import index-pattern & var updates

add savedsearch capability

* minimise merge conflicts with PR End-to-End test with filebeat apache2

* End-to-End test with filebeat apache2 (#7279)

This is a first run, but data flows from filebeat through Elasticsearch.

Template uploads from `$LS_HOME/modules/MODULENAME/configuration/elasticsearch/MODULENAME.json`

Specifying `--modules filebeat` from the command-line, with `-M "filebeat.var.elasticsearch.output.host=localhost:9200"`

Some of the saved searches don't get uploaded. @guyboertje is on this already.

The logstash configuration needs tweaking to allow receiving both access logs _and_ error logs.  The dashboards and visualizations all seem to expect the presence of both.

Set default to `localhost` in `elasticsearch_client.rb`

Changed command-line variable parsing to allow for a variable with only `modulename.key.subkey=value`, and updated the error message accordingly.

First draft of the filebeat module, as extracted from filebeat 5.4.0

* Add documentation for Modules

This is specific to the Master branch.  Multiple modules will not be supported in 5.5.

* Add READMEs and prune post-code comments

* Add comment regarding the variable name `modul`

Also, fix the default username for the Elasticsearch output in Logstash.  The default x-pack credentials are `elastic:changeme` rather than `elasticsearch:changeme`

* add cef module files (#7292)

* fixes from reviews of PR #7284
2017-06-05 18:49:35 +01:00
Suyog Rao
18627ba85f Add multipipelines doc 2017-06-04 20:23:09 -07:00
DeDe Morton
6b8b17a2fd Add float tag to fix doc strucure
Fixes #7308
2017-06-04 23:27:53 +00:00
Joao Duarte
bed8b8a084 support multiple pipelines in one logstash instance
* add multi_local source for multi pipelines
* introduce pipelines.yml
* introduce PipelineSettings class
* support reloading of pipeline parameters
* fix pipeline api call for _node/pipelines
* inform user pipelines.yml is ignored if -e or -f is enabled
2017-05-30 09:47:53 +01:00
DeDe Morton
34014fbb27 Change API docs to use curl examples instead of console syntax
Fixes #7208
2017-05-27 00:40:09 +00:00
DeDe Morton
78a974ece4 Fix bad edit link (#7216) 2017-05-25 11:27:10 -07:00
DeDe Morton
8b432094b9 Add edit URLs back into index.asciidoc
Fixes #7198
2017-05-25 18:10:55 +00:00
DeDe Morton
1ce0af8091 Add changes from review
Fixes #7180
2017-05-24 22:16:33 +00:00
DeDe Morton
9b857fa0de Clarify doc to indicate that offline install expects a file URI
Fixes #7180
2017-05-24 22:16:33 +00:00
Suyog Rao
cfecf81091 Only glob *.conf by default in conf.d folder (#7130)
* Only glob *.conf by default in conf.d folder

Fixes #6979, Fixes #1831
2017-05-19 11:26:10 -07:00
Armin
cffabc7a59 #7083 document logstash.yml ENV interpolation
Fixes #7084
2017-05-12 11:07:24 +00:00
Joao Duarte
d17a77e76d bump version to 6.0.0-alpha2 2017-05-11 18:18:29 +01:00
Tal Levy
458910bcb9 Introduce DeadLetterQueue to the Execution Context (#6894)
* Introduce a DeadLetterQueueFactory

DeadLetterQueueFactory is a static class that keeps
a static collection of DeadLetterQueueWriteManagers per
pipeline that has plugins requesting to use it.

* DeadLetterQueue was added as a first-class field in the execution context that input/filter/output plugins can leverage
2017-05-09 14:40:59 -07:00
Suyog Rao
69f5e46b0b [docs] Add release notes to 6.0 alpha1 2017-05-09 09:56:27 -07:00
Suyog Rao
cde8f88e47 Add -prerelease to yum sources 2017-05-09 09:39:51 -07:00
Suyog Rao
8ab7e2372a [docs] Mark as prerelease 2017-05-09 09:34:59 -07:00
Suyog Rao
553dd60819 Add 6.0.0-alpha1 release notes 2017-05-09 09:03:33 -07:00
DeDe Morton
45affa3f6e Add changes from the review
Fixes #6988
2017-05-05 01:54:28 +00:00
DeDe Morton
a912fea0f1 Document popular processing capabilities
Fixes #6988
2017-05-05 01:54:27 +00:00
DeDe Morton
31b2a36346 Add note to clarify at-least-one guarantee statement
Fixes #7029
2017-05-04 20:31:17 +00:00
DeDe Morton
23e195b48b Replace images in deployment guide
Fixes #7025
2017-05-04 16:42:39 +00:00
Toby McLaughlin
a76375d094 Add docs for settings via env vars on Docker (#7015) 2017-05-03 22:21:44 -07:00
DeDe Morton
b1a0af6103 Update LS deployment docs with Alvin's content
Fixes from review

Fixes #7005
2017-05-02 22:07:56 +00:00
DeDe Morton
e728b5eb4a Remove beta/experimental flags for features that are GA in 5.4
Fixes #6989
2017-04-28 23:20:48 +00:00
DeDe Morton
163d27d78f Merge pull request #6832 from dedemorton/fbm_docs_issue#6542
Add docs about configuring LS for use with Filebeat modules
2017-04-27 15:47:42 -07:00
DeDe Morton
c6926b9eaa Warn users not to use mutliline codec when multiple hosts are configured
Add changes from review

Fixes #6947
2017-04-24 22:49:02 +00:00
Mathieu Parent
dfbd637588 Persistent log4j2 configuration
Fixes #6905
2017-04-20 17:43:10 +00:00
DeDe Morton
4fedae80da Add fixes from the review 2017-04-17 14:31:20 -07:00
DeDe Morton
9128754b2b Add docs about configuring LS for use with Filebeat modules 2017-04-07 15:45:22 -07:00
Suyog Rao
51e3ec62b4 Add Logstash configs for filebeat modules (#6791)
* Add Logstash configs for filebeat modules

Hand transpiled Logstash configs for FB modules to be included in
the docs for 5.3.

See FB modules for more details: https://github.com/elastic/beats/tree/master/filebeat/module

* Fix newlines

* Remove multiline, fix bugs
2017-04-07 14:21:42 -07:00
Christopher Fujii
166170d807 Update glob-support.asciidoc
Fixed typo

Fixes #6876
2017-04-04 13:48:14 -04:00
Pier-Hugues Pellerin
4344e4b613 Allow Logstash to accept a PROXY configuration
Logstash's plugin manager will now follow proxy configuration from the environment.
If you configure `http_proxy` and `https_proxy`, the manager will now use theses information for all the ruby http
connection and will also pass that information down to maven.

Fixes: #6619, #6528

Fixes #6825
2017-03-31 14:20:38 -04:00
Pier-Hugues Pellerin
f74a7c9d56 Make sure prepare_offline_pack command only accept a filename
This PR changes the behavior of the command when you were specify a directory
instead of a filename, if the directory exists it would have deleted the
directory and the files in it.

The new flow and validation is now safer, if you specify a directory
Logstash will tell you to specify a complete filename.

If the ZIP extension is missing, the command line will ask you to make
sure you target a right filename.

If the output file already exist it will *not* delete it but instead
will return a warning message, you can force a delete by using the
`--overwrite` option.

Fixes: #6862

Fixes #6861
2017-03-31 11:05:17 -04:00
Josh Soref
a06dc211ff Spelling fixes (#6806)
Spelling fixes across the board.
2017-03-29 19:33:26 -07:00
DeDe Morton
a2436eb84b Clarify how cli options affect settings file
Fixes #6784
2017-03-02 20:29:59 -05:00
DeDe Morton
713309122a Make location of bin/logstash more obvious and fix TOC org
Fixes #6784
2017-03-02 20:29:58 -05:00
DeDe Morton
155c522279 Rename Event Dependent Configuration chapter
Fixes #6767
2017-03-02 19:31:49 -05:00
DeDe Morton
8bc2ce1e64 Add info about shutting down Logstash
Fixes #6771
2017-03-02 18:45:30 -05:00
DeDe Morton
c7fb20a656 Add conditional coding for install instructions
Fixes #6775
2017-03-02 13:07:49 -05:00
DeDe Morton
da6099b2f1 Add fixes from review
Fixes #6761
2017-03-02 12:55:46 -05:00
DeDe Morton
ebbf36f75f Add at least once delivery to persistent queue docs
Fixes #6761
2017-03-02 12:55:45 -05:00
Krishna Kalubandi
c1862b630a Update setting-up-logstash.asciidoc
Fixed typo

Fixes #6763
2017-02-28 16:29:06 -05:00
DeDe Morton
ad5cc81f96 Fix URL
Fixes #6768
2017-02-28 15:31:03 -05:00
DeDe Morton
af5aa0890a Fix typo 2017-02-28 11:41:47 -08:00
Aaron Mildenstein
b998d41cb0 Add note about JAVA_HOME to docs
Based on a suggestion to update the documentation: https://discuss.elastic.co/t/logstash-not-seeing-conf-files-in-conf-d-dir/75931/9

Fixes #6751
2017-02-23 16:41:02 -05:00