Commit graph

85 commits

Author SHA1 Message Date
kaisecheng
54b370ea48
Geoip database service (#12675)
geoip database service in xpack
dependency update and license note

Fixed: #12560
2021-02-18 14:18:28 +01:00
andsel
91996cf2a2 Fix Logstash pipelines management in case of slow loading pipelines or disabled webserver (#12571)
This commit avoid an error in gathering monitoring information when webserver is disabled or is not yet started;
which could happen with slow loading pipelines or no pipelines defined from the central management UI.
2021-02-16 05:54:12 -08:00
kaisecheng
4bc9dad69c
Remove ruby execution engine (#12517)
* remove Logstash::Pipeline
* remove logstash/event, Logstash::SignalEvent
* remove Engine.Ruby on java side
* remove config pipeline.java_execution
Fixed: #11236
2020-12-15 10:41:27 +01:00
kaisecheng
0f55de9c9e
add wildcard support in xpack pipeline id (#12370)
add wildcard support in xpack pipeline id
do the pattern matching with glob
add warning msg to wildcard with legacy api
check invalid pipeline in bootstrap
test cases for invalid checking
2020-11-03 11:02:07 +01:00
kaisecheng
999601cd90
replace direct access of hidden indices with system indices api (#12279)
* replace direct hidden indices access with system indices api

* fulfill backward compatibility

* fix log msg, rename class, simplify response handling

* modularise fetcher
2020-10-06 21:19:36 +02:00
Ry Biesemeyer
bba5e5dcc2 specs: don't start ES connection pool when only validating config
Accidentally succeeding at connecting to an HTTP resource that is not a real,
live Elasticsearch (such as an Elastic Cloud instance that has been shut down
and reaped) can cause client initialization to fail.
2020-08-04 02:24:59 -07:00
Colin Surprenant
87df15d236 ignore default username when no password is set
fixes a regression introduced with the api_key support for xpack monitoring and management in #11864 which disabled the possibility to not use any authentication by relying on the default options and only enabling monitoring for example. It now ignores the default username option when no password is explicitly set.
2020-07-13 14:54:10 -07:00
Colin Surprenant
f4ce80d956
add support for api_key authentication in xpack management and monitoring. (#11864) 2020-06-03 10:56:47 -04:00
Andrea Selva
832d597278
Refactor: move PipelineConfig from Ruby to Java (#11824)
Reimplement the Ruby class PipelinceConfig in Java trying to keep the method signatures to limit the changes in client code, this is a step of other that intend to move all the configuration code in Java language.
Having all that code in Java unlock some reasoning about how to better implement it and probably an improvement in performance during process startup.
Moved also the spec into a JUnit and fixed here and there the failing tests

Closes: #11824
2020-04-29 18:16:49 +02:00
Ry Biesemeyer
f1e301e0d7 settings: deprecate unit-less TimeValue values
We have "required" units for a variety of `TimeValue` settings when they are
provided as a `String`, but unquoted values in YAML have been passed through as
Integers, where we long assumed nanosecond units. This frequently leads to
surprise (e.g., when `config.reload.interval` is set to `60`, we consume 100%
of CPU in a tight loop trying to reload and re-parse the configs every 60
nanoseconds).

By making the setting retain the TimeValue object for the entirety of its
lifecycle, we can issue a deprecation notice the first time an Integer value is
encountered. As a secondary benefit, our usage of the setting value in code
becomes more clear since we are empowered to ask `TimeValue` for a numeric
value in a specific scale.

Fixes #11803
2020-04-20 16:15:49 +00:00
Karol Bucek
afce87a67e Fix: cloud_id not propagating from monitoring config
Fixes #11800
2020-04-16 18:52:42 +00:00
Andrea Selva
6a74872d50
Moved code out of LogStash::Compiler to org.logstash.config.ir.ConfigCompiler 2020-04-15 12:21:39 +02:00
Rob Bavey
2899c9f379 Handle Boolean Edges
Fixes #11779
2020-04-13 20:09:27 +00:00
Colin Surprenant
5a25c6f8e9 simplify batch classes, do not compute JE empty batches, refactor RE worker loop (#11737)
cleanup RubyArray "rawtypes"
remove all LinkedHashSet from batch and queue classes
avoid processing empty batches in Java worker loop
cleanup AckedReadBatch and MemoryReadBatch
refactor Ruby worker loop similar to Java Execution to not use batch merge
remove QueueBatch merge and replace LinkedHashSet with ArrayList
2020-04-02 16:26:54 -04:00
andsel
dfdc6f2aac Fixed flaky test, as fixed in packport PR #11641
Fixes #11679
2020-03-12 09:00:49 +00:00
João Duarte
e9c9865f40
Add apache and elastic license headers to source code files (#11673)
* add license header to ruby and java files
* add license header to erb and rake files
* add license headers to gradle files
2020-03-11 11:53:38 +00:00
andsel
3695580b92 Adaptations to internal collector to send data directly to monitoring cluster Close 11573
Fixes #11541
2020-02-28 14:26:19 +00:00
Karol Bucek
d8398351a2
Feat: x-pack cloud id/auth for monitoring/management (#11496)
resolves #11488
2020-01-14 22:06:50 +01:00
Joao Duarte
3294a4be61 refactor list of license_types
DRY up the list of license types as there were 10 places that listed the types explicitly

Fixes #11407
2019-12-10 12:31:16 +00:00
Mike Place
cc3c5ec00f
Refactoring of LIR serializer and exposing pipeline metrics (#10561)
* [WIP] Add ephemeral id and hash to pipeline stats

Refs #10119

* Fix incorrect hash id

* Add graph metrics

* Include node update

* LIR serializer refactor

* Remove commented code

* Remove more commented code

* Update spec

* Remove license and add encoding

* Style change to make code more vertical.

* Implement review suggestions

* Remove commented code

* Fix spec path

* Add cluster UUID

* Add graph?=true parameter

* Add options to pipelines/

* Use pluginmetadata

* Replicate change from java_pipeline to ruby pipeline
2019-05-01 12:45:11 +02:00
Jordan Johnson-Doyle
aa03eb3973 Inputs expect a NamespacedMetric, not the root metric instance
Fixes #10614
2019-04-11 13:10:18 +00:00
Jake Landis
0ca34f07bb Central management typeless API
This commit adopts Elasticsearch's typeless API for central management.

Relates: https://github.com/elastic/elasticsearch/issues/3863

Fixes #10421
2019-03-07 16:19:17 +00:00
Joao Duarte
115738ceef mute metrics_spec line 138 test
Fixes #10412
2019-02-07 16:25:45 +00:00
Joao Duarte
c1a78c0d59 mute frequently failing x-pack test
Fixes #10396
2019-02-06 09:39:55 +00:00
Ry Biesemeyer
328b83815c spec: make timing-sensitive specs a little more flexible
Fixes #10392
2019-02-06 07:48:57 +00:00
Ry Biesemeyer
149684d5b6 spec: noop refactor for clarity - use explicit metric_input subject instead of implicit subject - use rspec/wait full expectation syntax instead of be_falsey/be_truthy - eliminate unused shared examples - incorporate single-use shared examples
Fixes #10392
2019-02-06 07:48:56 +00:00
Colin Surprenant
6990d08be5
rename config option .url and .ca to .hosts and .certificate_authority (#10380) 2019-02-05 17:39:30 -05:00
Colin Surprenant
f08b8c5076
fix agent silent exit upon pipelines reloading (#10346) 2019-01-31 15:48:50 -05:00
João Duarte
fe7607abd4
only start monitoring pipeline after valid license (#10106)
Avoid unecessary logging errors and resource usage by only starting
the monitoring pipeline if we can validate the license.
2018-11-07 17:28:48 +00:00
Guy Boertje
3fdee027d9
Add SC and EB input TLS support for the Logstash ArcSight module (#10056)
* Add SC and EB input TLS support for the Logstash ArcSight module
I added the ssl for the smart connector (tcp) and ssl and sasl for the
event broker.
Needs verification on a current stack.

* So this is the actual extent of changes (I thought it was to simple)
Fixed the docs omission
Fixed the load path issue
Fixed the ERB binding context problem
Added some basic happy path tests

* remove module settings in logstash.yml

* Overwrite my logstash.yml with the content on master

* Add comments to yaml fixtures
2018-10-23 09:12:12 +01:00
Josh Soref
c6cd247ec3 Multiple spelling corrections (#9782)
* spelling: actually
* spelling: already
* spelling: concurrency
* spelling: constructor
* spelling: destinations
* spelling: different
* spelling: elasticsearch
* spelling: encoding
* spelling: error
* spelling: explicitly
* spelling: failings
* spelling: falsey
* spelling: guarantees
* spelling: having
* spelling: implementation
* spelling: logstash
* spelling: module
* spelling: multiple
* spelling: omitted
* spelling: overridden
* spelling: pipeline
* spelling: raspberry
* spelling: receive
* spelling: recommended
* spelling: registered
* spelling: registering
* spelling: shutdown
* spelling: signal
* spelling: specified
* spelling: successful
* spelling: successfully
* spelling: valuefied
* spelling: vector
2018-07-04 10:41:10 +01:00
Jake Landis
e22457252f Introduction of the Azure module (experimental)
This module leverages the Azure Event Hub to read events from the activity log and sql diagnostics.  Please note this module is considered experimental till otherwise noted.

This change also introduces logstash-input-azure_event_hubs as a default plugin

Fixes #9729
2018-07-02 16:34:49 +00:00
Tamara Braun
7d6acf1799 Port HooksRegistry to Java
Fixes #9665
2018-05-28 06:45:51 +00:00
Armin
8ebde04ae8 Load tests via JUnit
Fixes #9575
2018-05-15 16:15:39 +00:00
Jenkins CI
93cad10da1 Migrate x-pack-logstash source to logstash 2018-04-24 10:46:15 -05:00