* Update failing policy in Central Management fetcher and license checker if hit ES down node (#13689)
Wraps the calls to the central management Elasticsearch cluster with the utility class Stud::Try to handle the remote host error when the client used to connect hit a not available node.
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
(cherry picked from commit c544ecb380)
* Covered all calls to ES with retryable
* Mocked logger interaction in test after wrapping the client calls with retryable
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
* add product origin header to license checks
* add origin header to Central Management config fetcher
* add origin header to ES output for Monitoring pipeline
(cherry picked from commit 2892964ba1)
With #13308 configuration namespace that started with `http.` was renamed to `api.`, this commit fix a usage left behind.
Use the new `api.enabled` setting in one place instead of the deprecated `http.enable`.
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
(cherry picked from commit 88c80ebb19)
* Soften deprecation language and point module deprecations to agent integrations
* Remove extra `and`
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
This PR integrates Elasticsearch bootstrap script to help users keep Logstah geoip plugin run without online update check.
Add `xpack.geoip.download.endpoint` option to config geoip database service endpoint.
Users can point to `http://localhost:8080/overview.json` when using the script to bootstrap nginx docker
This PR changes the behavior of copying license files from .tgz
Originally, only two files, MaxMind LICENSE.txt and COPYRIGHT.txt, are required
Now more files, README.txt and Elastic ToC, are potentially required
Instead of targeting the files, this change copies all content in .tgz
This commit contains two fixes
* Fix Date class clash when used in pipelines with Date filter and GeoIP
* Pinned jruby-openssl version 0.10.5 to avoid SSL errors
(cherry picked from commit 6f55066b17)
Upstream `ElasticsearchOptions#es_options_from_settings` already uses the
setting `elasticsearch.ssl.verification_mode` to produce an appropriate
boolean-valued `ssl_certificate_verification` in our `es_settings` hash, so
we can rely on it instead of re-checking equality with a string.
(cherry picked from commit d5becc0082)
This commit avoid an error in gathering monitoring information when webserver is disabled or is not yet started;
which could happen with slow loading pipelines or no pipelines defined from the central management UI.
(cherry picked from commit 91996cf2a2)
add wildcard support in xpack pipeline id
do the pattern matching with glob
add warning msg to wildcard with legacy api
check invalid pipeline in bootstrap
test cases for invalid checking
Fixed: #10558
* replace direct hidden indices access with system indices api
* fulfill backward compatibility
* fix log msg, rename class, simplify response handling
* modularise fetcher
fixes a regression introduced with the api_key support for xpack monitoring and management in #11864 which disabled the possibility to not use any authentication by relying on the default options and only enabling monitoring for example. It now ignores the default username option when no password is explicitly set.
Refactor: move PipelineConfig from Ruby to Java
Reimplement the Ruby class PipelinceConfig in Java trying to keep the method signatures to limit the changes in client code, this is a step of other that intend to move all the configuration code in Java language.
Having all that code in Java unlock some reasoning about how to better implement it and probably an improvement in performance during process startup.
Moved also the spec into a JUnit and fixed here and there the failing tests
Closes: #11824
We have "required" units for a variety of `TimeValue` settings when they are
provided as a `String`, but unquoted values in YAML have been passed through as
Integers, where we long assumed nanosecond units. This frequently leads to
surprise (e.g., when `config.reload.interval` is set to `60`, we consume 100%
of CPU in a tight loop trying to reload and re-parse the configs every 60
nanoseconds).
By making the setting retain the TimeValue object for the entirety of its
lifecycle, we can issue a deprecation notice the first time an Integer value is
encountered. As a secondary benefit, our usage of the setting value in code
becomes more clear since we are empowered to ask `TimeValue` for a numeric
value in a specific scale.
Fixes#11803
* parent 8c5697c748
author Guy Boertje <guy@elastic.co> 1556806171 +0100
committer Mike Place <mike.place@elastic.co> 1557234770 +0200
Bump JrJackson to 0.4.8
Fixes#10748
LIR serializer refactor
Remove commented code
Remove more commented code
Remove license and add encoding
Style change to make code more vertical.
eid and hash
Use pipelines_info to construct the stats
Add tests for new fields
Add queue stats
* bad merge resolution
* bad merge resolution
* Don't merge if nil
* Better merge strategy
* add vertex gate
* Guard against nil
* Use extended queue stats in pipeline report
* Add cluster uuids to Elasticsearch outputters in pipeline output
* move uuid
* remove old uuid lookup
* Only populate cluster_uuids when present
* remove print
* cluster_uuids -> cluster_uuid
* Update logstash-core/lib/logstash/api/commands/stats.rb
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Update logstash-core/lib/logstash/api/commands/stats.rb
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Update logstash-core/lib/logstash/api/commands/stats.rb
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Make var singular
* Match singular var name
* Remove unnecessary nil check
* Pass in the matching pipeline for the report
* Remove old way of inserting cluster_uuids
* Update logstash-core/lib/logstash/api/commands/stats.rb
I like this much better and in testing it seems to work correctly.
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Remove unreferenced code that was part of debugging
* Remove events var which was unused
* Don't try to remove before insert
* Update logstash-core/lib/logstash/api/commands/stats.rb
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Make pipeline extended stats generation more efficient
* Implement suggestion to improve readability
* Cleaner merging per review recommendation
* Only generate extended_stats once
* remove unneeded comments
* Add cluster_uuid to node vertex
* remove top-level cluster_uuids
* Update logstash-core/lib/logstash/api/commands/stats.rb
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Implement change to make logic more simple suggested in review
* Rely on options gate to insert graph
Resolves concern here:
https://github.com/elastic/logstash/pull/10576#issuecomment-501774635
* Update logstash-core/lib/logstash/api/commands/stats.rb
Co-Authored-By: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Move UUID lookup to API layer
* Move private method to bottom per review recommandation
