Commit graph

1638 commits

Author SHA1 Message Date
Pere Urbon-Bayes
fcfebb7535 add option to filter payload (first level) in the node api
Fixes #5630
2016-07-20 06:01:34 -04:00
Pere Urbon-Bayes
9618b658f7 reorg specs to fit the new schema
Fixes #5623
2016-07-19 08:50:42 -04:00
Pere Urbon-Bayes
2249a03edb rename _node/stats/pipeline pipeline into plugins as is more clean that the resources means
Fixes #5623
2016-07-19 08:50:41 -04:00
Pere Urbon-Bayes
dafb7eafc6 /_node/stats/events object+resource is removed as these metrics now instead reside under the pipeline stats /_node/stats/pipeline
Fixes #5623
2016-07-19 08:50:41 -04:00
Pere Urbon-Bayes
97d679f933 Removed /_node/stats/mem and insted nested them under /_node/stats/jvm along with the ‘thread’ stats
Fixes #5623
2016-07-19 08:50:41 -04:00
Pere Urbon-Bayes
e35addbbb5 add /_node/stats/pipeline inside /_node/stats main resource
Fixes #5623
2016-07-19 08:50:41 -04:00
Pere Urbon-Bayes
d54c66a8dd fix human parameter processing
Fixes #5621
2016-07-15 03:12:15 -04:00
Joao Duarte
87e13ced5b fix string setting validation of possible values
Fixes #5628
2016-07-14 04:01:24 -04:00
Andrew Cholakian
8cd1a475fe Add support for hostname:port syntax to SafeURI class
This is a deviation from what the ruby URI class normally allows.
With this patch users can use "myhost:123" as an option and have it do the
right thing, as opposed to before where they'd get an error and have to
use "//myhost:123".

Fixes #5618
2016-07-12 12:27:18 -04:00
Joao Duarte
4cf8031a49 bump to 5.0.0-alpha5
Fixes #5585
2016-07-05 10:03:22 -04:00
Pier-Hugues Pellerin
64f9a27208 Review changes
Fixes #5542
2016-06-27 18:37:02 -04:00
Pier-Hugues Pellerin
0b02cf6651 No need to use rspec mocking to set the metric for a plugin we can use the accessor
Fixes #5542
2016-06-27 18:37:02 -04:00
Pier-Hugues Pellerin
6fdb1935c3 Making sure the input plugin have access to the metric
Fixes #5542
2016-06-27 18:37:02 -04:00
Pier-Hugues Pellerin
363879546d Always provide a default metric to the plugin instance
Fix an issue when running tests of the plugin in isolation, no default
metric was created and the metric was nil.

This commit change the behavior to always return a `NullMetric` when
plugin instances are created outside of the pipeline.

Added tests to cover all the base class of plugin that can record
metric.

Fixes #5542
2016-06-27 18:37:02 -04:00
Pier-Hugues Pellerin
cfe71cb58b When running logstash in debug mode, display the stacktrace
Fixes #5542
2016-06-27 18:37:01 -04:00
Pier-Hugues Pellerin
988793dbb1 Add more testing around the NullMetric implementation
The current test where only checking if the `NullMetric` Interface was
matching the original metric interface, this PR add more test around the
actual methods and fix an issue with `decrement` using too much
arguments in the context of a namespaced metric.

The test were also move into a shared example to be used in both the `NullMetric`
and the `NamedspacedMetric`.

Also the NullMetric class will no use the same validation for keys as the namespaced class and will
reject empty or nil keys.

Fixes #5545
2016-06-27 11:27:20 -04:00
Joao Duarte
c0c9a3babc add data.path setting
Fixes #5528
2016-06-24 07:32:25 -04:00
Pier-Hugues Pellerin
1dd44383c7 Make sure the namespace metric is send to the plugin correctly and MetricType#To_hash wont return all the values
Fixes: #5529

Fixes #5532
2016-06-22 09:24:33 -04:00
Andrew Cholakian
d01b5f6b9c Add stats default fields (hostname, http_address, version)
Fixes #5520
2016-06-20 14:13:40 -04:00
Joao Duarte
a859bd4a3f add single command release task and other release support tooling
Purpose:

* manage releases through a minimum number of rake tasks
* simplify building of snapshot builds
* create staged artifacts, candidates for releases, that required no changes to become releases
* this means the snapshot release process will not involve publishing gems, therefore:
* the gem artifacts should only be published to rubygems as a final artifact, at the time of GA

Changes:

* release artifacts no longer depend on gems of core components
* all core components are used locally AS-IS to minimize code changes between snapshot, RC and GA
* `versions.yml` describes the versions of all logstash parts and package
  * `rake version:set[version]` manage the yaml file and push the changes to the gemspecs/version.rb files
  * `rake version:set_plugin_api[version]` manage the yaml file and push the changes to the gemspecs/version.rb files
  * `rake artifact:all` generates SNAPSHOT artifacts: tar.gz, zip, rpm, deb
  * `RELEASE=1 rake artifact:all` creates release candidate artifacts + 4 gems: logstash-core, logstash-core-event, logstash-core-event-java and logstash-core-plugin-api

implements #5416 and #5414

Fixes #5460
2016-06-20 04:23:48 -04:00
Jordan Sissel
246a2e7ad3 Fix tests that broke as a result of recent changes
Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
d2400ac93a Add plugin name and wallclock duration to output metrics.
Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
fdf19eab2e Store duration_in_millis under the 'events' section in metrics
Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
97a2f16b48 Add a gauge 'name' that shows the plugin's name in metrics
Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
07be0dcb3b Use metric#time instead of computing duration locally.
Fix a bug in the metric that called 'milliseconds' 1_000_000 for a
multiplier when it should be 1000.

Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
c8f7aee9b1 Rename :time_ms to :duration_in_millis
This was chosen based on discussion w/ Suyog, Alvin, and others.

Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
f5335c990a Structure the "plugins" stats as an array of data points.
Each data point contains the stats for a plugin instance (one particular
grok, for example).

Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
a8282184f3 Slight tidying of specs
Any metric calls are now allowed. This should help us grow our metrics
usage without accidentally failing tests that were too rigid.

I also split some testing up to have more alignment between the 'it'
description and the execution.

Fixes #5472
2016-06-17 15:17:44 -04:00
Jordan Sissel
6cb0b538e3 Fix failing tests. Allow metric time_ms capture now.
Fixes #5472
2016-06-17 15:17:43 -04:00
Jordan Sissel
1eef8c9be7 Expose pipeline metrics to /_node/stats/pipelines
Fixes #5472
2016-06-17 15:17:43 -04:00
Jordan Sissel
5a4d4bc3ba Track wallclock time spent in each filter.
Fixes #5472
2016-06-17 15:17:43 -04:00
Pier-Hugues Pellerin
73c1e68050 Namespace should use a copy of the arrays when creating a sub space
Fix: #5510

Fixes #5513
2016-06-16 17:26:05 -04:00
Andrew Cholakian
74672c994b Fix direct require of environment.rb to properly load Socket class
This was causing devutils to fail

Fixes #5511
2016-06-16 15:37:57 -04:00
Andrew Cholakian
48b78e8c58 Fix a regression where untyped fields would return null values
This patch fixes a bug where config fields declared without a type
would just return null.

Fixes #5512
2016-06-16 15:37:23 -04:00
Pere Urbon-Bayes
95cdab3583 add cleaner error for when the webserver port is already in use
Fixes #5489
2016-06-15 16:51:52 -04:00
Andrew Cholakian
11c3b83d10 Bring back accidentally deleted regexp param required validation
Fixes #5500
2016-06-15 14:40:52 -04:00
Andrew Cholakian
0f478b4faa Remove hot_threads from default _node API endpoint results
Fixes #5381
2016-06-15 14:34:15 -04:00
Andrew Cholakian
6268e063e3 Move _plugin api to _node/plugin + add tests
Fixes #5381
2016-06-15 14:34:15 -04:00
Andrew Cholakian
4c38288d31 Refactor MetricStore#extract_metrics to handle every possible situation that makes sense
Fixes #5381
2016-06-15 14:34:14 -04:00
Andrew Cholakian
86f776fafa Add more node stats + refactor stats internals
Add node stats mem endpoint

Fixes #5381
2016-06-15 14:34:14 -04:00
Andrew Cholakian
76c4bc9971 Add new :list property to configuration parameters.
If set to try this will allow the user to specify one or more values.
This generally replaces the :array type, which had fewer type checks.

The array type is still needed for lists of complex objects, e.g. hashes.

Fixes #5453
2016-06-13 14:13:56 -04:00
Pier-Hugues Pellerin
a741534f7c Code review
Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
6acbdc3015 Make the test more reliable by used a fixed amount of generated events
Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
47239b30c7 Fix the ApiTest and the Metric inputs tests
Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
587debd3e6 typos
Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
d81ad6172f Fix a thread safety issue with the agent.
Make sure we pass the metric when we create a pipeline this make sure
the metric is correctly visible in the pipeline and remove the need to
lock the variables.

Make the expectation more robust in the metric reload scenario.

Fixes #5400 for the greater good.

Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
21a47b8db9 Fix an issue when data was not present in the collector
When you ran this test in isolation it failed to find the corresponding
metric, Changed the behavior to force a collect before running the
tests.

Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
d02d4e4429 Refactor the metric library interaction
The metric library now require an instance of the collector and will not
use the singleton

Fixes #5446
2016-06-10 09:51:36 -04:00
Pier-Hugues Pellerin
0b50e2e624 Do not declare the collector as a singleton
Fixes #5446
2016-06-10 09:51:35 -04:00
Andrew Cholakian
35cffb5b39 Add URI config validator/type
Often times plugins (like the Elasticsearch output) can naturally use URIs for their configuration.
Unfortunately using the :string type here means that the password portion of the URI can easily be leaked.

This wraps the URI class in a new LogStash::Util::SafeURI class that proxies all regular URI methods but masks
the password when `#to_s` and `#inspect` are invoked.

Fixes #5439
2016-06-09 08:33:00 -04:00