Commit graph

36 commits

Author SHA1 Message Date
João Duarte
1d1fa7a471
update commons-io commons-compress jrjackson databind (#15125)
* update commons-io to 2.13.0 and commons-compress to 1.23.0

* update databind and jackson
2023-06-29 09:02:32 +02:00
Andrea Selva
a126364102
Centralized version definition for snakeyaml and shadow plugin used in buildscripts (#14952)
Create a variable in the Gradle's 'ext' map to store the versions of libraries and plugins used across the various .gradle files.
2023-03-10 16:50:16 +01:00
João Duarte
15b2d88647
bump snakeyaml to 1.33 (#14881)
ensure logstash-core picks up 1.33 as well from jackson-data format-yaml's
transititive dependency.
2023-02-09 18:20:03 +00:00
João Duarte
4919286e42
rename references of master branch to main branch (#13301) 2021-11-08 10:23:46 +00:00
Andrea Selva
728289e862
Switched to Gradle 7 (#13184)
This commit:
- Updates the Gradle wrapper to version 7.2
- Remove the deprecated jcenter and where it was used to retrieve Gradle's plugins it switches to gradlePluginPortal
- Insert an explicit dependency from test task to the log4j.properties manipulation task ("copyProductionLog4jConfiguration") used in integration
2021-09-08 10:42:13 +02:00
Andrea Selva
a7f6c01a3a
Update Snakeyaml version to 1.29 (#13129)
Snakeyaml is used only in the build chain, and it's not packaged with Logstash
2021-08-16 10:33:47 +02:00
Andrea Selva
a0774c4e76
Explicitate the type of log format in appender's names (#12964)
Remove an useful dynamic creation of appender's log file which leverages the `log.format` property 
also when it's explicit by the appender itself.

Log4j configuration leverages the placeholder `${sys:ls.log.format}` to compose the name of the log file.
This generates some not evident conflicts in log4j internals, these conflicts became evident when enabling the `pipeline.separate_logs` feature is enabled and the log4j appender definitions contains both json and plain format.
The problem is that under those circumstances the rollover of the log file doesn't happen.

This commit also add a test against the production log4j configuration, to avoid future regressions.
2021-06-15 11:25:08 +02:00
andsel
99679870c6 Implements scripted log4j filters and appenders to Java, avoid usage of deprecated Javascript Nashorn (#12512)
- replaces all scripted filters with custom Java implementation
- implemented routing appender per pipeline in Java
- adapted log4j configuration shipped with Logstash
- exit the Logstash process if it detects an scripted log4j configuration
2021-01-25 02:25:09 -08:00
Joao Duarte
a7ae0d55e6 update log4j script routes definition
fixes the change introduced with https://issues.apache.org/jira/browse/LOG4J2-2647
2020-06-09 06:26:31 -07:00
Rob Bavey
ac95667c11
Use task avoidance API in gradle scripts (#11914)
* Use task avoidance API in gradle scripts

This commit uses the task avoidance api (tasks.register vs task.create/
task DSL), as recommended since Gradle 5.1

This should reduce the execution of unnecessary tasks in build jobs, and
hopefully improve build resiliency and execution time.
2020-05-29 10:25:52 -04:00
Joao Duarte
6cc4de77e3 update guava and httpclient dependencies
Fixes #11791
2020-04-15 13:49:54 +00:00
Rob Bavey
58314a7f0e
Update gradle version to 6.3 (#11742)
* Update gradle version to 6.3

Gradle versions prior to 6.3 cannot run under JDK14.
This commit upgrades the version of Gradle to 6.3, and removes all deprecation warnings that can currently be removed.

Changes include:
* Increase gradle memory to 2g
* Increase gradle memory in the license check job to 2g
* Replace use of `testCompile`
* Replace `runtime` with `runtimeOnly`
* Remove`compile` depedencies from gradle files
* Replace deprecated archive methods
* Fix dependencies report build
* Make jruby dependencies 'api', fix archiveVersion
* Set `duplicatesStrategy` for all tasks of type Copy
* Use `configureEach` for global 'withType' calls
** Use the recommended Tasks API calls
(https://blog.gradle.org/preview-avoiding-task-configuration-time)
* Run `./gradlew wrapper` earlier to improve caching
* Use copy with chown for resources that need to be run during `./gradlew wrapper`
2020-04-07 12:49:36 -04:00
João Duarte
e9c9865f40
Add apache and elastic license headers to source code files (#11673)
* add license header to ruby and java files
* add license header to erb and rake files
* add license headers to gradle files
2020-03-11 11:53:38 +00:00
andsel
71eed80d3c Fixes to JMH launch after upgrade to Gradle 5
Fixes #11459
2019-12-30 10:43:44 +00:00
João Duarte
e807a9e030
Update gradle wrapper to 5.6.4 (#11389)
* upgrade gradle wrapper to the latest 5.x release
* update shadow plugin to 4.x for gradle compatibility
* change dependency declaration to work around [breaking change in annotation processing](https://docs.gradle.org/current/userguide/upgrading_version_4.html#rel4.6:annotation_processor_configuration)
2019-12-06 15:39:17 +00:00
andsel
e58a6e01ce Added LS configuration variable 'pipeline.separate_logs' to separate logs per pipelines - use log4j RoutingAppender - avoid output to main log files when log per pipeline is enabled - closes 10427
Fixes #11108
2019-10-08 14:07:17 +00:00
Dan Hermann
48594baf2b Add license reporting task
Fixes #9331
2018-05-02 15:35:42 +00:00
Armin
26fe96c038 MINOR: Remove unused CheckpointIOFactory
Fixes #9054
2018-02-05 07:13:10 +00:00
Armin
0a1a761442 CLEANUP: Remove redundant PageIOFactory
Fixes #9012
2018-01-23 20:22:25 +00:00
Armin
4091ddfdb4 Remove mock file io
Fixes #8999
2018-01-22 23:22:15 +00:00
Colin Surprenant
e75eca4483 maximize PQ read bach size by using same read timeout semantic as MQ #8702
add from another LongVector

use timedout read batch and added test for maximization

WIP maximized batch read size

fixes and cleanups

cleanups

use the await returned boolean to assess timeout

remove useless constant

use Callable<Void> and cosmetic fix
2017-12-05 13:40:35 -05:00
Armin
823feaf69a MINOR: Remove JMH main entrypoints
Fixes #8591
2017-11-06 13:46:24 +00:00
Armin
44e68b4627 MINOR: Clean up some dead code around PQ + PQ Benchmarks
Fixes #8232
2017-09-13 13:19:19 +00:00
Armin
f06e859717 #7998 Serialization benchmark and Afterburner removal
Fixes #8002
2017-08-16 16:08:09 +00:00
Armin
96ff365997 Fixes #7676 and generally improves the performance of org.logstash.StringInterpolation#evaluate by:
* Don't use regex to find the contents between `%{}`, it's way too inefficient for a simple case like this with all its allocations
* Flatten logic out of the many helper classes
* Use a reusable, thread-local `StringBuilder` to serialize into
* Don't use `.substring` for comparisons
* `static` `ObjectMapper`
* `org.joda.time.format.DateTimeFormat#forPattern` not being cached is not an issue, since it's cached in `org.joda.time.format.DateTimeFormat#cPatternCache`

* Added benchmark
   * ~15% faster for a few cases I ran, even for repeated patterns with interpolation + couldn't find any case even with date formatting, where this implementation is slower than the cached one
   *  no leaking on dynamic patterns

Fixes #7692
2017-07-25 18:29:10 +00:00
Armin
5b0c86c4a4 MINOR: Align JMH JVM settings with production default JVM settings
Fixes #7534
2017-06-25 17:20:20 +00:00
Pier-Hugues Pellerin
6a4766a01f Centralize the JRuby version
We have multiple places to declare the jruby version, this commit
centralize everything into the `version.yml` file. This is now used by
the ruby part and the gradle build file to correctly fetch the jruby
library.

Fixes #7434
2017-06-13 15:44:31 +00:00
Pier-Hugues Pellerin
6e6589e773 Use uniform version withing our gradle dependencies
Make sure we use the same provided and runtime JRuby with our gradle
projects

Fixes: #7421

Fixes #7425
2017-06-13 12:44:01 +00:00
Armin
941446a6dc PERFORMANCE: Queue RW benchmark for in memory queue
Fixes #7410
2017-06-10 15:05:13 +00:00
Armin
4926f5b6bb PERFORMANCE: Queue RW benchmark baseline
Fixes #7400
2017-06-09 17:22:39 +00:00
Armin
b6e051acbc #7317 Queue RW Benchmark
Fixes #7355
2017-06-08 13:38:41 +00:00
Armin
54fa8d06a0 MINOR: Add log config to benchmarks
Fixes #7359
2017-06-08 04:51:33 +00:00
Armin
e80bfe45e7 MINOR: Dry up gradle settings
Fixes #7298
2017-06-03 07:07:28 +00:00
Armin
c80eabbc9d #7111 cleaned up one case of redundant CRC32 digest and byte[] creation. Adjusted benchmark checkpoint interval to default value to be able to judge performance impact of the move.
Fixes #7166
2017-05-26 04:47:19 +00:00
Armin
1a2c0c8597 #7053 Refactor Queue Settings Class to Using a Builder Pattern
Fixes #7062
2017-05-18 15:57:43 +00:00
Armin
b6c855a0a6 Added JMH Benchmark Suite + Queue Benchmark
Fixes #7118
2017-05-18 06:31:17 +00:00