This will eventually replace the pipeline definition in the catalog repo, using the local file instead.
---------
Co-authored-by: elastic-backstage-prod[bot] <123036547+elastic-backstage-prod[bot]@users.noreply.github.com>
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* This commit includes number of changes:
- adds initial pipeline steps, each step proceeds one branch report job
- a script to resolve current and previous release latest branches from the release URL. Noting that it considers the branches from snapshots where upcoming releases will be our interest.
- pseudo logic to download logstash, build and report to Snyk. In order to script fully work we need a logstash machine to access Buildkite vault, which I am working.
* Source optimization and use Buildkite Vault to access to Snyk token.
* Add git branch and head to Snyk project tag.
* Dynamically resolve latest branches instead of manually defining.
* Update .buildkite/scripts/snyk/report.sh
Improve comment readability.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
---------
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
This commit changed the java_pipeline.rb to include the pipeline/main thread on the stalling threads info list, that way, Logstash can provide users with more helpful information when the stalling thread is the pipeline/main one.
This commit adds missing Elasticsearch SSL settings and replaces deprecated options being used on `xpack.monitoring.*` and `xpack.management.*` settings:
Changes:
- Updated deprecated monitoring and management Elasticsearch's SSL settings so no warnings are logged.
- Added monitoring settings support for file-based certificates and for the cipher suites: `xpack.monitoring.elasticsearch.ssl.certificate`, `xpack.monitoring.elasticsearch.ssl.key`, and `xpack.monitoring.elasticsearch.ssl.cipher_suites`.
- Added management settings support for file-based certificates and for the cipher suites: `xpack.management.elasticsearch.ssl.certificate`, `xpack.management.elasticsearch.ssl.key`, and `xpack.management.elasticsearch.ssl.cipher_suites`.
* docs: fix example block syntax types and truncations
* docs: provide wrapping hints to flow metric tables
* docs: refresh node stats api response examples
include only `current` and `lifetime` metrics that are GA, and not
technology preview metrics.
* docs: use "m(onospace)" modifier for metric name columns
* docs: swap literal column to first
relies on `#guide table td:first-child .literal` having `white-space: nowrap`
With PR #13923 was described how the DLQ storage_policy behaves. With PR #14261 was introduced a duplication of the description in logstash.yml.
This commit resolves the duplication and keeps the latest description that seems more direct expressive.
Modifies the logic used by the scheduled task flusher so that execute age policy also in case the current (head) segments is not stale (haven't received any write, and the segment is empty).
This means that generally used finalize segment logic is applied plus a reinforcement step to grant the age policy is respected.
However this PR:
- introduced new debug log lines, improving the description of the context when a segment is finalized (because the DLQ is closing or because the segment file has reached its maximum size or because the flush interval expiration). This is done with the introduction of `SealReason` enumeration.
- introduces `Awaitility` test dependency to improve the testing of asychronous conditions.
* Revert "Revert "Unpin bundler to allow 2.4 (#14894)" (#14942)"
This reverts commit 5e3038a3d3.
* prevent bundler 2.4 from blowing up memory during ./gradlew generatePluginsVersion
for some reason, instances of Bundler::Dsl are retained in
org.jruby.MetaClass. Each Bundler::Dsl will retain Bundler::SourceList
which is > 150MB. This commit sets the internal state of each Dsl's
source list so the large objects can be collected.
The proper fix would be to either reuse a single Dsl object or fix the
code upstream to avoid having Dsl objects retained by org.jruby.MetaClass.
* skip input cloudwatch during generatePluginsVersion
the new bundler 2.4's resolution algorithm seems to struggle in certain
situations, like resolving combination of "logstash-input-cloudwatch" and
"logstash-integration-aws".
Until the issue is solved let's skip it.
Modify the WorkerLoop to catch the newly introduced exception org.logstash.execution.AbortedBatchException so that an inflight batch could be negatively ACK-ed. This feature is used in combination with PQs to let exit plugins without completing the processing. Any filter and output already executed for the batch will be executed again next time the batch is picked by the persistent queue.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
Fixes the system unit TimeoutStopSec option to set 0 on older version that do not support 'infinity'.
Also handles the upgrade case where the OS may not automatically upgrade the systemd file.
In commit, DLQ writer skips searching segments with 1 byte during the search of the oldest segment timestamp in the initialization
Co-authored-by: Andrea Selva <andrea.selva@elastic.co>
This reverts commit 6d08a7c1cd.
Bundler 2.4 is accumulating memory during the `generateDocsVersion` task
outside of the known Bundler 2.3 DepProxy class.
Until the memory mitigation patch is adapted to Bundler 2.4 it's best to revert 6d08a7c1cd.
This commit adds environment variable LOG_STYLE to control log behavior of docker container
`console` - this is the default to output to standout
`file` - log to disk.
Fixed: #14941
