This commit adds JDK matrix Buildkite pipelines for
Windows 2022, 2019 and 2016.
It also makes the groups easier to read (on both Linux and Windows
pipelines) by removing the os-jdk prefix from the job labels.
`testDLQWriterFlusherRemovesExpiredSegmentWhenCurrentHeadSegmentIsEmpty`
fails on Windows Buildkite agents and it's a test issue tracked in
https://github.com/elastic/logstash/issues/15562.
Relates:
- https://github.com/elastic/logstash/pull/15539
- https://github.com/elastic/ingest-dev/issues/1725
(cherry picked from commit 0ede19a0e1)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit is the first part of the migration of JDK matrix tests
from Jenkins to Buildkite. There will be two separate pipelines, for
Linux and Windows.
Linux is currently limited to Ubuntu 22.04 and 20.04, but
additional operating systems will be added outside of the Logstash
repository seamlessly through additional VM images.
Steps are created dynamically and the underlying script is meant to be
common for Linux and Windows. Windows is currently a stub and
will be added in a follow up PR.
Relates:
- https://github.com/elastic/ingest-dev/issues/1725
- https://github.com/elastic/ci-agent-images/pull/424
(cherry picked from commit 956bf483f2)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit is the follow up PR after #15466, which migrates away
the remaining aarch64 acceptance test Jenkins jobs to Buildkite.
Relates:
- #15466
- https://github.com/elastic/ingest-dev/issues/1724
(cherry picked from commit c384190718)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit is the first part of migrating away the aarch64 Jenkins
jobs to Buildkite. It adds a group of exhaustive test steps in the
aarch64 pipeline.
The java unit tests are temporarily disabled as they run SonarQube
scans which need to be associated with pull requests.
Relates:
https://github.com/elastic/ingest-dev/issues/1724
(cherry picked from commit 36656de4f0)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This is a backport of the initial Pull Request pipeline for Buildkite.
While currently we haven't migrated all PR jobs from Jenkins, this is needed so PRs against non `main` branches don't fail this step (also giving us the possibility to test functionality against non `main` branches).
Relates:
- #15402
- #15413
- #15415
- #15421
- https://github.com/elastic/ingest-dev/issues/1721
## Release notes
[rn:skip]
PR#15392 (which was a backport to `8.10` branch for the Buildkite DRA
scripts) had a bug.
This commit moves the dra scripts under the right directory as used
in the other branches.
* Snyk scans Logstash container vulnerabilities.
* Exclude integ test and tools when Snyk scanning.
* Remote repo url fix for main branch.
* Update .buildkite/scripts/snyk/report.sh
Simplify the logic to retrieve the version from `versions.yml`
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
* Add backstage definition for Snyk Report pipeline.
---------
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
* This commit includes number of changes:
- adds initial pipeline steps, each step proceeds one branch report job
- a script to resolve current and previous release latest branches from the release URL. Noting that it considers the branches from snapshots where upcoming releases will be our interest.
- pseudo logic to download logstash, build and report to Snyk. In order to script fully work we need a logstash machine to access Buildkite vault, which I am working.
* Source optimization and use Buildkite Vault to access to Snyk token.
* Add git branch and head to Snyk project tag.
* Dynamically resolve latest branches instead of manually defining.
* Update .buildkite/scripts/snyk/report.sh
Improve comment readability.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
---------
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
