This commit adds JDK matrix Buildkite pipelines for
Windows 2022, 2019 and 2016.
It also makes the groups easier to read (on both Linux and Windows
pipelines) by removing the os-jdk prefix from the job labels.
`testDLQWriterFlusherRemovesExpiredSegmentWhenCurrentHeadSegmentIsEmpty`
fails on Windows Buildkite agents and it's a test issue tracked in
https://github.com/elastic/logstash/issues/15562.
Relates:
- https://github.com/elastic/logstash/pull/15539
- https://github.com/elastic/ingest-dev/issues/1725
(cherry picked from commit 0ede19a0e1)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit is the first part of the migration of JDK matrix tests
from Jenkins to Buildkite. There will be two separate pipelines, for
Linux and Windows.
Linux is currently limited to Ubuntu 22.04 and 20.04, but
additional operating systems will be added outside of the Logstash
repository seamlessly through additional VM images.
Steps are created dynamically and the underlying script is meant to be
common for Linux and Windows. Windows is currently a stub and
will be added in a follow up PR.
Relates:
- https://github.com/elastic/ingest-dev/issues/1725
- https://github.com/elastic/ci-agent-images/pull/424
(cherry picked from commit 956bf483f2)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit splits the generic Buildkite pipelines introduced
in #15520 for JDK tests to separate pipelines for Linux and Windows.
(cherry picked from commit 07147b3e40)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit is the follow up PR after #15466, which migrates away
the remaining aarch64 acceptance test Jenkins jobs to Buildkite.
Relates:
- #15466
- https://github.com/elastic/ingest-dev/issues/1724
(cherry picked from commit c384190718)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
PR#15466 skipped the Java unit tests as on the `main` and `8.11`
branches they attempted to run sonar scans (which are only meant to
run for PRs).
This commit re-enables the Java unit tests, taking advantage of #15486,
disabling the sonar scan part of the test suite.
(cherry picked from commit 16da966290)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This commit is the first part of migrating away the aarch64 Jenkins
jobs to Buildkite. It adds a group of exhaustive test steps in the
aarch64 pipeline.
The java unit tests are temporarily disabled as they run SonarQube
scans which need to be associated with pull requests.
Relates:
https://github.com/elastic/ingest-dev/issues/1724
(cherry picked from commit 36656de4f0)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
This is a backport of the initial Pull Request pipeline for Buildkite.
While currently we haven't migrated all PR jobs from Jenkins, this is needed so PRs against non `main` branches don't fail this step (also giving us the possibility to test functionality against non `main` branches).
Relates:
- #15402
- #15413
- #15415
- #15421
- https://github.com/elastic/ingest-dev/issues/1721
## Release notes
[rn:skip]
PR#15392 (which was a backport to `8.10` branch for the Buildkite DRA
scripts) had a bug.
This commit moves the dra scripts under the right directory as used
in the other branches.
This is a backport of the DRA pipeline and related scripts from:
- #15366
- #15365
- #15356
- #15352
- #15344
- #15343
- #15337
- #15312
Note that it's a manual backport because some PRs (e.g. #15312) contain files (`catalog-info.yaml`) that should only
live on the `main` branch.
This commit changes the serverless tests to use a restricted API key with necessary permissions and disables Logstash legacy monitoring test.
Fixes: #15234
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* Snyk scans Logstash container vulnerabilities.
* Exclude integ test and tools when Snyk scanning.
* Remote repo url fix for main branch.
* Update .buildkite/scripts/snyk/report.sh
Simplify the logic to retrieve the version from `versions.yml`
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
* Add backstage definition for Snyk Report pipeline.
---------
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
This will eventually replace the pipeline definition in the catalog repo, using the local file instead.
---------
Co-authored-by: elastic-backstage-prod[bot] <123036547+elastic-backstage-prod[bot]@users.noreply.github.com>
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* This commit includes number of changes:
- adds initial pipeline steps, each step proceeds one branch report job
- a script to resolve current and previous release latest branches from the release URL. Noting that it considers the branches from snapshots where upcoming releases will be our interest.
- pseudo logic to download logstash, build and report to Snyk. In order to script fully work we need a logstash machine to access Buildkite vault, which I am working.
* Source optimization and use Buildkite Vault to access to Snyk token.
* Add git branch and head to Snyk project tag.
* Dynamically resolve latest branches instead of manually defining.
* Update .buildkite/scripts/snyk/report.sh
Improve comment readability.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
---------
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
