Currently for pipelines that have steps with auto retries, if there
is a failure of a step, even if the subsequent retry succeeds, we
received Slack alerts.
This commits uses new functionality available in the Buildkite PR Bot
to avoid Slack notifications in those cases and changes the settings
for all pipelines where we have, or expect to have, retries.
This commit adds a schedule to run the exhaustive pipeline
(biweekly, every other Wednesday @2AM UTC) and the aarch64
(weekly, every Monday@2AM UTC).
Closes https://github.com/elastic/ingest-dev/issues/2852
Having enabled branch pushes as triggers for the exhaustive test
pipeline, we have an unwanted sideeffect, that when using
PR automation (@logstashmachine) to create backports, the temporary
branches also triggered exhaustive tests.
This commit skips triggering this pipeline when upstream branches
starting with `backport` get pushed.
This commit enables running the exhaustive tests Buildkite pipeline
(i.e. the equivalent to the `main` Jenkins tests) ; the trigger is
code events, i.e. direct pushes, merge commits and creation of new branches.
CI is skipped if changes are only related to files under `docs/`.
As a follow up to #15705, this commit shushes Slack alerts for the
common Buildkite scheduling pipeline. This is because the pipeline
scheduler just triggers other pipelines which have their own dedicated
alerts setup, therefore we want to avoid duplicate alerts when
there is a failure with one of the triggered pipelines.
As an improvement from #15668 / #15700, rather than having one
dedicated side-car scheduling job per pipeline, we move to a single
scheduling job. Various pipelines that need triggering with different
schedules now live under each schedule in the new pipeline.
This reduces the amount of jobs we have to maintain in yaml.
This commit enhances the functionality introduced in #15668 and #15700
by allowing a single Buildkite scheduling job to trigger several
pipelines, in addition to multiple branches which it already does.
We rename the env var PIPELINE_TO_TRIGGER to PIPELINES_TO_TRIGGER
which now supports comma separate values.
This enhancement can be useful for pipelines like JDK matrix which
have variants (Linux and Windows) that we want to trigger with a single
scheduling job, thus reducing unnecessary entries in catalog-info.
This commit is the first making use of #15627 to remove hard coded
branches for the DRA Snapshot build schedule.
With this pattern, we will only need to keep `ci/branches.json` up to date,
as versions evolve, and not need to update/maintain hard coded branches
in `catalog-info.yaml` anymore.
Once this is verified working, we'll add a corresponding schedule
pipeline (in `catalog.info`) for the JDK matrix job.
Relates: https://github.com/elastic/ingest-dev/issues/2664
* Use Java installed BK agent and remove unnecessary git clone operation since repo is already cloned.
* Switch back to normal VM since Logstash BK agent doesn't support docker operations.
This commit adds a maximum default (global) timeout for every pipeline
definition (now that it's possible to define this programmatically in
an RRE).
The default values have been chosen arbitrarily based on intuition
about how much (in the worst case) we should wait for each job to
run until we consider them stuck/failed.
While at it, we update the yaml schema for RREs to point to the
latest commit (rather than a pinned commit that doesn't reflect the
latest changes, e.g. `maximum_default_timeout`).
Relates #15380
This commit enables scheduled runs of the JDK matrix pipelines for
both Windows and Linux, once a week, Tuesdays at 1am UTC, using the
pipeline defaults for OS and JDK.
This commit adds a global max timeout of 90min for the supported plugins
Buildkite pipeline. This prevents hanging builds (for 24hrs, which is
the default).
Relates: #15380
Build's maximum_timeout_in_minutes and default_timeout_in_minutes are available now through the catalog-info.yaml file.
As this change was made manually before we implemented this in RRE/Terrazzo, they got reverted to the default value (0); thus, I am raising this PR to get it as it was specified before the upgrade (120 mins.)
This commit enables automatically updated comments from Buildkite for
PRs. It posts comments to PRs whenever a Buildkite build finishes or
encounters its first failing step and keeps the comment updated.
This commit enables the snapshot job schedule for Buildkite jobs.
They are set to run after Jenkins (scheduled @midnight) so that any
remote/unknown chance of conflict with the release manager is limited.
While at it, we also enable slack notifications for failures to the
same channels as Jenkins.
This commit adds a Buildkite resource for pull requests and
the two simple jobs, license checking and ruby unit tests that are
already part of the Jenkins PR multi job setup.
As this is WiP, slack notifications aren't enabled.
Add initial buildkite pipeline for DRA artifacts.
This is a "hello-world" pipeline skeleton required to iterate on migrating the actual DRA jobs.
This commit also updates the backstage component for logstash to become a top level entity with the
necessary annotations and linked dependencies to pipelines.
Relates https://github.com/elastic/ingest-dev/issues/1720
* Snyk scans Logstash container vulnerabilities.
* Exclude integ test and tools when Snyk scanning.
* Remote repo url fix for main branch.
* Update .buildkite/scripts/snyk/report.sh
Simplify the logic to retrieve the version from `versions.yml`
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
* Add backstage definition for Snyk Report pipeline.
---------
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
This will eventually replace the pipeline definition in the catalog repo, using the local file instead.
---------
Co-authored-by: elastic-backstage-prod[bot] <123036547+elastic-backstage-prod[bot]@users.noreply.github.com>
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
