github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 09:48:58 -04:00
Code Issues Projects Releases Packages Wiki Activity

[CI] Enable failed test collection on ES-intake triggered serverless verification runs (#191937)

Browse source 
## Summary
Currently, failed runs on
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote
that were triggered by the ES Serverless pipeline's downstream, are not
collecting failures. This makes it harder to grasp what set of tests
failed.

I think we originally disabled this reporting not to muddy our stats
based on runs on various ES Serverless versions - however, with this,
it's really hard to get an overview on which tests are failing for a
run, so I'd like to get the failed suites to be displayed on the job.

With this change - if I'm not mistaken - we'll generate the annotation
on the Buildkite job, but we won't report to ci-stats or create github
issues.

### In short
What we have:
<img width="889" alt="Screenshot 2024-09-02 at 18 28 45"
src="https://github.com/user-attachments/assets/d294dc00-4dd0-4b29-9e2f-866d68246a5d">

What I'd like:
<img width="1195" alt="Screenshot 2024-09-02 at 18 29 14"
src="https://github.com/user-attachments/assets/455465ba-f176-4ab5-ba31-9578c167448f">


Tested in:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote/builds/1962
This commit is contained in:
Alex Szabo 2024-09-04 16:58:45 +02:00 • committed by GitHub
parent a2dbebca31
commit 243de82cc6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 9 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

15
.buildkite/scripts/lifecycle/post_command.sh
View file

@ -36,12 +36,15 @@ if [[ "$IS_TEST_EXECUTION_STEP" == "true" ]]; then
buildkite-agent artifact upload '.es/**/*.hprof'
buildkite-agent artifact upload 'data/es_debug_*.tar.gz'
if [[ $BUILDKITE_COMMAND_EXIT_STATUS -ne 0 ]] && \
# Skip when triggered from elasticsearch's validation pipeline
[[ $BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG != 'elasticsearch-serverless-intake' ]]
then
echo "--- Run Failed Test Reporter"
node scripts/report_failed_tests --build-url="${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}" 'target/junit/**/*.xml'
if [[ $BUILDKITE_COMMAND_EXIT_STATUS -ne 0 ]]; then
if [[ $BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG == 'elasticsearch-serverless-intake' ]]; then
echo "--- Run Failed Test Reporter (only junit)"
node scripts/report_failed_tests --build-url="${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}" 'target/junit/**/*.xml'\
--no-github-update --no-index-errors
else
echo "--- Run Failed Test Reporter"
node scripts/report_failed_tests --build-url="${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}" 'target/junit/**/*.xml'
fi
fi
if [[ -d 'target/test_failures' ]]; then