docs: interpreting ci failures (#153549)

## Summary

Replace the deprecated Jenkins specifics in the interpreting CI failures
section for something Buildkite specific.

Please bear with me, I'm not much familiar with the implementations but
found the current documentation to be obsoleted.
This commit is contained in:
Victor Martinez 2023-03-24 17:27:11 +01:00 committed by GitHub
parent bd50903109
commit e1b12fd61a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 11 additions and 21 deletions

View file

@ -1,44 +1,34 @@
[[interpreting-ci-failures]]
== Interpreting CI Failures
{kib} CI uses a Jenkins feature called "Pipelines" to automate testing of the code in pull requests and on tracked branches. Pipelines are defined within the repository via the `Jenkinsfile` at the root of the project.
{kib} CI uses a Buildkite feature called "Pipelines" to automate testing of the code in pull requests and on tracked branches. Pipelines are defined within the repository via the `Pipelines` at the `.buildkite/pipelines` folder.
More information about Jenkins Pipelines can be found link:https://jenkins.io/doc/book/pipeline/[in the Jenkins book].
More information about Buildkite Pipelines can be found link:https://buildkite.com/docs/pipelines[in the docs].
[discrete]
=== Github Checks
When a test fails it will be reported to Github via Github Checks. We currently bucket tests into several categories which run in parallel to make CI faster. Groups like `ciGroup{X}` get a single check in Github, and other tests like linting, or type checks, get their own checks.
Clicking the link next to the check in the conversation tab of a pull request will take you to the log output from that section of the tests. If that log output is truncated, or doesn't clearly identify what happened, you can usually get more complete information by visiting Jenkins directly.
Clicking the link next to the check in the conversation tab of a pull request will take you to the log output from that section of the tests. If that log output is truncated, or doesn't clearly identify what happened, you can usually get more complete information by visiting Buildkite directly.
[discrete]
=== Viewing Job Executions in Jenkins
=== Viewing Job Executions in Kibana
To view the results of a job execution in Jenkins, either click the link in the comment left by `@elasticmachine` or search for the `kibana-ci` check in the list at the bottom of the PR. This link will take you to the top-level page for the specific job execution that failed.
To view the results of a job execution in Buildkite, either click the link in the comment left by `@elasticmachine` or search for the `kibana-ci` check in the list at the bottom of the PR. This link will take you to the top-level page for the specific job execution that failed.
image::images/job_view.png[Jenkins job view showing a test failure]
image::images/job_view.png[Buildkite pipeline view showing a few test failures]
1. *Git Changes:* the list of commits that were in this build which weren't in the previous build. For Pull Requests this list is calculated by comparing against the most recent Pull Request which was tested, it is not limited to build for this specific Pull Request, so it's not very useful.
2. *Test Results:* A link to the test results screen, and shortcuts to the failed tests. Functional tests capture and store the log output from each specific test, and make it visible at these links. For other test runners only the error message is visible and log output must be tracked down in the *Pipeline Steps*.
3. *Google Cloud Storage (GCS) Upload Report:* Link to the screen which lists out the artifacts uploaded to GCS during this job execution.
4. *Pipeline Steps:*: A breakdown of the pipeline that was executed, along with individual log output for each step in the pipeline.
[discrete]
=== Viewing ciGroup/test Logs
To view the logs for a failed specific ciGroup, jest, type checkers, linters, etc., click on the *Pipeline Steps* link in from the Job page.
image::images/pipeline_steps_view.png[Jenkins pipeline steps screenshot]
Scroll down the page until you find a failed step *(1)*, and then look up a few lines for the `Branch:` step to see which specific job this is. If this is the job you're looking for click the little terminal icon next to the failed step *(1)* to view the logs for that specific step in the Pipeline.
1. *Git commit:* the git commit that caused this build.
2. *Test Results:* A link to the test results screen, and shortcuts to the logs and jobs of the failed tests. Functional tests capture and store the log output from each specific test, and make it visible at these links.
3. *Pipeline Steps:*: A breakdown of the pipeline that was executed, along with individual log output for each step in the pipeline.
[discrete]
=== Debugging Functional UI Test Failures
The logs in Pipeline Steps contain `Info` level logging. To debug Functional UI tests it's usually helpful to see the debug logging. You can go to the list of all tests including failures (1), or directly to the failures (2).
The logs in Pipeline Steps contain `Info` level logging. To debug Functional UI tests it's usually helpful to see the debug logging. You can go to the test failure details by clicking on the *logs* (1).
image::images/test_results.png[Jenkisn build screenshot]
image::images/test_results.png[Buildkite build screenshot]
Looking at the failure, we first look at the Error and stack trace. In the example below, this test failed to find an element within the timeout;
`Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="createSpace"])`

Binary file not shown.

Before

Width:  |  Height:  |  Size: 624 KiB

After

Width:  |  Height:  |  Size: 570 KiB

Before After
Before After

Binary file not shown.

Before

Width:  |  Height:  |  Size: 443 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 266 KiB

After

Width:  |  Height:  |  Size: 324 KiB

Before After
Before After