github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-22 08:49:27 -04:00
Code Issues Projects Releases Packages Wiki Activity
kibana/x-pack/build_chromium
History
Exact
Kibana Machine 21582a7ff5
[8.x] [Reporting] update puppeteer to version 23.3.1 (#192345) (#193357) 
# Backport

This will backport the following commits from `main` to `8.x`:
- [[Reporting] update puppeteer to version 23.3.1
(#192345)](https://github.com/elastic/kibana/pull/192345)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Eyo O.
Eyo","email":"7893459+eokoneyo@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-09-18T20:26:44Z","message":"[Reporting]
update puppeteer to version 23.3.1 (#192345)\n\n## Summary\r\n\r\nUpdate
for puppeteer, the following changeset updates puppeteer to\r\nversion
`23.3.1`.\r\n\r\nThe chromium version required for this version of
puppeteer is\r\n`128.0.6613.137` from revision `1331488`, as such the
chromium binary\r\nincluded for windows and darwin platforms either
match or were the\r\nclosest revision to the expectation. The linux
headless binary was built\r\nfrom commit
`fe621c5aa2d6b987e964fb1b5066833da5fb613d` of the
same\r\nrevision.\r\n\r\n_**N.B.**_ Puppeteer 23.0.0 is earmarked as
containing breaking
changes\r\nsee\r\n[here](abda5dcc99/packages/puppeteer-core/CHANGELOG.md (2300-2024-08-07)),\r\nthis
PR considers the outlined changes and makes relevant adjustments
so\r\nreporting continues working as is.\r\n\r\n<!--\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is
usable by keyboard only (learn more\r\nabout [keyboard
accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI
touched in this PR does not create any new axe failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n-->\r\n\r\n\r\n###
How to verify linux headless build\r\n- clone the following
repo\r\nhttps://github.com/tsullivan/kibana-dev-docker\r\n- pull this
particular PR\r\n- follow the steps outlined in the repo, replacing any
occurrence of\r\n`kibana-<version>-SNAPSHOT-linux-aarch64.tar.gz` from
the repo above's\r\nstep with the output of running build on this
changeset.\r\n- before running step 4, modify the `kibana.yml` file from
the\r\n`kibana-dev-docker` repo and include the following so we might be
able\r\nto verify the version of chromium running;\r\n ```yaml\r\n
logging.loggers:\r\n - name: plugins.reporting\r\n level: debug\r\n
```\r\n- complete the steps outlined in the README, you'll have a linux
distro\r\nof kibana running on port `5601`\r\n- Attempt creating exports
of PDF and PNG reports, in dashboard, canvas,\r\nand visualizations, on
report creation attempt we would see a log output\r\nthat prints out the
chromium version exactly matching this;\r\n\r\n<img width=\"1326\"
alt=\"Screenshot 2024-09-18 at 14 50
19\"\r\nsrc=\"https://github.com/user-attachments/assets/7206781a-e8f9-469c-ad65-fd13749766b2\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"91ca8ab95c01f88ea2d79779f3670c88bf269da0","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:SharedUX","backport:prev-major","v8.16.0","backport:version","v8.15.2"],"title":"[Reporting]
update puppeteer to version
23.3.1","number":192345,"url":"https://github.com/elastic/kibana/pull/192345","mergeCommit":{"message":"[Reporting]
update puppeteer to version 23.3.1 (#192345)\n\n## Summary\r\n\r\nUpdate
for puppeteer, the following changeset updates puppeteer to\r\nversion
`23.3.1`.\r\n\r\nThe chromium version required for this version of
puppeteer is\r\n`128.0.6613.137` from revision `1331488`, as such the
chromium binary\r\nincluded for windows and darwin platforms either
match or were the\r\nclosest revision to the expectation. The linux
headless binary was built\r\nfrom commit
`fe621c5aa2d6b987e964fb1b5066833da5fb613d` of the
same\r\nrevision.\r\n\r\n_**N.B.**_ Puppeteer 23.0.0 is earmarked as
containing breaking
changes\r\nsee\r\n[here](abda5dcc99/packages/puppeteer-core/CHANGELOG.md (2300-2024-08-07)),\r\nthis
PR considers the outlined changes and makes relevant adjustments
so\r\nreporting continues working as is.\r\n\r\n<!--\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is
usable by keyboard only (learn more\r\nabout [keyboard
accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI
touched in this PR does not create any new axe failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n-->\r\n\r\n\r\n###
How to verify linux headless build\r\n- clone the following
repo\r\nhttps://github.com/tsullivan/kibana-dev-docker\r\n- pull this
particular PR\r\n- follow the steps outlined in the repo, replacing any
occurrence of\r\n`kibana-<version>-SNAPSHOT-linux-aarch64.tar.gz` from
the repo above's\r\nstep with the output of running build on this
changeset.\r\n- before running step 4, modify the `kibana.yml` file from
the\r\n`kibana-dev-docker` repo and include the following so we might be
able\r\nto verify the version of chromium running;\r\n ```yaml\r\n
logging.loggers:\r\n - name: plugins.reporting\r\n level: debug\r\n
```\r\n- complete the steps outlined in the README, you'll have a linux
distro\r\nof kibana running on port `5601`\r\n- Attempt creating exports
of PDF and PNG reports, in dashboard, canvas,\r\nand visualizations, on
report creation attempt we would see a log output\r\nthat prints out the
chromium version exactly matching this;\r\n\r\n<img width=\"1326\"
alt=\"Screenshot 2024-09-18 at 14 50
19\"\r\nsrc=\"https://github.com/user-attachments/assets/7206781a-e8f9-469c-ad65-fd13749766b2\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"91ca8ab95c01f88ea2d79779f3670c88bf269da0"}},"sourceBranch":"main","suggestedTargetBranches":["8.x","8.15"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192345","number":192345,"mergeCommit":{"message":"[Reporting]
update puppeteer to version 23.3.1 (#192345)\n\n## Summary\r\n\r\nUpdate
for puppeteer, the following changeset updates puppeteer to\r\nversion
`23.3.1`.\r\n\r\nThe chromium version required for this version of
puppeteer is\r\n`128.0.6613.137` from revision `1331488`, as such the
chromium binary\r\nincluded for windows and darwin platforms either
match or were the\r\nclosest revision to the expectation. The linux
headless binary was built\r\nfrom commit
`fe621c5aa2d6b987e964fb1b5066833da5fb613d` of the
same\r\nrevision.\r\n\r\n_**N.B.**_ Puppeteer 23.0.0 is earmarked as
containing breaking
changes\r\nsee\r\n[here](abda5dcc99/packages/puppeteer-core/CHANGELOG.md (2300-2024-08-07)),\r\nthis
PR considers the outlined changes and makes relevant adjustments
so\r\nreporting continues working as is.\r\n\r\n<!--\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [ ] Any UI touched in this PR is
usable by keyboard only (learn more\r\nabout [keyboard
accessibility](https://webaim.org/techniques/keyboard/))\r\n- [ ] Any UI
touched in this PR does not create any new axe failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n-->\r\n\r\n\r\n###
How to verify linux headless build\r\n- clone the following
repo\r\nhttps://github.com/tsullivan/kibana-dev-docker\r\n- pull this
particular PR\r\n- follow the steps outlined in the repo, replacing any
occurrence of\r\n`kibana-<version>-SNAPSHOT-linux-aarch64.tar.gz` from
the repo above's\r\nstep with the output of running build on this
changeset.\r\n- before running step 4, modify the `kibana.yml` file from
the\r\n`kibana-dev-docker` repo and include the following so we might be
able\r\nto verify the version of chromium running;\r\n ```yaml\r\n
logging.loggers:\r\n - name: plugins.reporting\r\n level: debug\r\n
```\r\n- complete the steps outlined in the README, you'll have a linux
distro\r\nof kibana running on port `5601`\r\n- Attempt creating exports
of PDF and PNG reports, in dashboard, canvas,\r\nand visualizations, on
report creation attempt we would see a log output\r\nthat prints out the
chromium version exactly matching this;\r\n\r\n<img width=\"1326\"
alt=\"Screenshot 2024-09-18 at 14 50
19\"\r\nsrc=\"https://github.com/user-attachments/assets/7206781a-e8f9-469c-ad65-fd13749766b2\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"91ca8ab95c01f88ea2d79779f3670c88bf269da0"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.15","label":"v8.15.2","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Eyo O. Eyo <7893459+eokoneyo@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-09-19 08:10:51 -05:00
..
linux [8.x] [Reporting] update puppeteer to version 23.3.1 (#192345) (#193357) 2024-09-19 08:10:51 -05:00
.gitignore
build.py Puppeteer update v21.5.2 (#172332) 2023-12-05 16:45:51 +01:00
build_util.py Puppeteer update v21.5.2 (#172332) 2023-12-05 16:45:51 +01:00
init.py Puppeteer v18.1 (#143485) 2022-10-19 13:52:33 -07:00
README.md Reporting Puppeteer update v21.3.5 (#167554) 2023-10-03 15:55:52 -07:00

README.md

Chromium build

We ship our own headless build of Chromium which is significantly smaller than the standard binaries shipped by Google. The scripts in this folder can be used to accept a commit hash from the Chromium repository, and initialize the build on Ubuntu Linux.

Why do we do this

By default, Puppeteer will download a zip file containing the Chromium browser for any OS. This creates problems on Linux, because Chromium has a dependency on X11, which is often not installed for a server environment. We don't want to make a requirement for Linux that you need X11 to run Kibana. To work around this, we create our own Chromium build, using the headless_shell build target. There are no (trustworthy) sources of these builds available elsewhere.

Fortunately, creating the custom builds is only necessary for Linux. When you have a build of Kibana for Linux, or if you use a Linux desktop to develop Kibana, you have a copy of headless_shell bundled inside. When you have a Windows or Mac build of Kibana, or use either of those for development, you have a copy of the full build of Chromium, which was downloaded from the main Chromium download location.

Build Script Usage

The system OS requires a few setup steps:

  1. Required packages: bzip2, git, lsb_release, python3
  2. The python command needs to launch Python 3.
  3. Recommended: tmux, as your ssh session may get interrupted

These commands show how to set up an environment to build:

# Allow our scripts to use depot_tools commands
export PATH=$HOME/chromium/depot_tools:$PATH

# Create a dedicated working directory for this directory of Python scripts.
mkdir ~/chromium && cd ~/chromium

# Copy the scripts from the Kibana team's GCS bucket
gsutil cp -r gs://headless_shell_staging/build_chromium .

# Install the OS packages, configure the environment, download the chromium source (25GB)
python ./build_chromium/init.py

# Run the build script with the path to the chromium src directory, the git commit hash
python ./build_chromium/build.py 70f5d88ea95298a18a85c33c98ea00e02358ad75 x64

# Make sure you are using python3, you can state the path explicitly if needed
/usr/bin/python3 ./build_chromium/build.py 67649b10b92bb182fba357831ef7dd6a1baa5648 x64

# OR You can build for ARM
python ./build_chromium/build.py 70f5d88ea95298a18a85c33c98ea00e02358ad75 arm64

NOTE: The init.py script updates git config to make it more possible for the Chromium repo to be cloned successfully. If checking out the Chromium fails with "early EOF" errors, the instance could be low on memory or disk space.

Getting the Commit Hash

If you need to bump the version of Puppeteer, you need to get a new git commit hash for Chromium that corresponds to the Puppeteer version.

node scripts/chromium_version.js [PuppeteerVersion]

When bumping the Puppeteer version, make sure you also update the ChromiumArchivePaths.revision variable in x-pack/plugins/reporting/server/browsers/chromium/paths.ts.

In some cases the revision number might not be available for the darwin or windows builds in https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html. For example, 1181205 was not available for darwin arm64 or windows. In that case, the next available revision numbers 1181286 and 1181280 were used.

Build args

A good how-to on building Chromium from source is here.

We have an linux/args.gn file that is automatically copied to the build target directory.

To get a list of the build arguments that are enabled, install depot_tools and run gn args out/headless --list from the chromium/src directory. It prints out all of the flags and their settings, including the defaults. Some build flags are documented here.

NOTE: Please, make sure you consult @elastic/kibana-security before you change, remove or add any of the build flags.

Directions for Elasticians

If you wish to use a remote VM to build, you'll need access to our GCP account.

NOTE: The builds should be done in Ubuntu on x64 architecture. ARM builds are created in x64 using cross-compiling. CentOS is not supported for building Chromium.

  1. Login to Google Cloud Console
  2. Click the "Compute Engine" tab.
  3. Create a Linux VM:
    • 8 CPU
    • 30GB memory
    • 80GB free space on disk (Try ncdu /home to see where space is used.)
    • "Cloud API access scopes": must have read / write scope for the Storage API. Access scopes in the GCP VM instance needs to be set to allow full access to all Cloud APIs vs default access (this will return a 403 otherwise in the build.py script)
  4. Install Google Cloud SDK locally to ssh into the GCP instance

Artifacts

After the build completes, there will be a .zip file and a .md5 file in ~/chromium/chromium/src/out/headless. These are named like so: chromium-{first_7_of_SHA}-{platform}-{arch}, for example: chromium-4747cc2-linux-x64. The zip files and md5 files are copied to a staging bucket in GCP storage.

To publish the built artifacts for bunding in Kibana, copy the files from the headless_shell_staging bucket to the headless_shell bucket.

gsutil cp gs://headless_shell_staging/chromium-67649b1-linux_arm64.md5 gs://headless_shell/
gsutil cp gs://headless_shell_staging/chromium-67649b1-linux_arm64.zip gs://headless_shell/

IMPORTANT: Do not replace builds in the headless_shell bucket that are referenced in an active Kibana branch. CI tests on that branch will fail since the archive checksum no longer matches the original version.

Testing

Search the Puppeteer Github repo for known issues that could affect our use case, and make sure to test anywhere that is affected.

Here's the steps on how to test a Puppeteer upgrade, run these tests on Mac, Windows, Linux x64 and Linux arm64:

  • Make sure the Reporting plugin is fetching the correct version of the browser at start-up time, and that it can successfully unzip it and copy the files to x-pack/plugins/reporting/chromium
  • Make sure there are no errors when using the Reporting diagnostic tool
  • All functional and API tests that generate PDF and PNG files should pass.
  • Use a VM to run Kibana in a low-memory environment and try to generate a PNG of a dashboard that outputs as a 4MB file. Document the minimum requirements in the PR.

Testing Chromium upgrades on a Windows Machine

Directions on creating a build of Kibana off an existing PR can be found here: https://www.elastic.co/guide/en/kibana/current/building-kibana.html You will need this build to install on your windows device to test the in progress PR.

The default extractor for Windows might give Path too long errors.

For an elasticsearch cluster to base the latest kibana build with, you can use a snapshot.sh bash script to generate the latest build. Create a file called snapshot.sh and put the following into the file:

runQuery() {
  curl --silent -XGET https://artifacts-api.elastic.co${1}
}
BUILD_HASH=$(runQuery /v1/versions/${VERSION}-SNAPSHOT/builds | jq -r '.builds[0]')
echo "Latest build hash :: $BUILD_HASH"
KBN_DOWNLOAD=$(runQuery /v1/versions/${VERSION}-SNAPSHOT/builds/$BUILD_HASH/projects/elasticsearch/packages/elasticsearch-${VERSION}-SNAPSHOT-windows-x86_64.zip)
echo $KBN_DOWNLOAD | jq -r '.package.url'

In the terminal once you have the snapshot.sh file written run: chmod a+x snapshot.sh to make the file executable Then set the version variable within the script to what you need by typing the following (in this example 8.8.0): VERSION=8.8.0 ./snapshot.sh

In the terminal you should see a web address that will give you a download of elasticsearch.

You may need to disable xpack security in the elasticsearch.yml xpack.security.enabled: false

Make sure nothing is set in the kibana.yml

Run .\bin\elasticsearch.bat in the elasticsearch directory first and then once it's up run .\bin\kibana.bat

Navigate to localhost:5601 and there shouldn't be any prompts to set up security etc. To test PNG reporting, you may need to upload a license. Navigate to https://wiki.elastic.co/display/PM/Internal+License+-+X-Pack+and+Endgame and download the license.json from Internal Licenses.

Navigate to Stack Management in Kibana and you can upload the license.json from internal licenses. You won't need to restart the cluster and should be able to test the Kibana feature as needed at this point.

Resources

The following links provide helpful context about how the Chromium build works, and its prerequisites: