Commit graph

206 commits

Author SHA1 Message Date
Rene Groeschke
a2d5aa4bc5
[Gradle] Fix deprecation warning in release tests after 8.12 update (#119608) (#119626) 2025-01-07 20:05:58 +11:00
Mark Vieira
5386acee8f
Fix vcs revision label in docker images (#119531) (#119540) 2025-01-04 06:08:21 +11:00
Rene Groeschke
4d17b2193a
Update Gradle wrapper to 8.12 (#118683) (#119357)
This updates the gradle wrapper to 8.12

We addressed deprecation warnings due to the update that includes:

- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions

(cherry picked from commit ba61f8c7f7)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerCloudElasticsearchDistributionType.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerUbiElasticsearchDistributionType.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/Fixture.java
#	plugins/repository-hdfs/hadoop-client-api/build.gradle
#	server/src/main/java/org/elasticsearch/inference/ChunkingOptions.java
#	x-pack/plugin/kql/build.gradle
#	x-pack/plugin/migrate/build.gradle
#	x-pack/plugin/security/qa/security-basic/build.gradle
2024-12-31 08:37:28 +01:00
Mark Vieira
88f96b503d
chore: deps(ironbank): Bump ubi version to 9.5 (#119203) (#119229)
(cherry picked from commit 2c736f49b0)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-24 05:00:37 +11:00
Mark Vieira
d5cf398621
chore: deps(ironbank): Bump ubi version to 9.5 (#119039) (#119143)
<Actions>     <action
id="ad27da7f660d61c82c61599e0e6945827ced1590f4bf36a5f74db07e99c04215">
<h3>deps: Bump ironbank version</h3>         <details
id="c8ee27cd13736547d240c88751dee86bddfbda339d0ee25795d0cc066ff6ea01">
<summary>deps(ironbank): Bump ubi version to 9.5</summary>
<p>change detected:&#xA;&#x9;* key &#34;$.args.BASE_TAG&#34; updated
from &#34;\&#34;9.4\&#34;&#34; to &#34;\&#34;9.5\&#34;&#34;, in file
&#34;distribution/docker/src/docker/iron_bank/hardening_manifest.yaml&#34;</p>
</details>         <details
id="ddfe323476a100a4fecf1aab633e1209ffebe2f0e55366500df03a529b067050">
<summary>deps(ironbank): Bump ubi version to 9.5</summary>
<p>changed lines [25] of file
&#34;/tmp/updatecli/github/elastic/elasticsearch/distribution/docker/src/docker/Dockerfile&#34;</p>
</details>         <a
href="1240723046">GitHub
Action workflow link</a>     </action> </Actions>

---

<table>   <tr>     <td width="77">       <img
src="https://www.updatecli.io/images/updatecli.png" alt="Updatecli logo"
width="50" height="50">     </td>     <td>       <p>         Created
automatically by <a href="https://www.updatecli.io/">Updatecli</a>
</p>       <details><summary>Options:</summary>         <br />
<p>Most of Updatecli configuration is done via <a
href="https://www.updatecli.io/docs/prologue/quick-start/">its
manifest(s)</a>.</p>         <ul>           <li>If you close this pull
request, Updatecli will automatically reopen it, the next time it
runs.</li>           <li>If you close this pull request and delete the
base branch, Updatecli will automatically recreate it, erasing all
previous commits made.</li>         </ul>         <p>           Feel
free to report any issues at <a
href="https://github.com/updatecli/updatecli/issues">github.com/updatecli/updatecli</a>.<br
/>           If you find this tool useful, do not hesitate to star <a
href="https://github.com/updatecli/updatecli/stargazers">our GitHub
repository</a> as a sign of appreciation, and/or to tell us directly on
our <a
href="https://matrix.to/#/#Updatecli_community:gitter.im">chat</a>!
</p>       </details>     </td>   </tr> </table>

(cherry picked from commit 46356bd64d)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-20 07:47:23 +11:00
Rene Groeschke
f28df38704
[8.17] Revert cloud wolfi migration (#119068) (#119103)
This reverts the wolfi migration for cloud ess docker images.

(cherry picked from commit 81f725d3d8)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java
#	distribution/docker/build.gradle
#	qa/packaging/src/test/java/org/elasticsearch/packaging/test/DockerTests.java
2024-12-20 03:22:05 +11:00
Rene Groeschke
581b9ab7c0
[8.16] [Gradle] Remove static use of BuildParams (#115122) (#117434)
* [Gradle] Remove static use of BuildParams (#115122)

Static fields dont do well in Gradle with configuration cache enabled.

- Use buildParams extension in build scripts
- Keep BuildParams.ci for now for easy serverless migration
-  Tweak testing doc

(cherry picked from commit 13c8aaeffa)

# Conflicts:
#	TESTING.asciidoc
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/InternalDistributionBwcSetupPlugin.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/RestTestBasePlugin.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/AbstractYamlRestCompatTestPlugin.java
#	build.gradle
#	modules/ingest-geoip/qa/full-cluster-restart/build.gradle
#	qa/mixed-cluster/build.gradle
#	x-pack/plugin/ent-search/qa/full-cluster-restart/build.gradle
#	x-pack/plugin/eql/qa/rest/build.gradle
#	x-pack/plugin/fleet/qa/rest/build.gradle
#	x-pack/plugin/kql/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/ml/qa/multi-cluster-tests-with-security/build.gradle
#	x-pack/plugin/security/qa/multi-cluster/build.gradle
#	x-pack/plugin/sql/qa/jdbc/build.gradle
#	x-pack/plugin/transform/qa/multi-cluster-tests-with-security/build.gradle

* Fix merge

* [Build] Fix fips testing after buildparams rework (#116934)

* More Cleanup

* [Build] Fix checkstyle exclusions on windows (#115185)

* More merge fixes

* Delete x-pack/plugin/kql/build.gradle
2024-11-27 12:34:32 +01:00
Rene Groeschke
6192818528
Replace cloud-ess docker image with wolfi-ess (#114413) (#114684)
* Replace cloud-ess docker image with wolfi-ess
   We just replaced the existing implementation of cloud-ess with what was wolfi-ess which is a wolfi based ess image.
   The cloud image itself will be removed in a future commit it was not used anywhere

* Switch to test cloud docker image instead of default docker in packaging pr tests.
  This adds way more coverage than the default docker image which is also barely touched

(cherry picked from commit f1f5ee06a3)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java
2024-10-14 16:07:11 +02:00
Rene Groeschke
45f57f11a2
[8.x] Update wolfi image and fix breaking change (#114390) (#114489)
* Update wolfi image and fix breaking change (#114390)

(cherry picked from commit 190513dce7)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java

* Fix spotlight
2024-10-11 21:08:33 +11:00
Rene Groeschke
54ecb2c666
Add wolfi ess docker image (#113810) (#114111)
(cherry picked from commit 54c83d7fa7)
2024-10-04 13:44:42 +02:00
Brian Seeders
c0be2e2c1b
Fix packaging tests after addition of new wolfi-based image (#112831) (#113715) 2024-09-27 16:01:14 -04:00
Mark Vieira
0279c0a909
Add AGPLv3 as a supported license 2024-09-13 14:30:33 -07:00
Brian Seeders
5c9710caf5
Fix wolfi docker tasks (#112721) 2024-09-10 17:39:53 -04:00
Rene Groeschke
4dee614707
Add chainguard docker image (#112103) 2024-09-06 10:32:42 -07:00
Victor Martinez
e7c0ba953a
Update IronBank docker image base to ubi:9.4 (#111743) 2024-08-28 19:18:58 +02:00
Rene Groeschke
7226100458
Make :distribution:docker:check Gradle configuration cache compatible (#109083)
fix cache compatibility issue in TestFixturesPlugin and support
yamlRestTest in `:distribution:docker:yamlRestTest`
2024-06-05 19:23:04 +10:00
Rene Groeschke
2704d3a8d1
Remove cross project support in TestFixturesPlugin (#109077)
- One step closer to configuration cache support
- Crossproject support has been replaced by using testcontainer based fixtures
2024-05-27 17:59:14 +02:00
Rene Groeschke
077d8be599
Fix typo in BuildParams.isCi() usage (#104812) 2024-01-26 09:24:12 -05:00
Rene Groeschke
ab8ee60bba
Fix SamlAuthenticationIT flakyness (#103867) 2024-01-24 13:44:31 -05:00
Jon
c0bfab9338
Update IronBank docker image base to ubi:9.3 (#102721) 2023-12-06 10:18:47 -08:00
Jon
6787765c37
Update IronBank BASE_IMAGE with ironbank prefix (#102720)
This supports local testing.  It should not be included in
hardening_manifest.yml, which injects the scope at runtime.
2023-11-30 13:47:33 -08:00
Mark Vieira
328ebc4145
Update IronBank docker image base to ubi:9.2 (#101393) 2023-10-26 11:29:44 -07:00
Rene Groeschke
69d87e1bfd
Rework beats artifacts resolution in docker build (#100157)
* Rework metricbeat artifacts resolution to avoid misleading error message
* Pass File object to beats repo uri to workaround parsing problems
2023-10-06 17:04:46 +02:00
Rene Groeschke
cc2a711b2d
Always use es tarball for creating IronBank docker context (#95958)
This fixes an issue with building iron bank images that are handled differently
than other docker images.
2023-05-09 17:39:35 +02:00
Rene Groeschke
778adc8622
Make es docker image creation more efficient (#95883)
We want avoid packing and unpacking elasticsearch archives to safe build time
2023-05-08 18:20:10 +02:00
Michael Montgomery
a0056941e5
Use UID for dockerfile to allow runAsNonRoot to be used. (#95390)
Signed-off-by: Michael Montgomery <mmontg1@gmail.com>
2023-04-19 12:26:26 -07:00
Rene Groeschke
e735455431
Provide hints to Gradle Task.onlyIf declarations (#94953)
* Update gradle enterprise plugin to show skip reasons in build scans
* Unify OS specific handling in build logic
2023-03-31 23:57:55 +02:00
Mark Vieira
315fa321ae
Improve efficiency of packer cache script (#94701) 2023-03-23 13:29:45 -07:00
Mark Vieira
30faac838a
Ensure correct architecture for beats is used in cloud docker images (#92997) 2023-01-17 10:56:45 -08:00
Mark Vieira
c2eda511de
Add JUnit rule based integration test cluster orchestration framework (#92379)
This commit adds a new test framework for configuring and orchestrating
test clusters for both Java and YAML REST testing. This will eventually
replace the existing "test-clusters" Gradle plugin and the build-time
cluster orchestration.
2022-12-21 15:33:46 -08:00
Dimitrios Liappis
a6351aae3b
Add docs about multi-arch Docker builds (#92185)
Enhance docker distribution project docs with prerequisites for building
multi-architecture images.
2022-12-07 13:08:53 +02:00
Mark Vieira
e68e28e4cb Ensure beats are bundled in Cloud docker image during DRA builds
(cherry picked from commit 210d4576fa)
2022-11-16 16:15:23 -08:00
Rene Groeschke
f6b1087f30
Provide compressed docker image exports for DRA process (#91593)
Initial fix for providing the expected Docker files for our DRA process
- Can be tweaked to be more efficient, but kept simple for now
2022-11-15 11:39:56 -05:00
Mark Vieira
143828e4f4
Add exportDockerImages lifecycle task for exporting docker tarballs (#91571)
We need exported Docker artifacts for the release process. Add a
lifecycle task to generate these artifacts.
2022-11-14 20:00:16 -05:00
Rene Groeschke
f880d41669
Add DRA snapshot build (#91018)
* Add support for building against defined build-ids of dependent artifacts
* Introduce DraResolvePlugin and functional tests
* Declare initial dra snapshot ci job for elasticsearch

We run the DRA builds against ubuntu as other release related ci jobs have in the past. Also we only have libs2xmlutills available on our ubuntu based ci workers
2022-11-09 16:16:21 +01:00
Mark Vieira
3c3a70cbeb Add extensions points for building customized Docker images 2022-09-30 15:43:31 -07:00
Mark Vieira
3603aa7151
Use architecture specific tag for building cloud-ess image (#90074) 2022-09-15 16:19:30 -07:00
Mark Vieira
fa38d1ba8f
Allow for building multi-arch docker images via buildx (#89986) 2022-09-13 08:32:55 -07:00
Mark Vieira
5ebaece2ba
Use DRA artifacts for beats and ML dependencies (#89951) 2022-09-09 13:05:19 -07:00
Rory Hunter
ec8af834f4
Write docs about our Docker build setup (#89036)
Add a README.md to the docker distribution project, with details
about how we build and test our images.
2022-08-03 11:02:27 +01:00
Mark Vieira
41af3cb8e5
Ignore beats artifacts when resolving all artifact dependencies (#88960) 2022-07-29 13:32:24 -07:00
Rory Hunter
d5b1356b6c
Fix docker positional params (take 2) (#88584)
As part of #50277, we removed the `TAKE_FILE_OWNERSHIP` option from the
Docker entrypoint script and the associated chroot calls, and instead
just defaulted to running the image as `elasticsearch` instead of
`root`.

However, we didn't check that it was still possible to pass CLI options
to Elasticsearch via CLI arguments, and broke this by mistake. This is
probably an uncommon pattern, versus environment variables or a config
file.  Nevertheless, it is supposed to be possible and is mentioned in
the documentation.

Fix the problem by suppling the missing positional params when calling
Elasticsearch, and add a test case so that we don't break it again.
2022-07-19 09:17:36 +01:00
Rory Hunter
ffc6a14755 Revert "Fix passing positional args to ES in Docker (#88502)"
This reverts commit 9f4b32a20a.
2022-07-18 09:28:49 +01:00
Rory Hunter
9f4b32a20a
Fix passing positional args to ES in Docker (#88502)
As part of #50277, we removed the `TAKE_FILE_OWNERSHIP` option from the
Docker entrypoint script and the associated chroot calls, and instead
just defaulted to running the image as `elasticsearch` instead of
`root`.

However, we didn't check that it was still possible to pass CLI options
to Elasticsearch via CLI arguments, and broke this by mistake. This is
probably an uncommon pattern, versus environment variables or a config
file.  Nevertheless, it is supposed to be possible and is mentioned in
the documentation.

Fix the problem by suppling the missing positional params when calling
Elasticsearch, and add a test case so that we don't break it again.
2022-07-14 09:12:04 +01:00
Rene Groeschke
dbf39741a0
Make LoggedExec gradle task configuration cache compatible (#87621)
This changes the LoggedExec task to be configuration cache compatible. We changed the implementation
to use `ExecOperations` instead of extending `Exec` task. As double checked with the Gradle team this task
is not planned to be made configuration cache compatible out of the box anytime soon.

This is part of the effort on https://github.com/elastic/elasticsearch/issues/57918
2022-07-11 08:46:54 +02:00
Rory Hunter
cb70dac016
Remove vim-tiny from Docker build (#87812)
Closes #85634.
2022-06-21 10:22:18 +01:00
Rene Groeschke
bf4304d18a
Remove elasticsearch.rest-test gradle plugin (#85491)
Removes and remaining usages of `elasticsearch.rest-test` and the plugin itself from the codebase

Relates to https://github.com/elastic/elasticsearch/issues/63696
2022-05-17 11:23:04 +02:00
Rory Hunter
31be78b13d
Update Iron Bank base image to 8.6 (#86796)
Iron Bank have advised us to upgrade the UBI base image.
2022-05-16 21:35:17 +01:00
Przemyslaw Gomulka
4958642404
Disable deprecation log indexing in docker test (#85150)
Deprecation log indexing can make test very fragile
and randomly failing during cluster cleanup.
This commit disables indexing of deprecation logs for :distribution:docker tests

closes #79803
2022-03-21 12:05:49 +01:00
Rory Hunter
728ef33844
Remove use of Cloudflare zlib (#84680)
Relates to #81662. This library isn't ready for public consumption.
Remove it from the Docker build.
2022-03-07 10:03:39 +00:00