This commit cherry-picked the missing becnhmark script PRs
The deprecated artifacts-api is removed
[CI] benchmark uses the new artifacts-api (#17224)
[CI] benchmark readme (#16783)
Introduce a new flag to explicitly permit legacy monitoring (#16586) (Only take the benchmark script)
[ci] fix wrong queue type in benchmark marathon (#16465)
[CI] fix benchmark marathon (#16447)
[CI] benchmark dashboard and pipeline for testing against multiple versions (#16421)
- fixes the CI benchmark script to always runs against the latest snapshot version
- uses `/v1/versions/$VERSION/builds/latest` to get the latest build id
Fixes: #16307
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Add a buildkite pipeline to do benchmark.
The script does benchmark by running Filebeats (docker) -> Logstash (docker) -> ES Cloud.
Logstash metrics and benchmark results are sent to the same ES Cloud.
- Secrets store in vault `secret/ci/elastic-logstash/benchmark`
- Use flog (docker) to generate ~2GB logs
- Pull the snapshot docker image of the main branch every day
- Logstash runs two pipelines, main and node_stats
- The main pipeline handles beats ingestion, sending data to the data stream `logs-generic-default`
- It runs for all combinations. (pq + mq) x worker x batch size
- Each test runs for ~7 minutes
- The node_stats pipeline retrieves /_node/stats API every 30s and sends it to the data stream `metrics-nodestats-logstash`
- The script sends a summary of EPS and resource usage to index `benchmark_summary`
The buildkite pipeline accepts ENV variables to customize the test
| Variable Name | Default Value | Comment |
|-----------------|---------------------|----------------------------------------------------|
| FB_VERSION | 8.13.4 | docker tag |
| LS_VERSION | | docker tag |
| LS_JAVA_OPTS | -Xmx2g | by default, Xmx is set to half of memory |
| MULTIPLIERS | 2,4,6 | determine the number of workers (cpu * multiplier) |
| BATCH_SIZES | 125,1000 | |
| CPU | 4 | number of cpu for Logstash container |
| MEM | 4 | number of GB for Logstash container |
| QTYPE | memory | queue type to test -- persisted; memory; all |
| FB_CNT | 4 | number of filebeats to use in benchmark |
To check the result
- `vault read secret/ci/elastic-logstash/benchmark` to get the host and credentials
- `curl -u "$ES_USER:$ES_PW" "$ES_HOST/benchmark_summary/_search"`
Fixes: https://github.com/elastic/ingest-dev/issues/3377
