github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-23 14:17:58 -04:00
Code Issues Projects Releases Packages Wiki Activity
logstash/.buildkite/scripts/benchmark
History
Exact
Mashhur 38d780a5e4
Update marathon CI pipeline to reflect recent LS released versions. (#17579)
2025-04-22 17:34:43 -07:00
..
config Introduce a new flag to explicitly permit legacy monitoring (#16586) 2024-11-19 08:52:28 +01:00
save-objects [CI] benchmark readme (#16783) 2024-12-12 11:09:22 +00:00
setup [CI] benchmark dashboard and pipeline for testing against multiple versions (#16421) 2024-09-12 19:45:23 +01:00
core.sh [CI] handling v9 setting in benchmark marathon (#17581) 2025-04-22 16:06:11 -07:00
main.sh [CI] fix benchmark marathon (#16447) 2024-09-17 22:29:02 +01:00
marathon.sh Update marathon CI pipeline to reflect recent LS released versions. (#17579) 2025-04-22 17:34:43 -07:00
README.md [CI] benchmark readme (#16783) 2024-12-12 11:09:22 +00:00
util.sh [CI] benchmark dashboard and pipeline for testing against multiple versions (#16421) 2024-09-12 19:45:23 +01:00

README.md

Steps to set up GCP instance to run benchmark script

  • Create an instance "n2-standard-16" with Ubuntu image
  • Install docker
    • sudo snap install docker
    • sudo usermod -a -G docker $USER
  • Install jq
  • Install vault
    • sudo snap install vault
    • vault login --method github
    • vault kv get -format json secret/ci/elastic-logstash/benchmark
  • Setup Elasticsearch index mapping and alias with setup/*
  • Import Kibana dashboard with save-objects/*
  • Run the benchmark script
    • Send data to your own Elasticsearch. Customise VAULT_PATH="secret/ci/elastic-logstash/your/path"
    • Run the script main.sh
      • or run in background nohup bash -x main.sh > log.log 2>&1 &

Notes

  • Benchmarks should only be compared using the same hardware setup.
  • Please do not send the test metrics to the benchmark cluster. You can set VAULT_PATH to send data and metrics to your own server.
  • Run all.sh as calibration which gives you a baseline of performance in different versions.
  • #16586 allows legacy monitoring using the configuration xpack.monitoring.allow_legacy_collection: true, which is not recognized in version 8. To run benchmarks in version 8, use the script of the corresponding branch (e.g. 8.16) instead of main in buildkite.