logstash/ci/serverless
kaisecheng 9af7c7cef4
Serverless test against restricted API key (#15224)
This commit changes the serverless tests to use a restricted API key with necessary permissions and disables Logstash legacy monitoring test.

Fixes: #15234

Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
2023-08-02 15:27:46 +01:00
..
config Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00
metricbeat buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
pipeline Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00
test_data buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
common.sh Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00
cpm_tests.sh Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00
dlq_rspec_tests.sh buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
elastic_integration_filter_tests.sh buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
es_filter_tests.sh buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
es_input_tests.sh buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
es_output_tests.sh buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
kibana_api_tests.sh Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00
metricbeat_monitoring_tests.sh buildkite serverless test (#15150) 2023-07-19 17:21:53 +01:00
monitoring_tests.sh Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00
README.md Serverless test against restricted API key (#15224) 2023-08-02 15:27:46 +01:00

The test cases against serverless Elasticsearch covers the following scenarios

  • es-output
  • es-input
  • es-filter
  • elastic-integration-filter (Logstash run ingest pipeline)
  • DLQ
  • central pipeline management
  • Kibana API for pipeline management (CPM)
  • Metricbeat monitoring
  • Logstash legacy monitoring

Credentials

The username, password, API key and hosts are stored in Vault secret/ci/elastic-logstash/serverless-test.

Vault field
es_user username of superuser
es_user_pw password of superuser
ls_role_api_key_encoded base64 api_key for integration-filter
ls_plugin_api_key id:api_key for Logstash plugins
es_host Elasticsearch endpoint
kb_host Kibana endpoint

Generate API key for Logstash with limited privileges instead of using superuser elastic.

POST /_security/api_key
{
  "name": "logstash_serverless_apikey",
  "expiration": "365d",   
  "role_descriptors": { 
    "logstash_serverless_role": {
      "cluster": ["monitor", "manage_index_templates", "manage_logstash_pipelines", "cluster:admin/ingest/pipeline/get", "read_pipeline"], 
      "indices": [
        {
          "names": [ "logstash", "logstash-*", "ecs-logstash", "ecs-logstash-*", "serverless*", "logs-*", "metrics-*", "synthetics-*", "traces-*" ], 
          "privileges": ["manage", "write", "create_index", "read", "view_index_metadata"]  
        }
      ]
    }
  }
}