mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 01:38:56 -04:00
5f72e78f82
## Summary This PR adds the [High Cardinality Indexer](https://github.com/elastic/high-cardinality-cluster) to Kibana as a new package called `kbn-data-forge`. It also replaces `kbn-infra-forge` usage in the test and is the preferred way to generate data for Observability use cases, specifically for SLO testing. ### Todo - [x] Replace `kbn-infra-forge` usage - [x] Create convenience functions for testing (`generate` and `cleanup`) - [x] Make the logger (`LoggingTool`) configurable as an injected dependency - [x] Make the Elasticsearch client (`Client`) configurable as an injected dependency - [x] Fix the ECS Generate commands - [x] Add CLI options via Commander ### CLI Help Screen ``` Usage: data_forge.js [options] A data generation tool that will create realistic data with different scenarios. Options: --config <filepath> The YAML config file --lookback <datemath> When to start the indexing (default: "now-15m") --events-per-cycle <number> The number of events per cycle (default: 1) --payload-size <number> The size of the ES bulk payload (default: 10000) --concurrency <number> The number of concurrent connections to Elasticsearch (default: 5) --index-interval <milliseconds> The interval of the data in milliseconds (default: 60000) --dataset <dataset> The name of the dataset to use. Valid options: "fake_logs", "fake_hosts", "fake_stack" (default: "fake_logs") --scenario <scenerio> The scenario to label the events with (default: "good") --elasticsearch-host <address> The address to the Elasticsearch cluster (default: "http://localhost:9200") --elasticsearch-username <username> The username to for the Elasticsearch cluster (default: "elastic") --elasticsearch-password <password> The password for the Elasticsearch cluster (default: "changeme") --elasticsearch-api-key <key> The API key to connect to the Elasticsearch cluster --kibana-url <address> The address to the Kibana server (default: "http://localhost:5601") --kibana-username <username> The username for the Kibana server (default: "elastic") --kibana-password <password> The password for the Kibana server (default: "changeme") --install-kibana-assets This will install index patterns, visualizations, and dashboards for the dataset --event-template <template> The name of the event template (default: "good") --reduce-weekend-traffic-by <ratio> This will reduce the traffic on the weekends by the specified amount. Example: 0.5 will reduce the traffic by half (default: 0) --ephemeral-project-ids <number> The number of ephemeral projects to create. This is only enabled for the "fake_stack" dataset. It will create project IDs that will last 5 to 12 hours. (default: 0) -h, --help output usage information ``` ### Testing an Example Run the following command against a clean Kibana development enviroment: ``` node x-pack/scripts/data_forge.js --events-per-cycle 200 --lookback now-1h --install-kibana-assets --ephemeral-project-ids 10 --dataset fake_stack ``` This should install a handful of DataViews (Admin Console, Message Processor, Nginx Logs, Mongodb Logs) along with a few dashboards and visualizations. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| src | ||
| index.ts | ||
| jest.config.js | ||
| kibana.jsonc | ||
| package.json | ||
| README.md | ||
| tsconfig.json | ||
@kbn/infra-forge
@kbn/data-forge is a tool to generate infra data for integration testing.
Datasets
- fake hosts