logstash/x-pack
Cas Donoghue 6f89de5b35
Improve smoke tests for observability SRE image (#17486)
* Improve smoke tests for observability SRE image

This commit adds a new rspec test to run the observability SRE container in a
docker compose network with filebeat and elasticsearch. It uses some simple test
data through a pipeline with plugins we expect to be used in production. The
rspec tests will ensure the test data is flowing from filebeat to logstash to
elasticsearch by querying elasticsearch for expected transformed data.

* REVERT ME: debug whats goig on in CI :(

* Run filebeat container as root

* Work around strict file ownership perms for filebeat

We add the filebeat config in a volume, the permissions checks fail due test
runner not being a root user. This commit disables that check in filebeat as
seems to be the consensus solution online for example: https://event-driven.io/en/tricks_on_how_to_set_up_related_docker_images/

* Dynaimcally generate PKI instead of checking it in

Instead of checking in PKI, dynamically generate it with gradle task for
starting containers and running the tests. This improvement avoids github
warning of checked in keys and avoid expiration headaches. Generation is very
fast and does not add any significant overhead to test setup.

* Remove use of "should" in rspec docstrings

see https://github.com/rubocop/rspec-style-guide?tab=readme-ov-file#should-in-example-docstrings

* Ensure permissions readable for volume

Now that certs are dynamically generated, ensure they are able to be read in container

* Use elasticsearch-fips image for smoke testing

* Add git ignore for temp certs
2025-04-08 13:11:26 -07:00
..
ci Pull in latests commits from 8.x and update based on new patterns (#17385) 2025-03-21 14:37:08 -07:00
distributions/internal/observabilitySRE Improve smoke tests for observability SRE image (#17486) 2025-04-08 13:11:26 -07:00
lib x-pack: add fips validation plugin from x-pack (#16940) 2025-04-08 09:04:05 -07:00
modules Update depreacation warning to provide the version the ArcSight module is removed. (#16648) 2024-11-06 12:30:45 +01:00
qa/integration x-pack: add fips validation plugin from x-pack (#16940) 2025-04-08 09:04:05 -07:00
spec [Backport 8.x] Reimplement LogStash::String setting in Java (#16576) (#16959) 2025-01-27 16:08:51 +01:00
src/test/java/org/logstash/xpack/test Improve smoke tests for observability SRE image (#17486) 2025-04-08 13:11:26 -07:00
build.gradle Improve smoke tests for observability SRE image (#17486) 2025-04-08 13:11:26 -07:00
CHANGELOG.md Migrate x-pack-logstash source to logstash 2018-04-24 10:46:15 -05:00
gradle.properties Defensively mark x-pack as a distributed artifact 2018-05-15 19:42:31 +00:00
README.md Migrate x-pack-logstash source to logstash 2018-04-24 10:46:15 -05:00
settings.gradle Load tests via JUnit 2018-05-15 16:15:39 +00:00

Elastic License Functionality

This directory tree contains files subject to the Elastic License. The files subject to the Elastic License are grouped in this directory to clearly separate them from files licensed under the Apache License 2.0.

Logstash X-Pack

Set of plugins that form Logstash X-Pack features.

Setup

Logstash X-Pack features are a default part of Logstash; as X-Pack features provide integrations with the rest of the Elastic Stack, they will need to be configured to point to an Elasticsearch instance in order to work.

Opting Out of X-Pack

If you are unable or unwilling to run the Elastic-Licensed X-Pack Features and Functions, OSS-only distributions are available from the downloads page; to run Logstash from source without X-Pack, ensure that your environment variable OSS is exported with a value of true:

export OSS=true

Configuration

To configure x-pack settings, you can edit config/logstash.yml and add xpack.* configs from the logstash x-pack settings.

Building documentation

This repo contains information that is used in the Logstash Reference.

To build the Logstash Reference on your local machine, use the docbldls or docbldlsx build commands defined in https://github.com/elastic/docs/blob/master/doc_build_aliases.sh