mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 14:47:19 -04:00
122 lines
5.7 KiB
Text
122 lines
5.7 KiB
Text
[float]
|
|
[[power-of-logstash]]
|
|
== The Power of Logstash
|
|
|
|
*The ingestion workhorse for Elasticsearch and more*
|
|
|
|
Horizontally scalable data processing pipeline with strong Elasticsearch and Kibana synergy
|
|
|
|
*Pluggable pipeline architecture*
|
|
|
|
Mix, match, and orchestrate different inputs, filters, and outputs to play in pipeline harmony
|
|
|
|
*Community-extensible and developer-friendly plugin ecosystem*
|
|
|
|
Over 200 plugins available, plus the flexibility of creating and contributing your own
|
|
|
|
image:static/images/logstash.png[]
|
|
|
|
[float]
|
|
== Logstash Loves Data
|
|
|
|
Collect more, so you can know more. Logstash welcomes data of all shapes and sizes.
|
|
|
|
[float]
|
|
=== Logs and Metrics
|
|
|
|
Where it all started.
|
|
|
|
* Handle all types of logging data
|
|
** Easily ingest a multitude of web logs like {logstash-ref}/advanced-pipeline.html[Apache], and application
|
|
logs like {logstash-ref}/plugins-inputs-log4j.html[log4j] for Java
|
|
** Capture many other log formats like {logstash-ref}/plugins-inputs-syslog.html[syslog], networking and firewall logs, and more
|
|
* Enjoy complementary secure log forwarding capabilities with https://www.elastic.co/products/beats/filebeat[Filebeat]
|
|
* Collect metrics from {logstash-ref}/plugins-inputs-ganglia.html[Ganglia], {logstash-ref}/plugins-codecs-collectd.html[collectd],
|
|
{logstash-ref}/plugins-codecs-netflow.html[NetFlow], {logstash-ref}/plugins-inputs-jmx.html[JMX], and many other infrastructure
|
|
and application platforms over {logstash-ref}/plugins-inputs-tcp.html[TCP] and {logstash-ref}/plugins-inputs-udp.html[UDP]
|
|
|
|
[float]
|
|
=== The Web
|
|
|
|
Unlock the World Wide Web.
|
|
|
|
* Transform {logstash-ref}/plugins-inputs-http.html[HTTP requests] into events
|
|
** Consume from web service firehoses like {logstash-ref}/plugins-inputs-twitter.html[Twitter] for social sentiment analysis
|
|
** Webhook support for GitHub, HipChat, JIRA, and countless other applications
|
|
** Enables many https://www.elastic.co/products/x-pack/alerting[Watcher] alerting use cases
|
|
* Create events by polling {logstash-ref}/plugins-inputs-http_poller.html[HTTP endpoints] on demand
|
|
** Universally capture health, performance, metrics, and other types of data from web application interfaces
|
|
** Perfect for scenarios where the control of polling is preferred over receiving
|
|
|
|
[float]
|
|
=== Data Stores and Streams
|
|
|
|
Discover more value from the data you already own.
|
|
|
|
* Better understand your data from any relational database or NoSQL store with a
|
|
{logstash-ref}/plugins-inputs-jdbc.html[JDBC] interface
|
|
* Unify diverse data streams from messaging queues like Apache {logstash-ref}/plugins-outputs-kafka.html[Kafka],
|
|
{logstash-ref}/plugins-outputs-rabbitmq.html[RabbitMQ], and {logstash-ref}/plugins-outputs-sqs.html[Amazon SQS]
|
|
|
|
[float]
|
|
=== Sensors and IoT
|
|
|
|
Explore an expansive breadth of other data.
|
|
|
|
* In this age of technological advancement, the massive IoT world unleashes endless use cases through capturing and
|
|
harnessing data from connected sensors.
|
|
* Logstash is the common event collection backbone for ingestion of data shipped from mobile devices to intelligent
|
|
homes, connected vehicles, healthcare sensors, and many other industry specific applications.
|
|
|
|
[float]
|
|
== Easily Enrich Everything
|
|
|
|
The better the data, the better the knowledge. Clean and transform your data during ingestion to gain near real-time
|
|
insights immediately at index or output time. Logstash comes out-of-box with many aggregations and mutations along
|
|
with pattern matching, geo mapping, and dynamic lookup capabilities.
|
|
|
|
* {logstash-ref}/plugins-filters-grok.html[Grok] is the bread and butter of Logstash filters and is used ubiquitously to derive
|
|
structure out of unstructured data. Enjoy a wealth of integrated patterns aimed to help quickly resolve web, systems,
|
|
networking, and other types of event formats.
|
|
* Expand your horizons by deciphering {logstash-ref}/plugins-filters-geoip.html[geo coordinates] from IP addresses, normalizing
|
|
{logstash-ref}/plugins-filters-date.html[date] complexity, simplifying {logstash-ref}/plugins-filters-kv.html[key-value pairs] and
|
|
{logstash-ref}/plugins-filters-csv.html[CSV] data, {logstash-ref}/plugins-filters-fingerprint.html[fingerprinting](anonymizing) sensitive information,
|
|
and further enriching your data with {logstash-ref}/plugins-filters-translate.html[local lookups] or Elasticsearch
|
|
{logstash-ref}/plugins-filters-elasticsearch.html[queries].
|
|
* Codecs are often used to ease the processing of common event structures like {logstash-ref}/plugins-codecs-json.html[JSON]
|
|
and {logstash-ref}/plugins-codecs-multiline.html[multiline] events.
|
|
|
|
See {logstash-ref}/transformation.html[Transforming Data] for an overview of some of the popular data processing plugins.
|
|
|
|
[float]
|
|
== Choose Your Stash
|
|
|
|
Route your data where it matters most. Unlock various downstream analytical and operational use cases by storing,
|
|
analyzing, and taking action on your data.
|
|
|
|
*Analysis*
|
|
|
|
* {logstash-ref}/plugins-outputs-elasticsearch.html[Elasticsearch]
|
|
* Data stores such as {logstash-ref}/plugins-outputs-mongodb.html[MongoDB] and {logstash-ref}/plugins-outputs-riak.html[Riak]
|
|
|
|
*Archiving*
|
|
|
|
* {logstash-ref}/plugins-outputs-webhdfs.html[HDFS]
|
|
* {logstash-ref}/plugins-outputs-s3.html[S3]
|
|
|
|
*Monitoring*
|
|
|
|
* {logstash-ref}/plugins-outputs-nagios.html[Nagios]
|
|
* {logstash-ref}/plugins-outputs-ganglia.html[Ganglia]
|
|
* {logstash-ref}/plugins-outputs-zabbix.html[Zabbix]
|
|
* {logstash-ref}/plugins-outputs-graphite.html[Graphite]
|
|
* {logstash-ref}/plugins-outputs-datadog.html[Datadog]
|
|
* {logstash-ref}/plugins-outputs-cloudwatch.html[CloudWatch]
|
|
|
|
*Alerting*
|
|
|
|
* https://www.elastic.co/products/watcher[Watcher] with Elasticsearch
|
|
* {logstash-ref}/plugins-outputs-email.html[Email]
|
|
* {logstash-ref}/plugins-outputs-pagerduty.html[Pagerduty]
|
|
* {logstash-ref}/plugins-outputs-irc.html[IRC]
|
|
* {logstash-ref}/plugins-outputs-sns.html[SNS]
|