mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 14:47:19 -04:00
195 lines
6.5 KiB
Text
195 lines
6.5 KiB
Text
[[netflow-module]]
|
|
=== Logstash Netflow Module
|
|
|
|
++++
|
|
<titleabbrev>Netflow Module (deprecated)</titleabbrev>
|
|
++++
|
|
|
|
deprecated[7.4.0, Replaced by the {filebeat-ref}/filebeat-module-netflow.html[{Filebeat} Netflow Module] which is compliant with the {ecs-ref}/index.html[Elastic Common Schema (ECS)]]
|
|
|
|
The Logstash Netflow module simplifies the collection, normalization, and
|
|
visualization of network flow data. With a single command, the module parses
|
|
network flow data, indexes the events into Elasticsearch, and installs a suite
|
|
of Kibana dashboards to get you exploring your data immediately.
|
|
|
|
Logstash modules support Netflow Version 5 and 9.
|
|
|
|
|
|
==== What is Flow Data?
|
|
|
|
Netflow is a type of data record streamed from capable network devices. It
|
|
contains information about connections traversing the device, and includes
|
|
source IP addresses and ports, destination IP addresses and ports, types of
|
|
service, VLANs, and other information that can be encoded into frame and
|
|
protocol headers. With Netflow data, network operators can go beyond monitoring
|
|
simply the volume of data crossing their networks. They can understand where the
|
|
traffic originated, where it is going, and what services or applications it is
|
|
part of.
|
|
|
|
[[netflow-requirements]]
|
|
===== Requirements
|
|
|
|
These instructions assume you have already installed Elastic Stack
|
|
(Logstash, Elasticsearch, and Kibana) version 5.6 or higher. The products you
|
|
need are https://www.elastic.co/downloads[available to download] and easy to
|
|
install.
|
|
|
|
[[netflow-getting-started]]
|
|
==== Getting Started
|
|
|
|
NOTE: The {ls} Netflow Module has been deprecated and replaced by the
|
|
{filebeat-ref}/filebeat-module-netflow.html[{Filebeat} Netflow Module], which is
|
|
compliant with the {ecs-ref}/index.html[Elastic Common Schema (ECS)].
|
|
|
|
. Start the Logstash Netflow module by running the following command in the
|
|
Logstash installation directory:
|
|
+
|
|
[source,shell]
|
|
-----
|
|
bin/logstash --modules netflow --setup -M netflow.var.input.udp.port=NNNN
|
|
-----
|
|
+
|
|
Where `NNNN` is the UDP port on which Logstash will listen for network traffic
|
|
data. If you don't specify a port, Logstash listens on port 2055 by default.
|
|
+
|
|
The `--modules netflow` option spins up a Netflow-aware Logstash pipeline
|
|
for ingestion.
|
|
+
|
|
The `--setup` option creates a `netflow-*` index pattern in Elasticsearch and
|
|
imports Kibana dashboards and visualizations. Running `--setup` is a one-time
|
|
setup step. Omit this option for subsequent runs of the module to avoid
|
|
overwriting existing Kibana dashboards.
|
|
+
|
|
The command shown here assumes that you're running Elasticsearch and Kibana on
|
|
your localhost. If you're not, you need to specify additional connection
|
|
options. See <<configuring-netflow>>.
|
|
|
|
. Explore your data in Kibana:
|
|
.. Open your browser and navigate to
|
|
http://localhost:5601[http://localhost:5601]. If security is enabled, you'll
|
|
need to specify the Kibana username and password that you used when you set up
|
|
security.
|
|
.. Open *Netflow: Network Overview Dashboard*.
|
|
.. See <<exploring-data-netflow>> for additional details on data exploration.
|
|
|
|
[[exploring-data-netflow]]
|
|
==== Exploring Your Data
|
|
|
|
Once the Logstash Netflow module starts processing events, you can immediately
|
|
begin using the packaged Kibana dashboards to explore and visualize your
|
|
network flow data.
|
|
|
|
You can use the dashboards as-is, or tailor them to work better with existing
|
|
use cases and business requirements.
|
|
|
|
[[network-dashboards-netflow]]
|
|
===== Example Dashboards
|
|
|
|
On the *Overview* dashboard, you can see a summary of basic traffic data and set
|
|
up filters before you drill down to gain deeper insight into the data.
|
|
|
|
[role="screenshot"]
|
|
image::static/images/netflow-overview.png[Netflow overview dashboard]
|
|
|
|
For example, on the *Conversation Partners* dashboard, you can see the source
|
|
and destination addresses of the client and server in any conversation.
|
|
|
|
[role="screenshot"]
|
|
image::static/images/netflow-conversation-partners.png[Netflow conversation partners dashboard]
|
|
|
|
On the *Traffic Analysis* dashboard, you can identify high volume conversations
|
|
by viewing the traffic volume in bytes.
|
|
|
|
[role="screenshot"]
|
|
image::static/images/netflow-traffic-analysis.png[Netflow traffic analysis dashboard]
|
|
|
|
Then you can go to the *Geo Location* dashboard where you can visualize the
|
|
location of destinations and sources on a heat map.
|
|
|
|
[role="screenshot"]
|
|
image::static/images/netflow-geo-location.png[Netflow geo location dashboard]
|
|
|
|
|
|
[[configuring-netflow]]
|
|
==== Configuring the Module
|
|
|
|
You can further refine the behavior of the Logstash Netflow module by specifying
|
|
settings in the `logstash.yml` settings file, or overriding settings at the
|
|
command line.
|
|
|
|
For example, the following configuration in the `logstash.yml` file sets
|
|
Logstash to listen on port 9996 for network traffic data:
|
|
[source,yaml]
|
|
-----
|
|
modules:
|
|
- name: netflow
|
|
var.input.udp.port: 9996
|
|
-----
|
|
|
|
To specify the same settings at the command line, you use:
|
|
|
|
[source,shell]
|
|
-----
|
|
bin/logstash --modules netflow -M netflow.var.input.udp.port=9996
|
|
-----
|
|
|
|
For more information about configuring modules, see
|
|
<<logstash-modules>>.
|
|
|
|
[[netflow-module-config]]
|
|
===== Configuration Options
|
|
|
|
The Netflow module provides the following settings for configuring the behavior
|
|
of the module. These settings include Netflow-specific options plus common
|
|
options that are supported by all Logstash modules.
|
|
|
|
When you override a setting at the command line, remember to prefix the setting
|
|
with the module name, for example, `netflow.var.input.udp.port` instead of
|
|
`var.input.udp.port`.
|
|
|
|
If you don't specify configuration settings, Logstash uses the defaults.
|
|
|
|
*Netflow Options*
|
|
|
|
*`var.input.udp.port:`*::
|
|
+
|
|
--
|
|
* Value type is <<number,number>>
|
|
* Default value is 2055.
|
|
--
|
|
+
|
|
Sets the UDP port on which Logstash listens for network traffic data. Although
|
|
2055 is the default for this setting, some devices use ports in the range of
|
|
9995 through 9998, with 9996 being the most commonly used alternative.
|
|
|
|
*`var.input.udp.workers:`*::
|
|
+
|
|
--
|
|
* Value type is <<number,number>>
|
|
* Default value is 2.
|
|
--
|
|
+
|
|
Number of threads processing packets.
|
|
|
|
*`var.input.udp.receive_buffer_bytes:`*::
|
|
+
|
|
--
|
|
* Value type is <<number,number>>
|
|
* Default value is 212992.
|
|
--
|
|
+
|
|
The socket receive buffer size in bytes.
|
|
The operating system will use the max allowed value if receive_buffer_bytes is larger than allowed.
|
|
Consult your operating system documentation if you need to increase this max allowed value.
|
|
|
|
*`var.input.udp.queue_size:`*::
|
|
+
|
|
--
|
|
* Value type is <<number,number>>
|
|
* Default value is 2000.
|
|
--
|
|
+
|
|
This is the number of unprocessed UDP packets you can hold in memory before
|
|
packets will start dropping.
|
|
|
|
include::shared-module-options.asciidoc[]
|