Add netflow module docs

Fixes #8071

docs/index-shared2.asciidoc

@@ -35,6 +35,8 @@ include::static/modules.asciidoc[]
 
 include::static/arcsight-module.asciidoc[]
 
+include::static/netflow-module.asciidoc[]
+
 // Working with Filebeat Modules
 
 :edit_url: https://github.com/elastic/logstash/edit/master/docs/static/filebeat-modules.asciidoc

docs/static/netflow-module.asciidoc

@@ -0,0 +1,158 @@
+[[netflow-module]]
+=== Logstash Netflow Module
+
+++++
+<titleabbrev>Netflow Module</titleabbrev>
+++++
+
+The Logstash Netflow module simplifies the collection, normalization, and
+visualization of network flow data. With a single command, the module parses
+network flow data, indexes the events into Elasticsearch, and installs a suite
+of Kibana dashboards to get you exploring your data immediately. 
+
+Logstash modules support Netflow Version 5 and 9.
+
+==== What is Flow Data?
+
+Netflow is a type of data record streamed from capable network devices. It
+contains information about connections traversing the device, and includes
+source IP addresses and ports, destination IP addresses and ports, types of
+service, VLANs, and other information that can be encoded into frame and
+protocol headers. With Netflow data, network operators can go beyond monitoring
+simply the volume of data crossing their networks. They can understand where the
+traffic originated, where it is going, and what services or applications it is
+part of.
+
+[[netflow-requirements]]
+===== Requirements
+
+These instructions assume you have already installed Elastic Stack
+(Logstash, Elasticsearch, and Kibana) version 5.6 or higher. The products you
+need are https://www.elastic.co/downloads[available to download] and easy to
+install.
+
+[[netflow-getting-started]]
+==== Getting Started
+
+. Start the Logstash Netflow module by running the following command in the
+Logstash installation directory:
++
+[source,shell]
+-----
+bin/logstash --modules netflow --setup -M netflow.var.input.udp.port=NNNN
+-----
++
+Where `NNNN` is the UDP port on which Logstash will listen for network traffic
+data. If you don't specify a port, Logstash listens on port 2055 by default.
++
+The `--modules netflow` option spins up a Netflow-aware Logstash pipeline
+for ingestion.
++
+The `--setup` option creates a `netflow-*` index pattern in Elasticsearch and
+imports Kibana dashboards and visualizations. Running `--setup` is a one-time
+setup step. Omit this option for subsequent runs of the module to avoid
+overwriting existing Kibana dashboards.
++
+The command shown here assumes that you're running Elasticsearch and Kibana on
+your localhost. If you're not, you need to specify additional connection
+options. See <<configuring-netflow>>.
+
+. Explore your data in Kibana:
+.. Open your browser and navigate to
+http://localhost:5601[http://localhost:5601]. If security is enabled, you'll
+need to specify the Kibana username and password that you used when you set up
+security.
+.. Open *Netflow: Network Overview Dashboard*.
+.. See <<exploring-data-netflow>> for additional details on data exploration.
+
+[[exploring-data-netflow]]
+==== Exploring Your Data
+
+Once the Logstash Netflow module starts processing events, you can immediately
+begin using the packaged Kibana dashboards to explore and visualize your
+network flow data. 
+
+You can use the dashboards as-is, or tailor them to work better with existing
+use cases and business requirements.
+
+[[network-dashboards-netflow]]
+===== Example Dashboards
+
+On the *Overview* dashboard, you can see a summary of basic traffic data and set
+up filters before you drill down to gain deeper insight into the data.
+
+[role="screenshot"]
+image::static/images/netflow-overview.png[Netflow overview dashboard]
+
+For example, on the *Conversation Partners* dashboard, you can see the source
+and destination addresses of the client and server in any conversation.
+
+[role="screenshot"]
+image::static/images/netflow-conversation-partners.png[Netflow conversation partners dashboard]
+
+On the *Traffic Analysis* dashboard, you can identify high volume conversations
+by viewing the traffic volume in bytes.
+
+[role="screenshot"]
+image::static/images/netflow-traffic-analysis.png[Netflow traffic analysis dashboard]
+
+Then you can go to the *Geo Location* dashboard where you can visualize the
+location of destinations and sources on a heat map. 
+
+[role="screenshot"]
+image::static/images/netflow-geo-location.png[Netflow geo location dashboard]
+
+
+[[configuring-netflow]]
+==== Configuring the Module
+
+You can further refine the behavior of the Logstash Netflow module by specifying
+settings in the `logstash.yml` settings file, or overriding settings at the
+command line. 
+
+For example, the following configuration in the `settings.yml` file sets
+Logstash to listen on port 9996 for network traffic data: 
+[source,yaml]
+-----
+modules:
+  - name: netflow
+    var.input.udp.port: 9996
+-----
+
+To specify the same settings at the command line, you use:
+
+[source,shell]
+-----
+bin/logstash --modules netflow -M netflow.var.input.udp.port=9996
+-----
+
+For more information about configuring modules, see
+<<logstash-modules>>.
+
+[[netflow-module-config]]
+===== Configuration Options
+
+The Netflow module provides the following settings for configuring the behavior
+of the module. These settings include Netflow-specific options plus common
+options that are supported by all Logstash modules. 
+
+When you override a setting at the command line, remember to prefix the setting
+with the module name, for example,  `netflow.var.input.udp.port` instead of
+`var.input.udp.port`.
+
+If you don't specify configuration settings, Logstash uses the defaults.
+
+*Netflow Options*
+
+*`var.input.udp.port:`*::
++
+--
+* Value type is <<number,number>>
+* Default value is 2055. 
+--
++
+Sets the UDP port on which Logstash listens for network traffic data. Although
+2055 is the default for this setting, some devices use ports in the range of
+9995 through 9998, with 9996 being the most commonly used alternative.
+
+include::shared-module-options.asciidoc[]

docs/static/shared-module-options.asciidoc

@@ -0,0 +1,169 @@
+*Common options*
+
+The following configuration options are supported by all modules:
+
+*`var.elasticsearch.hosts`*::
++
+--
+* Value type is <<uri,uri>>
+* Default value is “localhost:9200”
+--
++
+Sets the host(s) of the Elasticsearch cluster. If given an <<array,array>> it
+will load balance requests across the hosts specified in the hosts parameter. It
+is important to exclude {ref}/modules-node.html[dedicated master nodes] from the
+hosts list to prevent Logstash from sending bulk requests to the master nodes.
+So this parameter should only reference either data or client nodes in
+Elasticsearch.
++
+Any special characters present in the URLs here MUST be URL escaped! This means #
+should be put in as %23 for instance.
+
+*`var.elasticsearch.username`*::
++
+--
+* Value type is <<string,string>>
+* Default value is “elastic”
+--
++
+The username to authenticate to a secure Elasticsearch cluster.
+
+*`var.elasticsearch.password`*::
++
+--
+* Value type is <<string,string>>
+* Default value is “changeme”
+--
++
+The password to authenticate to a secure Elasticsearch cluster.
+
+*`var.elasticsearch.ssl.enabled`*::
++
+--
+* Value type is <<boolean,boolean>>
+* There is no default value for this setting.
+--
++
+Enable SSL/TLS secured communication to the Elasticsearch cluster. Leaving this
+unspecified will use whatever scheme is specified in the URLs listed in `hosts`.
+If no explicit protocol is specified, plain HTTP will be used. If SSL is
+explicitly disabled here, the plugin will refuse to start if an HTTPS URL is
+given in hosts.
+
+*`var.elasticsearch.ssl.verification_mode`*::
++
+--
+* Value type is <<string,string>>
+* Default value is "strict"
+--
++
+The hostname verification setting when communicating with Elasticsearch. Set to
+`disable` to turn off hostname verification. Disabling this has serious security
+concerns.
+
+*`var.elasticsearch.ssl.certificate_authority`*::
++
+--
+* Value type is <<string,string>>
+* There is no default value for this setting
+--
++
+The path to an X.509 certificate to use to validate SSL certificates when
+communicating with Kibana.
+
+*`var.elasticsearch.ssl.certificate`*::
++
+--
+* Value type is <<string,string>>
+* There is no default value for this setting
+--
++
+The path to an X.509 certificate to use for client authentication when
+communicating with Elasticsearch.
+
+*`var.elasticsearch.ssl.key`*::
++
+--
+* Value type is <<string,string>>
+* There is no default value for this setting
+--
++
+The path to the certificate key for client authentication when communicating
+with Elasticsearch.
+
+*`var.kibana.host`*::
++
+--
+* Value type is <<string,string>>
+* Default value is “localhost:5601”
+--
++
+Sets the host of the Kibana instance to import dashboards and visualizations.
+
+*`var.kibana.username`*::
++
+--
+* Value type is <<string,string>>
+* Default value is “elastic”
+--
++
+The username to authenticate to a secured Kibana instance.
+
+*`var.kibana.password`*::
++
+--
+* Value type is <<string,string>>
+* Default value is “changeme”
+--
++
+The password to authenticate to a secure Kibana instance.
+
+*`var.kibana.ssl.enabled`*::
++
+--
+* Value type is <<boolean,boolean>>
+* Default value is false
+--
++
+Enable SSL/TLS secured communication to the Kibana instance.
+
+*`var.kibana.ssl.verification_mode`*::
++
+--
+* Value type is <<string,string>>
+* Default value is "strict"
+--
++
+The hostname verification setting when communicating with Kibana. Set to
+`disable` to turn off hostname verification. Disabling this has serious security
+concerns.
+
+*`var.kibana.ssl.certificate_authority`*::
++
+--
+* Value type is <<string,string>>
+* There is no default value for this setting
+--
++
+The path to an X.509 certificate to use to validate SSL certificates when
+communicating with Kibana.
+
+*`var.kibana.ssl.certificate`*::
++
+--
+* Value type is <<string,string>>
+* There is no default value for this setting
+--
++
+The path to an X.509 certificate to use for client authentication when
+communicating with Kibana.
+
+*`var.kibana.ssl.key`*::
++
+--
+* Value type is <<string,string>>
+* There is no default value for this setting
+--
++
+The path to the certificate key for client authentication when communicating
+with Kibana.