[[tutorials]]
== Tutorials

Learn how to use *Lens*, *Vega*, and *Timelion* by going through one of the step-by-step tutorials.

[[lens-tutorial]]
=== Compare sales over time with Lens

Ready to create your own visualization with Lens? Use the following tutorial to create a visualization that lets you compare sales over time.

[float]
[[lens-before-begin]]
==== Before you begin

To start, you'll need to add the <<add-sample-data, sample ecommerce data>>.

[float]
==== Build the visualization

Drag and drop your data onto the visualization builder pane.

. Select the *kibana_sample_data_ecommerce* index pattern.

. Click image:images/time-filter-calendar.png[], then click *Last 7 days*. 
+
The fields in the data panel update.

. Drag and drop the *taxful_total_price* data field to the visualization builder pane.
+
[role="screenshot"]
image::images/lens_tutorial_1.png[Lens tutorial]

To display the average order prices over time, *Lens* automatically added in *order_date* field.

To break down your data, drag the *category.keyword* field to the visualization builder pane. Lens
knows that you want to show the top categories and compare them across the dates,
and creates a chart that compares the sales for each of the top three categories:

[role="screenshot"]
image::images/lens_tutorial_2.png[Lens tutorial]

[float]
[[customize-lens-visualization]]
==== Customize your visualization

Make your visualization look exactly how you want with the customization options.

. Click *Average of taxful_total_price*, then change the *Label* to `Sales`.
+
[role="screenshot"]
image::images/lens_tutorial_3.1.png[Lens tutorial]

. Click *Top values of category.keyword*, then change *Number of values* to `10`.
+
[role="screenshot"]
image::images/lens_tutorial_3.2.png[Lens tutorial]
+
The visualization updates to show there are only six available categories.
+
Look at the *Suggestions*. An area chart is not an option, but for the sales data, a stacked area chart might be the best option.

. To switch the chart type, click *Stacked bar chart* in the column, then click *Stacked area* from the *Select a visualizations* window.
+
[role="screenshot"]
image::images/lens_tutorial_3.png[Lens tutorial]

[float]
[[lens-tutorial-next-steps]]
==== Next steps

Now that you've created your visualization, you can add it to a <<dashboard,dashboard>> or <<canvas,Canvas workpad>>.

[[vega-lite-tutorial-create-your-first-visualizations]]
=== Create your first visualization with Vega-Lite

experimental[]

In this tutorial, you will learn about how to edit Vega-Lite in {kib} to create
a stacked area chart from an {es} search query. It will give you a starting point
for a more comprehensive
https://vega.github.io/vega-lite/tutorials/getting_started.html[introduction to Vega-Lite],
while only covering the basics.

In this tutorial, you will build a stacked area chart from one of the {kib} sample data
sets.

[role="screenshot"]
image::visualize/images/vega_lite_tutorial_1.png[]

Before beginning this tutorial, install the <<add-sample-data, eCommerce sample data>>
set.

When you first open the Vega editor in {kib}, you will see a pre-populated
line chart which shows the total number of documents across all your indices
within the time range.

[role="screenshot"]
image::visualize/images/vega_lite_default.png[]

The text editor contains a Vega-Lite spec written in https://hjson.github.io/[HJSON],
which is similar to JSON but optimized for human editing. HJSON supports:

* Comments using // or /* syntax
* Object keys without quotes
* String values without quotes
* Optional commas
* Double or single quotes
* Multiline strings

[float]
[[small-steps]]
==== Small steps

Always work on Vega in the smallest steps possible, and save your work frequently.
Small changes will cause unexpected results. Click the "Save" button now.

The first step is to change the index to one of the <<add-sample-data, sample data>>
sets. Change

```yaml
index: _all
```

to:

```yaml
index: kibana_sample_data_ecommerce
```

Click "Update". The result is probably not what you expect. You should see a flat
line with 0 results.

You've only changed the index, so the difference must be the query is returning
no results. You can try the <<vega-browser-debugging-console, Vega debugging process>>,
but intuition may be faster for this particular problem.

In this case, the problem is that you are querying the field `@timestamp`,
which does not exist in the `kibana_sample_data_ecommerce` data. Find and replace
`@timestamp` with `order_date`. This fixes the problem, leaving you with this spec:

.Expand Vega-Lite spec
[%collapsible%closed]
====
[source,yaml]
----
{
  $schema: https://vega.github.io/schema/vega-lite/v4.json
  title: Event counts from ecommerce
  data: {
    url: {
      %context%: true
      %timefield%: order_date
      index: kibana_sample_data_ecommerce
      body: {
        aggs: {
          time_buckets: {
            date_histogram: {
              field: order_date
              interval: {%autointerval%: true}
              extended_bounds: {
                min: {%timefilter%: "min"}
                max: {%timefilter%: "max"}
              }
              min_doc_count: 0
            }
          }
        }
        size: 0
      }
    }
    format: {property: "aggregations.time_buckets.buckets" }
  }

  mark: line

  encoding: {
    x: {
      field: key
      type: temporal
      axis: { title: null }
    }
    y: {
      field: doc_count
      type: quantitative
      axis: { title: "Document count" }
    }
  }
}
----

====

Now, let's make the visualization more interesting by adding another aggregation
to create a stacked area chart. To verify that you have constructed the right
query, it is easiest to use the {kib} Dev Tools in a separate tab from the
Vega editor. Open the Dev Tools from the Management section of the navigation.

This query is roughly equivalent to the one that is used in the default
Vega-Lite spec. Copy it into the Dev Tools:

```js
POST kibana_sample_data_ecommerce/_search
{
  "query": {
    "range": {
      "order_date": {
        "gte": "now-7d"
      }
    }
  },
  "aggs": {
    "time_buckets": {
      "date_histogram": {
        "field": "order_date",
        "fixed_interval": "1d",
        "extended_bounds": {
          "min": "now-7d"
        },
        "min_doc_count": 0
      }
    }
  },
  "size": 0
}
```

There's not enough data to create a stacked bar in the original query, so we
will add a new
{ref}/search-aggregations-bucket-terms-aggregation.html[terms aggregation]:

```js
POST kibana_sample_data_ecommerce/_search
{
  "query": {
    "range": {
      "order_date": {
        "gte": "now-7d"
      }
    }
  },
  "aggs": {
    "categories": {
      "terms": { "field": "category.keyword" },
      "aggs": {
        "time_buckets": {
          "date_histogram": {
            "field": "order_date",
            "fixed_interval": "1d",
            "extended_bounds": {
              "min": "now-7d"
            },
            "min_doc_count": 0
          }
        }
      }
    }
  },
  "size": 0
}
```

You'll see that the response format looks different from the previous query:

```json
{
  "aggregations" : {
    "categories" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [{
        "key" : "Men's Clothing",
        "doc_count" : 1661,
        "time_buckets" : {
          "buckets" : [{
            "key_as_string" : "2020-06-30T00:00:00.000Z",
            "key" : 1593475200000,
            "doc_count" : 19
          }, {
            "key_as_string" : "2020-07-01T00:00:00.000Z",
            "key" : 1593561600000,
            "doc_count" : 71
          }]
        }
      }]
    }
  }
}
```

Now that we have data that we're happy with, it's time to convert from an
isolated {es} query into a query with {kib} integration. Looking at the
<<vega-queries, reference for writing {es} queries in Vega>>, you will
see the full list of special tokens that are used in this query, such
as `%context: true`. This query has also replaced `"fixed_interval": "1d"`
with `interval: {%autointerval%: true}`. Copy the final query into
your spec:

```yaml
  data: {
    url: {
      %context%: true
      %timefield%: order_date
      index: kibana_sample_data_ecommerce
      body: {
        aggs: {
          categories: {
            terms: { field: "category.keyword" }
            aggs: {
              time_buckets: {
                date_histogram: {
                  field: order_date
                  interval: {%autointerval%: true}
                  extended_bounds: {
                    min: {%timefilter%: "min"}
                    max: {%timefilter%: "max"}
                  }
                  min_doc_count: 0
                }
              }
            }
          }
        }
        size: 0
      }
    }
    format: {property: "aggregations.categories.buckets" }
  }
```

If you copy and paste that into your Vega-Lite spec, and click "Update",
you will see a warning saying `Infinite extent for field "key": [Infinity, -Infinity]`.
Let's use our <<vega-browser-debugging-console, Vega debugging skills>> to understand why.

Vega-Lite generates data using the names `source_0` and `data_0`. `source_0` contains
the results from the {es} query, and `data_0` contains the visually encoded results
which are shown in the chart. To debug this problem, you need to compare both.

To look at the source, open the browser dev tools console and type
`VEGA_DEBUG.view.data('source_0')`. You will see:

```js
[{
  doc_count: 454
  key: "Men's Clothing"
  time_buckets: {buckets: Array(57)}
  Symbol(vega_id): 12822
}, ...]
```

To compare to the visually encoded data, open the browser dev tools console and type
`VEGA_DEBUG.view.data('data_0')`. You will see:

```js
[{
  doc_count: 454
  key: NaN
  time_buckets: {buckets: Array(57)}
  Symbol(vega_id): 13879
}]
```

The issue seems to be that the `key` property is not being converted the right way,
which makes sense because the `key` is now `Men's Clothing` instead of a timestamp.

To fix this, try updating the `encoding` of your Vega-Lite spec to:

```yaml
  encoding: {
    x: {
      field: time_buckets.buckets.key
      type: temporal
      axis: { title: null }
    }
    y: {
      field: time_buckets.buckets.doc_count
      type: quantitative
      axis: { title: "Document count" }
    }
  }
```

This will show more errors, and you can inspect `VEGA_DEBUG.view.data('data_0')` to
understand why. This now shows:

```js
[{
  doc_count: 454
  key: "Men's Clothing"
  time_buckets: {buckets: Array(57)}
  time_buckets.buckets.doc_count: undefined
  time_buckets.buckets.key: null
  Symbol(vega_id): 14094
}]
```

It looks like the problem is that the `time_buckets` inner array is not being
extracted by Vega. The solution is to use a Vega-lite
https://vega.github.io/vega-lite/docs/flatten.html[flatten transformation], available in {kib} 7.9 and later.
If using an older version of Kibana, the flatten transformation is available in Vega
but not Vega-Lite.

Add this section in between the `data` and `encoding` section:

```yaml
  transform: [{
    flatten: ["time_buckets.buckets"]
  }]
```

This does not yet produce the results you expect. Inspect the transformed data
by typing `VEGA_DEBUG.view.data('data_0')` into the console again:

```js
[{
  doc_count: 453
  key: "Men's Clothing"
  time_bucket.buckets.doc_count: undefined
  time_buckets: {buckets: Array(57)}
  time_buckets.buckets: {
    key_as_string: "2020-06-30T15:00:00.000Z",
    key: 1593529200000,
    doc_count: 2
  }
  time_buckets.buckets.key: null
  Symbol(vega_id): 21564
}]
```

The debug view shows `undefined` values where you would expect to see numbers, and
the cause is that there are duplicate names which are confusing Vega-Lite. This can
be fixed by making this change to the `transform` and `encoding` blocks:

```yaml
  transform: [{
    flatten: ["time_buckets.buckets"],
    as: ["buckets"]
  }]

  mark: area

  encoding: {
    x: {
      field: buckets.key
      type: temporal
      axis: { title: null }
    }
    y: {
      field: buckets.doc_count
      type: quantitative
      axis: { title: "Document count" }
    }
    color: {
      field: key
      type: nominal
    }
  }
```

At this point, you have a stacked area chart that shows the top categories,
but the chart is still missing some common features that we expect from a {kib}
visualization. Let's add hover states and tooltips next.

Hover states are handled differently in Vega-Lite and Vega. In Vega-Lite this is
done using a concept called `selection`, which has many permutations that are not
covered in this tutorial. We will be adding a simple tooltip and hover state.

Because {kib} has enabled the https://vega.github.io/vega-lite/docs/tooltip.html[Vega tooltip plugin],
tooltips can be defined in several ways:

* Automatic tooltip based on the data, via `{ content: "data" }`
* Array of fields, like `[{ field: "key", type: "nominal" }]`
* Defining a custom Javascript object using the `calculate` transform

For the simple tooltip, add this to your encoding:

```yaml
  encoding: {
    tooltip: [{
      field: buckets.key
      type: temporal
      title: "Date"
    }, {
      field: key
      type: nominal
      title: "Category"
    }, {
      field: buckets.doc_count
      type: quantitative
      title: "Count"
    }]
  }
```

As you hover over the area series in your chart, a multi-line tooltip will
appear, but it won't indicate the nearest point that it's pointing to. To
indicate the nearest point, we need to add a second layer.

The first step is to remove the `mark: area` from your visualization.
Once you've removed the previous mark, add a composite mark at the end of
the Vega-Lite spec:

```yaml
  layer: [{
    mark: area
  }, {
    mark: point
  }]
```

You'll see that the points are not appearing to line up with the area chart,
and the reason is that the points are not being stacked. Change your Y encoding
to this:

```yaml
    y: {
      field: buckets.doc_count
      type: quantitative
      axis: { title: "Document count" }
      stack: true
    }
```

Now, we will add a `selection` block inside the point mark:

```yaml
  layer: [{
    mark: area
  }, {
    mark: point
    
    selection: {
      pointhover: {
        type: single
        on: mouseover
        clear: mouseout
        empty: none
        fields: ["buckets.key", "key"]
        nearest: true
      }
    }

    encoding: {
      size: {
        condition: {
          selection: pointhover
          value: 100
        }
        value: 5
      }
      fill: {
        condition: {
          selection: pointhover
          value: white
        }
      }
    }
  }]
```

Now that you've enabled a selection, try moving the mouse around the visualization
and seeing the points respond to the nearest position:

[role="screenshot"]
image::visualize/images/vega_lite_tutorial_2.png[]

The final result of this tutorial is this spec:

.Expand final Vega-Lite spec
[%collapsible%closed]
====
[source,yaml]
----
{
  $schema: https://vega.github.io/schema/vega-lite/v4.json
  title: Event counts from ecommerce
  data: {
    url: {
      %context%: true
      %timefield%: order_date
      index: kibana_sample_data_ecommerce
      body: {
        aggs: {
          categories: {
            terms: { field: "category.keyword" }
            aggs: {
              time_buckets: {
                date_histogram: {
                  field: order_date
                  interval: {%autointerval%: true}
                  extended_bounds: {
                    min: {%timefilter%: "min"}
                    max: {%timefilter%: "max"}
                  }
                  min_doc_count: 0
                }
              }
            }
          }
        }
        size: 0
      }
    }
    format: {property: "aggregations.categories.buckets" }
  }
  
  transform: [{
    flatten: ["time_buckets.buckets"]
    as: ["buckets"]
  }]

  encoding: {
    x: {
      field: buckets.key
      type: temporal
      axis: { title: null }
    }
    y: {
      field: buckets.doc_count
      type: quantitative
      axis: { title: "Document count" }
      stack: true
    }
    color: {
      field: key
      type: nominal
      title: "Category"
    }
    tooltip: [{
      field: buckets.key
      type: temporal
      title: "Date"
    }, {
      field: key
      type: nominal
      title: "Category"
    }, {
      field: buckets.doc_count
      type: quantitative
      title: "Count"
    }]
  }
  
  layer: [{
    mark: area
  }, {
    mark: point
    
    selection: {
      pointhover: {
        type: single
        on: mouseover
        clear: mouseout
        empty: none
        fields: ["buckets.key", "key"]
        nearest: true
      }
    }

    encoding: {
      size: {
        condition: {
          selection: pointhover
          value: 100
        }
        value: 5
      }
      fill: {
        condition: {
          selection: pointhover
          value: white
        }
      }
    }
  }]
}
----

====

[[vega-tutorial-update-kibana-filters-from-vega]]
=== Update {kib} filters from Vega

experimental[]

In this tutorial you will build an area chart in Vega using an {es} search query,
and add a click handler and drag handler to update {kib} filters.
This tutorial is not a full https://vega.github.io/vega/tutorials/[Vega tutorial],
but will cover the basics of creating Vega visualizations into {kib}.

First, create an almost-blank Vega chart by pasting this into the editor:

```yaml
{
  $schema: "https://vega.github.io/schema/vega/v5.json"
  data: [{
    name: source_0
  }]
  
  scales: [{
    name: x
    type: time
    range: width
  }, {
    name: y
    type: linear
    range: height
  }]
  
  axes: [{
    orient: bottom
    scale: x
  }, {
    orient: left
    scale: y
  }]
  
  marks: [
    {
      type: area
      from: {
        data: source_0
      }
      encode: {
        update: {
        }
      }
    }
  ]
}
```

Despite being almost blank, this Vega spec still contains the minimum requirements:

* Data
* Scales
* Marks
* (optional) Axes

Next, add a valid {es} search query in the `data` block:

```yaml
  data: [
    {
      name: source_0
      url: {
        %context%: true
        %timefield%: order_date
        index: kibana_sample_data_ecommerce
        body: {
          aggs: {
            time_buckets: {
              date_histogram: {
                field: order_date
                fixed_interval: "3h"
                extended_bounds: {
                  min: {%timefilter%: "min"}
                  max: {%timefilter%: "max"}
                }
                min_doc_count: 0
              }
            }
          }
          size: 0
        }
      }
      format: { property: "aggregations.time_buckets.buckets" }
    }
  ]
```

Click "Update", and nothing will change in the visualization. The first step
is to change the X and Y scales based on the data:

```yaml
  scales: [{
    name: x
    type: time
    range: width
    domain: {
      data: source_0
      field: key
    }
  }, {
    name: y
    type: linear
    range: height
    domain: {
      data: source_0
      field: doc_count
    }
  }]
```

Click "Update", and you will see that the X and Y axes are now showing labels based
on the real data.

Next, encode the fields `key` and `doc_count` as the X and Y values:

```yaml
  marks: [
    {
      type: area
      from: {
        data: source_0
      }
      encode: {
        update: {
          x: {
            scale: x
            field: key
          }
          y: {
            scale: y
            value: 0
          }
          y2: {
            scale: y
            field: doc_count
          }
        }
      }
    }
  ]
```

Click "Update" and you will get a basic area chart:

[role="screenshot"]
image::visualize/images/vega_tutorial_3.png[]

Next, add a new block to the `marks` section. This will show clickable points to filter for a specific
date:

```yaml
  {
    name: point
    type: symbol
    style: ["point"]
    from: {
      data: source_0
    }
    encode: {
      update: {
        x: {
          scale: x
          field: key
        }
        y: {
          scale: y
          field: doc_count
        }
        size: {
          value: 100
        }
        fill: {
          value: black
        }
      }
    }
  }
```

Next, we will create a Vega signal to make the points clickable. You can access
the clicked `datum` in the expression used to update. In this case, you want
clicks on points to add a time filter with the 3-hour interval defined above.

```yaml
  signals: [
    {
      name: point_click
      on: [{
        events: {
          source: scope
          type: click
          markname: point
        }
        update: '''kibanaSetTimeFilter(datum.key, datum.key + 3 * 60 * 60 * 1000)'''
      }]
    }
  ]
```

This event is using the {kib} custom function `kibanaSetTimeFilter` to generate a filter that
gets applied to the entire dashboard on click.

The mouse cursor does not currently indicate that the chart is interactive. Find the `marks` section,
and update the mark named `point` by adding `cursor: { value: "pointer" }` to
the `encoding` section like this:

```yaml
  {
    name: point
    type: symbol
    style: ["point"]
    from: {
      data: source_0
    }
    encode: {
      update: {
        ...
        cursor: { value: "pointer" }
      }
    }
  }
```

Next, we will add a drag interaction which will allow the user to narrow into
a specific time range in the visualization. This will require adding more signals, and
adding a rectangle overlay:

[role="screenshot"]
image::visualize/images/vega_tutorial_4.png[]

The first step is to add a new `signal` to track the X position of the cursor:

```yaml
    {
      name: currentX
      value: -1
      on: [{
        events: {
          type: mousemove
          source: view
        },
        update: "clamp(x(), 0, width)"
      }, {
        events: {
          type: mouseout
          source: view
        }
        update: "-1"
      }]
    }
```

Now add a new `mark` to indicate the current cursor position:

```yaml
    {
      type: rule
      interactive: false
      encode: {
        update: {
          y: {value: 0}
          y2: {signal: "height"}
          stroke: {value: "gray"}
          strokeDash: {
            value: [2, 1]
          }
          x: {signal: "max(currentX,0)"}
          defined: {signal: "currentX > 0"}
        }
      }
    }
```

Next, add a signal to track the current selected range, which will update
until the user releases the mouse button or uses the escape key:


```yaml
    {
      name: selected
      value: [0, 0]
      on: [{
        events: {
          type: mousedown
          source: view
        }
        update: "[clamp(x(), 0, width), clamp(x(), 0, width)]"
      }, {
        events: {
          type: mousemove
          source: window
          consume: true
          between: [{
            type: mousedown
            source: view
          }, {
            merge: [{
              type: mouseup
              source: window
            }, {
              type: keydown
              source: window
              filter: "event.key === 'Escape'"
            }]
          }]
        }
        update: "[selected[0], clamp(x(), 0, width)]"
      }, {
        events: {
          type: keydown
          source: window
          filter: "event.key === 'Escape'"
        }
        update: "[0, 0]"
      }]
    }
```

Now that there is a signal which tracks the time range from the user, we need to indicate
the range visually by adding a new mark which only appears conditionally:

```yaml
    {
      type: rect
      name: selectedRect
      encode: {
        update: {
          height: {signal: "height"}
          fill: {value: "#333"}
          fillOpacity: {value: 0.2}
          x: {signal: "selected[0]"}
          x2: {signal: "selected[1]"}
          defined: {signal: "selected[0] !== selected[1]"}
        }
      }
    }
```

Finally, add a new signal which will update the {kib} time filter when the mouse is released while
dragging:

```yaml
    {
      name: applyTimeFilter
      value: null
      on: [{
        events: {
          type: mouseup
          source: view
        }
        update: '''selected[0] !== selected[1] ? kibanaSetTimeFilter(
               invert('x',selected[0]),
               invert('x',selected[1])) : null'''
      }]
    }
```

Putting this all together, your visualization now supports the main features of
standard visualizations in {kib}, but with the potential to add even more control.
The final Vega spec for this tutorial is here:

.Expand final Vega spec
[%collapsible%closed]
====
[source,yaml]
----
{
  $schema: "https://vega.github.io/schema/vega/v5.json"
  data: [
    {
      name: source_0
      url: {
        %context%: true
        %timefield%: order_date
        index: kibana_sample_data_ecommerce
        body: {
          aggs: {
            time_buckets: {
              date_histogram: {
                field: order_date
                fixed_interval: "3h"
                extended_bounds: {
                  min: {%timefilter%: "min"}
                  max: {%timefilter%: "max"}
                }
                min_doc_count: 0
              }
            }
          }
          size: 0
        }
      }
      format: { property: "aggregations.time_buckets.buckets" }
    }
  ]
  
  scales: [{
    name: x
    type: time
    range: width
    domain: {
      data: source_0
      field: key
    }
  }, {
    name: y
    type: linear
    range: height
    domain: {
      data: source_0
      field: doc_count
    }
  }]
  
  axes: [{
    orient: bottom
    scale: x
  }, {
    orient: left
    scale: y
  }]
  
  marks: [
    {
      type: area
      from: {
        data: source_0
      }
      encode: {
        update: {
          x: {
            scale: x
            field: key
          }
          y: {
            scale: y
            value: 0
          }
          y2: {
            scale: y
            field: doc_count
          }
        }
      }
    },
    {
      name: point
      type: symbol
      style: ["point"]
      from: {
        data: source_0
      }
      encode: {
        update: {
          x: {
            scale: x
            field: key
          }
          y: {
            scale: y
            field: doc_count
          }
          size: {
            value: 100
          }
          fill: {
            value: black
          }
          cursor: { value: "pointer" }
        }
      }
    },
    {
      type: rule
      interactive: false
      encode: {
        update: {
          y: {value: 0}
          y2: {signal: "height"}
          stroke: {value: "gray"}
          strokeDash: {
            value: [2, 1]
          }
          x: {signal: "max(currentX,0)"}
          defined: {signal: "currentX > 0"}
        }
      }
    },
    {
      type: rect
      name: selectedRect
      encode: {
        update: {
          height: {signal: "height"}
          fill: {value: "#333"}
          fillOpacity: {value: 0.2}
          x: {signal: "selected[0]"}
          x2: {signal: "selected[1]"}
          defined: {signal: "selected[0] !== selected[1]"}
        }
      }
    }
  ]
  
  signals: [
    {
      name: point_click
      on: [{
        events: {
          source: scope
          type: click
          markname: point
        }
        update: '''kibanaSetTimeFilter(datum.key, datum.key + 3 * 60 * 60 * 1000)'''
      }]
    }
    {
      name: currentX
      value: -1
      on: [{
        events: {
          type: mousemove
          source: view
        },
        update: "clamp(x(), 0, width)"
      }, {
        events: {
          type: mouseout
          source: view
        }
        update: "-1"
      }]
    }
    {
      name: selected
      value: [0, 0]
      on: [{
        events: {
          type: mousedown
          source: view
        }
        update: "[clamp(x(), 0, width), clamp(x(), 0, width)]"
      }, {
        events: {
          type: mousemove
          source: window
          consume: true
          between: [{
            type: mousedown
            source: view
          }, {
            merge: [{
              type: mouseup
              source: window
            }, {
              type: keydown
              source: window
              filter: "event.key === 'Escape'"
            }]
          }]
        }
        update: "[selected[0], clamp(x(), 0, width)]"
      }, {
        events: {
          type: keydown
          source: window
          filter: "event.key === 'Escape'"
        }
        update: "[0, 0]"
      }]
    }
    {
      name: applyTimeFilter
      value: null
      on: [{
        events: {
          type: mouseup
          source: view
        }
        update: '''selected[0] !== selected[1] ? kibanaSetTimeFilter(
               invert('x',selected[0]),
               invert('x',selected[1])) : null'''
      }]
    }
  ]
}

----
====

[[timelion-tutorial-create-time-series-visualizations]]
=== Create time series visualizations with Timelion

To compare the real-time percentage of CPU time spent in user space to the results offset by one hour, create a time series visualization.

[float]
[[define-the-functions]]
==== Define the functions

To start tracking the real-time percentage of CPU, enter the following in the *Timelion Expression* field:

[source,text]
----------------------------------
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
----------------------------------

[role="screenshot"]
image::images/timelion-create01.png[]
{nbsp}

[float]
[[compare-the-data]]
==== Compare the data

To compare the two data sets, add another series with data from the previous hour, separated by a comma:

[source,text]
----------------------------------
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct'),
.es(offset=-1h,             <1>
    index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
----------------------------------

<1> `offset` offsets the data retrieval by a date expression. In this example, `-1h` offsets the data back by one hour.

[role="screenshot"]
image::images/timelion-create02.png[]
{nbsp}

[float]
[[add-label-names]]
==== Add label names

To easily distinguish between the two data sets, add the label names:

[source,text]
----------------------------------
.es(offset=-1h,index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct').label('last hour'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct').label('current hour') <1>
----------------------------------

<1> `.label()` adds custom labels to the visualization.

[role="screenshot"]
image::images/timelion-create03.png[]
{nbsp}

[float]
[[add-a-title]]
==== Add a title

Add a meaningful title:

[source,text]
----------------------------------
.es(offset=-1h,
    index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('last hour'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('current hour')
  .title('CPU usage over time') <1>
----------------------------------

<1> `.title()` adds a title with a meaningful name. Titles make is easier for unfamiliar users to understand the purpose of the visualization.

[role="screenshot"]
image::images/timelion-customize01.png[]
{nbsp}

[float]
[[change-the-chart-type]]
==== Change the chart type

To differentiate between the current hour data and the last hour data, change the chart type:

[source,text]
----------------------------------
.es(offset=-1h,
    index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('last hour')
  .lines(fill=1,width=0.5), <1>
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('current hour')
  .title('CPU usage over time')
----------------------------------

<1> `.lines()` changes the appearance of the chart lines. In this example, `.lines(fill=1,width=0.5)` sets the fill level to `1`, and the border width to `0.5`.

[role="screenshot"]
image::images/timelion-customize02.png[]
{nbsp}

[float]
[[change-the-line-colors]]
==== Change the line colors

To make the current hour data stand out, change the line colors:

[source,text]
----------------------------------
.es(offset=-1h,
    index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('last hour')
  .lines(fill=1,width=0.5)
  .color(gray), <1>
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('current hour')
  .title('CPU usage over time')
  .color(#1E90FF)
----------------------------------

<1> `.color()` changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data. In this example, `.color(gray)` represents the last hour, and `.color(#1E90FF)` represents the current hour.

[role="screenshot"]
image::images/timelion-customize03.png[]
{nbsp}

[float]
[[make-adjustments-to-the-legend]]
==== Make adjustments to the legend

Change the position and style of the legend:

[source,text]
----------------------------------
.es(offset=-1h,
    index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('last hour')
  .lines(fill=1,width=0.5)
  .color(gray),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='avg:system.cpu.user.pct')
  .label('current hour')
  .title('CPU usage over time')
  .color(#1E90FF)
  .legend(columns=2, position=nw) <1>
----------------------------------

<1> `.legend()` sets the position and style of the legend. In this example, `.legend(columns=2, position=nw)` places the legend in the north west position of the visualization with two columns.

[role="screenshot"]
image::images/timelion-customize04.png[]
{nbsp}

[[timelion-tutorial-create-visualizations-with-mathematical-functions]]
=== Timelion tutorial: Create visualizations with mathematical functions

To create a visualization for inbound and outbound network traffic, use mathematical functions.

[float]
[[mathematical-functions-define-functions]]
==== Define the functions

To start tracking the inbound and outbound network traffic, enter the following in the *Timelion Expression* field:

[source,text]
----------------------------------
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.in.bytes)
----------------------------------

[role="screenshot"]
image::images/timelion-math01.png[]
{nbsp}

[float]
[[mathematical-functions-plot-change]]
==== Plot the rate of change

Change how the data is displayed so that you can easily monitor the inbound traffic:

[source,text]
----------------------------------
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.in.bytes)
  .derivative() <1>
----------------------------------

<1> `.derivative` plots the change in values over time.

[role="screenshot"]
image::images/timelion-math02.png[]
{nbsp}

Add a similar calculation for outbound traffic:

[source,text]
----------------------------------
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.in.bytes)
  .derivative(),
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.out.bytes)
  .derivative()
  .multiply(-1) <1>
----------------------------------

<1> `.multiply()` multiplies the data series by a number, the result of a data series, or a list of data series. For this example, `.multiply(-1)` converts the outbound network traffic to a negative value since the outbound network traffic is leaving your machine.

[role="screenshot"]
image::images/timelion-math03.png[]
{nbsp}

[float]
[[mathematical-functions-convert-data]]
==== Change the data metric

To make the visualization easier to analyze, change the data metric from bytes to megabytes:

[source,text]
----------------------------------
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.in.bytes)
  .derivative()
  .divide(1048576),
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.out.bytes)
  .derivative()
  .multiply(-1)
  .divide(1048576) <1>
----------------------------------

<1> `.divide()` accepts the same input as `.multiply()`, then divides the data series by the defined divisor.

[role="screenshot"]
image::images/timelion-math04.png[]
{nbsp}

[float]
[[mathematical-functions-add-labels]]
==== Customize and format the visualization

Customize and format the visualization using functions:

[source,text]
----------------------------------
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.in.bytes)
  .derivative()
  .divide(1048576)
  .lines(fill=2, width=1)
  .color(green)
  .label("Inbound traffic")         <1>
  .title("Network traffic (MB/s)"), <2>
.es(index=metricbeat*,
    timefield=@timestamp,
    metric=max:system.network.out.bytes)
  .derivative()
  .multiply(-1)
  .divide(1048576)
  .lines(fill=2, width=1)           <3>
  .color(blue)                      <4>
  .label("Outbound traffic")
  .legend(columns=2, position=nw) <5>
----------------------------------

<1> `.label()` adds custom labels to the visualization.
<2> `.title()` adds a title with a meaningful name.
<3> `.lines()` changes the appearance of the chart lines. In this example, `.lines(fill=2, width=1)` sets the fill level to `2`, and the border width to `1`.
<4> `.color()` changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data. In this example, `.color(green)` represents the inbound network traffic, and `.color(blue)` represents the outbound network traffic.
<5> `.legend()` sets the position and style of the legend. For this example, `legend(columns=2, position=nw)` places the legend in the north west position of the visualization with two columns.

[role="screenshot"]
image::images/timelion-math05.png[]
{nbsp}

[[timelion-tutorial-create-visualizations-withconditional-logic-and-tracking-trends]]
=== Create visualizations with conditional logic and tracking trends using Timelion

To easily detect outliers and discover patterns over time, modify time series data with conditional logic and create a trend with a moving average.

With Timelion conditional logic, you can use the following operator values to compare your data:

[horizontal]
`eq`:: equal
`ne`:: not equal
`lt`:: less than
`lte`:: less than or equal to
`gt`:: greater than
`gte`:: greater than or equal to

[float]
[[conditional-define-functions]]
==== Define the functions

To chart the maximum value of `system.memory.actual.used.bytes`, enter the following in the *Timelion Expression* field:

[source,text]
----------------------------------
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
----------------------------------

[role="screenshot"]
image::images/timelion-conditional01.png[]
{nbsp}

[float]
[[conditional-track-memory]]
==== Track used memory

To track the amount of memory used, create two thresholds:

[source,text]
----------------------------------
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .if(gt,                             <1>
      11300000000,                    <2>
      .es(index=metricbeat-*,
          timefield='@timestamp',
          metric='max:system.memory.actual.used.bytes'),
      null)
    .label('warning')
    .color('#FFCC11'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .if(gt,
      11375000000,
      .es(index=metricbeat-*,
          timefield='@timestamp',
          metric='max:system.memory.actual.used.bytes'),
      null)
  .label('severe')
  .color('red')
----------------------------------

<1> Timelion conditional logic for the _greater than_ operator. In this example, the warning threshold is 11.3GB (`11300000000`), and the severe threshold is 11.375GB (`11375000000`). If the threshold values are too high or low for your machine, adjust the values accordingly.
<2> `if()` compares each point to a number. If the condition evaluates to `true`, adjust the styling. If the condition evaluates to `false`, use the default styling.

[role="screenshot"]
image::images/timelion-conditional02.png[]
{nbsp}

[float]
[[conditional-determine-trend]]
==== Determine the trend

To determine the trend, create a new data series:

[source,text]
----------------------------------
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .if(gt,11300000000,
      .es(index=metricbeat-*,
          timefield='@timestamp',
          metric='max:system.memory.actual.used.bytes'),
      null)
      .label('warning')
      .color('#FFCC11'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .if(gt,11375000000,
      .es(index=metricbeat-*,
          timefield='@timestamp',
          metric='max:system.memory.actual.used.bytes'),
      null).
      label('severe')
      .color('red'),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .mvavg(10) <1>
----------------------------------

<1> `mvavg()` calculates the moving average over a specified period of time. In this example, `.mvavg(10)` creates a moving average with a window of 10 data points.

[role="screenshot"]
image::images/timelion-conditional03.png[]
{nbsp}

[float]
[[conditional-format-visualization]]
==== Customize and format the visualization

Customize and format the visualization using functions:

[source,text]
----------------------------------
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .label('max memory')                    <1>
  .title('Memory consumption over time'), <2>
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .if(gt,
      11300000000,
      .es(index=metricbeat-*,
          timefield='@timestamp',
          metric='max:system.memory.actual.used.bytes'),
      null)
    .label('warning')
    .color('#FFCC11')                 <3>
    .lines(width=5),                  <4>
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .if(gt,
      11375000000,
      .es(index=metricbeat-*,
          timefield='@timestamp',
          metric='max:system.memory.actual.used.bytes'),
      null)
    .label('severe')
    .color('red')
    .lines(width=5),
.es(index=metricbeat-*,
    timefield='@timestamp',
    metric='max:system.memory.actual.used.bytes')
  .mvavg(10)
  .label('mvavg')
  .lines(width=2)
  .color(#5E5E5E)
  .legend(columns=4, position=nw)    <5>
----------------------------------

<1> `.label()` adds custom labels to the visualization.
<2> `.title()` adds a title with a meaningful name.
<3> `.color()` changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data.
<4> `.lines()` changes the appearance of the chart lines. In this example, .lines(width=5) sets border width to `5`.
<5> `.legend()` sets the position and style of the legend. For this example, `(columns=4, position=nw)` places the legend in the north west position of the visualization with four columns.

[role="screenshot"]
image::images/timelion-conditional04.png[]
{nbsp}

For additional information on Timelion conditional capabilities, go to https://www.elastic.co/blog/timeseries-if-then-else-with-timelion[I have but one .condition()].