kibana/packages/analytics/client

@kbn/analytics-client

This module implements the Analytics client used for Event-Based Telemetry. The intention of the client is to be usable on both: the UI and the Server sides.

How to use it

It all starts by creating the client with the createAnalytics API:

import { createAnalytics } from '@kbn/analytics-client';

const analytics = createAnalytics({
  // Set to `true` when running in developer mode.
  // It enables development helpers like schema validation and extra debugging features.
  isDev: false,
  // Set to `staging` if you don't want your events to be sent to the production cluster. Useful for CI & QA environments.
  sendTo: 'production',
  // The application's instrumented logger 
  logger,
});

Reporting events

Reporting events is as simple as calling the reportEvent API every time your application needs to track an event:

analytics.reportEvent('my_unique_event_name', myEventProperties);

But first, it requires a setup phase where the application must declare the event and the structure of the eventProperties:

analytics.registerEventType({
  eventType: 'my_unique_event_name',
  schema: {
    my_keyword: {
      type: 'keyword',
      _meta: {
        description: 'Represents the key property...'
      }
    },
    my_number: {
      type: 'long',
      _meta: {
        description: 'Indicates the number of times...',
        optional: true
      }
    },
    my_complex_unknown_meta_object: {
      type: 'pass_through',
      _meta: {
        description: 'Unknown object that contains the key-values...'
      }
    },
    my_array_of_str: {
      type: 'array',
      items: {
        type: 'text',
        _meta: {
          description: 'List of tags...'
        }
      }
    },
    my_object: {
      properties: {
        my_timestamp: {
          type: 'date',
          _meta: {
            description: 'timestamp when the user...'
          }
        }
      }
    },
    my_array_of_objects: {
      type: 'array',
      items: {
        properties: {
          my_bool_prop: {
            type: 'boolean',
            _meta: {
              description: '`true` when...'
            }
          }
        }
      }
    }
  }
});

For more information about how to declare the schemas, refer to the section Schema definition.

Enriching events

Context is important! For that reason, the client internally appends the timestamp in which the event was generated and any additional context provided by the Context Providers. To register a context provider use the registerContextProvider API:

analytics.registerContextProvider({
  name: 'my_context_provider',
  // RxJS Observable that emits every time the context changes. For example: a License changes from `basic` to `trial`.
  context$,
  // Similar to the `reportEvent` API, schema defining the structure of the expected output of the context$ observable.
  schema,
})

Setting the user's opt-in consent

The client cannot send any data until the user provides consent. At the beginning, the client will internally enqueue any incoming events until the consent is either granted or refused.

To set the user's selection use the optIn API:

analytics.optIn({
  global: {
    enabled: true, // The user granted consent
    shippers: {
      shipperA: false, // Shipper A is explicitly disabled for all events
    }
  },
  event_types: {
    my_unique_event_name: {
      enabled: true, // The consent is explictly granted to send this type of event (only if global === true)
      shippers: {
        shipperB: false, // Shipper B is not allowed to report this event.
      }
    },
    my_other_event_name: {
      enabled: false, // The consent is not granted to send this type of event.
    }
  }
})

Explicit flush of the events

If, at any given point (usually testing or during shutdowns) we need to make sure that all the pending events in the queue are sent. The flush API returns a promise that will resolve as soon as all events in the queue are sent.

await analytics.flush()

Shipping events

In order to report the event to an analytics tool, we need to register the shippers our application wants to use. To register a shipper use the API registerShipper:

analytics.registerShipper(ShipperClass, shipperOptions);

There are some prebuilt shippers in this package that can be enabled using the API above. Additionally, each application can register their own custom shippers.

Prebuilt shippers

Refer to the shippers' documentation for more information.

Custom shippers

To use your own shipper, you just need to implement and register it!:

import type { 
  AnalyticsClientInitContext, 
  Event, 
  EventContext, 
  IShipper, 
  TelemetryCounter 
} from '@kbn/analytics-client';

class MyVeryOwnShipper implements IShipper {
  constructor(myOptions: MyOptions, initContext: AnalyticsClientInitContext) {
    // ...
  }

  public reportEvents(events: Event[]): void {
    // Send the events to the analytics platform
  }
  public optIn(isOptedIn: boolean): void {
    // Start/stop any sending mechanisms
  }
  
  public extendContext(newContext: EventContext): void {
    // Call any custom APIs to internally set the context
  }
  
  // Emit any success/failed/dropped activity
  public telemetryCounter$: Observable<TelemetryCounter>;
}

// Register the custom shipper
analytics.registerShipper(MyVeryOwnShipper, myOptions);

Schema definition

Schemas are a framework that allows us to document the structure of the events that our application will report. It is useful to understand the meaning of the events that we report. And, at the same time, it serves as an extra validation step from the developer's point of view.

The syntax of a schema is a simplified ES mapping on steroids: it removes some of the ES mapping complexity, and at the same time, it includes features that are specific to the telemetry collection.

DISCLAIMER: The schema is not a direct mapping to ES indices. The final structure of how the event is stored will depend on many factors like the context providers, shippers and final analytics solution.

Schema Specification: Primitive data types (string, number, boolean)

When declaring primitive values like string or number, the basic schema must contain both: type and _meta.

The type value depends on the type of the content to report in that field. Refer to the table below for the values allowed in the schema type:

Typescript type	Schema type
boolean	boolean
string	keyword
string	text
string	date (for ISO format)
number	date (for ms format)
number	byte
number	short
number	integer
number	long
number	double
number	float

const stringSchema: SchemaValue<string> = {
  type: 'text',
  _meta: {
    description: 'Description of the feature that was broken',
    optional: false,
  },
}

For the _meta, refer to Schema Specification: _meta.

Schema Specification: Objects

Declaring the schema of an object contains 2 main attributes: properties and an optional _meta:

The properties attribute is an object with all the keys that the original object may include:

interface MyObject {
  an_id: string;
  a_description: string;
  a_number?: number;
  a_boolean: boolean;
}

const objectSchema: SchemaObject<MyObject> = {
  properties: {
    an_id: {
      type: 'keyword',
      _meta: {
        description: 'The ID of the element that generated the event',
        optional: false,
      },
    },
    a_description: {
      type: 'text',
      _meta: {
        description: 'The human readable description of the element that generated the event',
        optional: false,
      },
    },
    a_number: {
      type: 'long',
      _meta: {
        description: 'The number of times the element is used',
        optional: true,
      },
    },
    a_boolean: {
      type: 'boolean',
      _meta: {
        description: 'Is the element still active',
        optional: false,
      },
    },
  },
  _meta: {
    description: 'MyObject represents the events generated by elements in the UI when ...',
    optional: false,
  }
}

For the optional _meta, refer to Schema Specification: _meta.

Schema Specification: Arrays

Declaring the schema of an array contains 2 main attributes: items and an optional _meta:

The items attribute is an object declaring the schema of the elements inside the array. At the moment, we only support arrays of one type, so Array<string | number> are not allowed.

type MyArray = string[];

const arraySchema: SchemaArray<MyArray> = {
  items:  {
    type: 'keyword',
    _meta: {
      description: 'Tag attached to the element...',
      optional: false,
    },
  },
  _meta: {
    description: 'List of tags attached to the element...',
    optional: false,
  }
}

For the optional _meta, refer to Schema Specification: _meta.

Schema Specification: Special type pass_through

In case a property in the schema is just used to pass through some unknown content that is declared and validated somewhere else, or that it can dynamically grow and shrink, you may use the type: 'pass_through' option. It behaves like a first-order data type:

type MyUnknownType = unknown;

const passThroughSchema: SchemaValue<MyUnknownType> = {
  type: 'pass_through',
  _meta: {
    description: 'Payload context recevied from the HTTP request...',
    optional: false,
  },
}

For the optional _meta, refer to Schema Specification: _meta.

Schema Specification: _meta

The _meta adds the invaluable information of a description and whether a field is optional in the payload.

It can be attached to any schema definition as seen in the examples above. For high-order types, like arrays or objects, the _meta field is optional. For first-order types, like numbers, strings, booleans or pass_through, the _meta key is mandatory.

The field _meta.optional is not required unless the schema is describing an optional field. In that case, _meta.optional: true is required. However, it's highly encouraged to be explicit about declaring it even when the described field is not optional.

Schema Validation

Apart from documentation, the schema is used to validate the payload during the dev cycle. This adds an extra layer of confidence over the data to be sent.

The validation, however, is disabled in production because users cannot do anything to fix the bug after it is released. Additionally, receiving buggy events can be considered an additional insight into how our users use our products. For example, the buggy event can be caused by a user following an unexpected path in the UI like clicking an "Upload" button when the file has not been selected #125013. In those cases, receiving the incomplete event tells us the user didn't select a file, but they still hit the "Upload" button.

The validation is performed with the io-ts library. In order to do that, the schema is firstly parsed into the io-ts equivalent, and then used to validate the event & context payloads.