mirror of
https://github.com/elastic/kibana.git
synced 2025-04-23 17:28:26 -04:00
Add saved object docs (#90860)
* iwp * add docs on saved objects * add saved object docs * Update dev_docs/key_concepts/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * Update dev_docs/tutorials/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * Update dev_docs/tutorials/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * Update dev_docs/tutorials/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * Update dev_docs/tutorials/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * Update dev_docs/tutorials/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * Update dev_docs/tutorials/saved_objects.mdx Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> * review updates * remove this line, support being added Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
This commit is contained in:
Stacey Gammon
GitHub
parent
1fdd6ad639
commit
9fca7a9012
3 changed files with 324 additions and 0 deletions
BIN
dev_docs/assets/saved_object_vs_data_indices.png
Normal file
BIN
dev_docs/assets/saved_object_vs_data_indices.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
74
dev_docs/key_concepts/saved_objects.mdx
Normal file
74
dev_docs/key_concepts/saved_objects.mdx
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
---
|
||||
id: kibDevDocsSavedObjectsIntro
|
||||
slug: /kibana-dev-docs/saved-objects-intro
|
||||
title: Saved Objects
|
||||
summary: Saved Objects are a key concept to understand when building a Kibana plugin.
|
||||
date: 2021-02-02
|
||||
tags: ['kibana','dev', 'contributor', 'api docs']
|
||||
---
|
||||
|
||||
"Saved Objects" are developer defined, persisted entities, stored in the Kibana system index (which is also sometimes referred to as the `.kibana` index).
|
||||
The Saved Objects service allows Kibana plugins to use Elasticsearch like a primary database. Think of it as an Object Document Mapper for Elasticsearch.
|
||||
Some examples of Saved Object types are dashboards, lens, canvas workpads, index patterns, cases, ml jobs, and advanced settings. Some Saved Object types are
|
||||
exposed to the user in the [Saved Object management UI](https://www.elastic.co/guide/en/kibana/current/managing-saved-objects.html), but not all.
|
||||
|
||||
Developers create and manage their Saved Objects using the SavedObjectClient, while other data in Elasticsearch should be accessed via the data plugin's search
|
||||
services.
|
||||
|
||||
|
||||
|
||||
|
||||
<DocLink id="kibDevTutorialSavedObject" text="Tutorial: Register a new Saved Object type"/>
|
||||
|
||||
## References
|
||||
|
||||
In order to support import and export, and space-sharing capabilities, Saved Objects need to explicitly list any references they contain to other Saved Objects.
|
||||
The parent should have a reference to it's children, not the other way around. That way when a "parent" is exported (or shared to a space),
|
||||
all the "children" will be automatically included. However, when a "child" is exported, it will not include all "parents".
|
||||
|
||||
<DocLink id="kibDevTutorialSavedObject" section="references" text="Learn how to define Saved Object references"/>
|
||||
|
||||
## Migrations and Backward compatibility
|
||||
|
||||
As your plugin evolves, you may need to change your Saved Object type in a breaking way (for example, changing the type of an attribtue, or removing
|
||||
an attribute). If that happens, you should write a migration to upgrade the Saved Objects that existed prior to the change.
|
||||
|
||||
<DocLink id="kibDevTutorialSavedObject" section="migrations" text="How to write a migration"/>.
|
||||
|
||||
## Security
|
||||
|
||||
Saved Objects can be secured using Kibana's Privileges model, unlike data that comes from data indices, which is secured using Elasticsearch's Privileges model.
|
||||
|
||||
### Space awareness
|
||||
|
||||
Saved Objects are "space aware". They exist in the space they were created in, and any spaces they have been shared with.
|
||||
|
||||
### Feature controls and RBAC
|
||||
|
||||
Feature controls provide another level of isolation and shareability for Saved Objects. Admins can give users and roles read, write or none permissions for each Saved Object type.
|
||||
|
||||
### Object level security (OLS)
|
||||
|
||||
OLS is an oft-requested feature that is not implemented yet. When it is, it will provide users with even more sharing and privacy flexibility. Individual
|
||||
objects can be private to the user, shared with a selection of others, or made public. Much like how sharing Google Docs works.
|
||||
|
||||
## Scalability
|
||||
|
||||
By default all saved object types go into a single index. If you expect your saved object type to have a lot of unique fields, or if you expect there
|
||||
to be many of them, you can have your objects go in a separate index by using the `indexPattern` field. Reporting and task manager are two
|
||||
examples of features that use this capability.
|
||||
|
||||
## Searchability
|
||||
|
||||
Because saved objects are stored in system indices, they cannot be searched like other data can. If you see the phrase “[X] as data” it is
|
||||
referring to this searching limitation. Users will not be able to create custom dashboards using saved object data, like they would for data stored
|
||||
in Elasticsearch data indices.
|
||||
|
||||
## Saved Objects by value
|
||||
|
||||
Sometimes Saved Objects end up persisted inside another Saved Object. We call these Saved Objects “by value”, as opposed to "by
|
||||
reference". If an end user creates a visualization and adds it to a dashboard without saving it to the visualization
|
||||
library, the data ends up nested inside the dashboard Saved Object. This helps keep the visualization library smaller. It also avoids
|
||||
issues with edits propagating - since an entity can only exist in a single place.
|
||||
Note that from the end user stand point, we don’t use these terms “by reference” and “by value”.
|
||||
|
||||
250
dev_docs/tutorials/saved_objects.mdx
Normal file
250
dev_docs/tutorials/saved_objects.mdx
Normal file
|
|
@ -0,0 +1,250 @@
|
|||
---
|
||||
id: kibDevTutorialSavedObject
|
||||
slug: /kibana-dev-docs/tutorial/saved-objects
|
||||
title: Register a new saved object type
|
||||
summary: Learn how to register a new saved object type.
|
||||
date: 2021-02-05
|
||||
tags: ['kibana','onboarding', 'dev', 'architecture', 'tutorials']
|
||||
---
|
||||
|
||||
Saved Object type definitions should be defined in their own `my_plugin/server/saved_objects` directory.
|
||||
|
||||
The folder should contain a file per type, named after the snake_case name of the type, and an index.ts file exporting all the types.
|
||||
|
||||
**src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts**
|
||||
|
||||
```ts
|
||||
import { SavedObjectsType } from 'src/core/server';
|
||||
|
||||
export const dashboardVisualization: SavedObjectsType = {
|
||||
name: 'dashboard_visualization', [1]
|
||||
hidden: false,
|
||||
namespaceType: 'single',
|
||||
mappings: {
|
||||
dynamic: false,
|
||||
properties: {
|
||||
description: {
|
||||
type: 'text',
|
||||
},
|
||||
hits: {
|
||||
type: 'integer',
|
||||
},
|
||||
},
|
||||
},
|
||||
migrations: {
|
||||
'1.0.0': migratedashboardVisualizationToV1,
|
||||
'2.0.0': migratedashboardVisualizationToV2,
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
[1] Since the name of a Saved Object type forms part of the url path for the public Saved Objects HTTP API,
|
||||
these should follow our API URL path convention and always be written as snake case.
|
||||
|
||||
**src/plugins/my_plugin/server/saved_objects/index.ts**
|
||||
|
||||
```ts
|
||||
export { dashboardVisualization } from './dashboard_visualization';
|
||||
export { dashboard } from './dashboard';
|
||||
```
|
||||
|
||||
**src/plugins/my_plugin/server/plugin.ts**
|
||||
|
||||
```ts
|
||||
import { dashboard, dashboardVisualization } from './saved_objects';
|
||||
|
||||
export class MyPlugin implements Plugin {
|
||||
setup({ savedObjects }) {
|
||||
savedObjects.registerType(dashboard);
|
||||
savedObjects.registerType(dashboardVisualization);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Mappings
|
||||
|
||||
Each Saved Object type can define its own Elasticsearch field mappings. Because multiple Saved Object
|
||||
types can share the same index, mappings defined by a type will be nested under a top-level field that matches the type name.
|
||||
|
||||
For example, the mappings defined by the dashboard_visualization Saved Object type:
|
||||
|
||||
**src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts**
|
||||
|
||||
```ts
|
||||
import { SavedObjectsType } from 'src/core/server';
|
||||
|
||||
export const dashboardVisualization: SavedObjectsType = {
|
||||
name: 'dashboard_visualization',
|
||||
...
|
||||
mappings: {
|
||||
properties: {
|
||||
dynamic: false,
|
||||
description: {
|
||||
type: 'text',
|
||||
},
|
||||
hits: {
|
||||
type: 'integer',
|
||||
},
|
||||
},
|
||||
},
|
||||
migrations: { ... },
|
||||
};
|
||||
```
|
||||
|
||||
Will result in the following mappings being applied to the .kibana index:
|
||||
|
||||
```ts
|
||||
{
|
||||
"mappings": {
|
||||
"dynamic": "strict",
|
||||
"properties": {
|
||||
...
|
||||
"dashboard_vizualization": {
|
||||
"dynamic": false,
|
||||
"properties": {
|
||||
"description": {
|
||||
"type": "text",
|
||||
},
|
||||
"hits": {
|
||||
"type": "integer",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Do not use field mappings like you would use data types for the columns of a SQL database. Instead, field mappings are analogous to a
|
||||
SQL index. Only specify field mappings for the fields you wish to search on or query. By specifying `dynamic: false`
|
||||
in any level of your mappings, Elasticsearch will accept and store any other fields even if they are not specified in your mappings.
|
||||
|
||||
Since Elasticsearch has a default limit of 1000 fields per index, plugins should carefully consider the
|
||||
fields they add to the mappings. Similarly, Saved Object types should never use `dynamic: true` as this can cause an arbitrary
|
||||
amount of fields to be added to the .kibana index.
|
||||
|
||||
## References
|
||||
|
||||
Declare <DocLink id="kibDevDocsSavedObjectsIntro" section="References" text="Saved Object references"/> by adding an id, type and name to the
|
||||
`references` array.
|
||||
|
||||
```ts
|
||||
router.get(
|
||||
{ path: '/some-path', validate: false },
|
||||
async (context, req, res) => {
|
||||
const object = await context.core.savedObjects.client.create(
|
||||
'dashboard',
|
||||
{
|
||||
title: 'my dashboard',
|
||||
panels: [
|
||||
{ visualization: 'vis1' }, [1]
|
||||
],
|
||||
indexPattern: 'indexPattern1'
|
||||
},
|
||||
{ references: [
|
||||
{ id: '...', type: 'visualization', name: 'vis1' },
|
||||
{ id: '...', type: 'index_pattern', name: 'indexPattern1' },
|
||||
]
|
||||
}
|
||||
)
|
||||
...
|
||||
}
|
||||
);
|
||||
```
|
||||
[1] Note how `dashboard.panels[0].visualization` stores the name property of the reference (not the id directly) to be able to uniquely
|
||||
identify this reference. This guarantees that the id the reference points to always remains up to date. If a
|
||||
visualization id was directly stored in `dashboard.panels[0].visualization` there is a risk that this id gets updated without
|
||||
updating the reference in the references array.
|
||||
|
||||
## Writing migrations
|
||||
|
||||
Saved Objects support schema changes between Kibana versions, which we call migrations. Migrations are
|
||||
applied when a Kibana installation is upgraded from one version to the next, when exports are imported via
|
||||
the Saved Objects Management UI, or when a new object is created via the HTTP API.
|
||||
|
||||
Each Saved Object type may define migrations for its schema. Migrations are specified by the Kibana version number, receive an input document,
|
||||
and must return the fully migrated document to be persisted to Elasticsearch.
|
||||
|
||||
Let’s say we want to define two migrations: - In version 1.1.0, we want to drop the subtitle field and append it to the title - In version
|
||||
1.4.0, we want to add a new id field to every panel with a newly generated UUID.
|
||||
|
||||
First, the current mappings should always reflect the latest or "target" schema. Next, we should define a migration function for each step in the schema evolution:
|
||||
|
||||
**src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts**
|
||||
|
||||
```ts
|
||||
import { SavedObjectsType, SavedObjectMigrationFn } from 'src/core/server';
|
||||
import uuid from 'uuid';
|
||||
|
||||
interface DashboardVisualizationPre110 {
|
||||
title: string;
|
||||
subtitle: string;
|
||||
panels: Array<{}>;
|
||||
}
|
||||
interface DashboardVisualization110 {
|
||||
title: string;
|
||||
panels: Array<{}>;
|
||||
}
|
||||
|
||||
interface DashboardVisualization140 {
|
||||
title: string;
|
||||
panels: Array<{ id: string }>;
|
||||
}
|
||||
|
||||
const migrateDashboardVisualization110: SavedObjectMigrationFn<
|
||||
DashboardVisualizationPre110, [1]
|
||||
DashboardVisualization110
|
||||
> = (doc) => {
|
||||
const { subtitle, ...attributesWithoutSubtitle } = doc.attributes;
|
||||
return {
|
||||
...doc, [2]
|
||||
attributes: {
|
||||
...attributesWithoutSubtitle,
|
||||
title: `${doc.attributes.title} - ${doc.attributes.subtitle}`,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const migrateDashboardVisualization140: SavedObjectMigrationFn<
|
||||
DashboardVisualization110,
|
||||
DashboardVisualization140
|
||||
> = (doc) => {
|
||||
const outPanels = doc.attributes.panels?.map((panel) => {
|
||||
return { ...panel, id: uuid.v4() };
|
||||
});
|
||||
return {
|
||||
...doc,
|
||||
attributes: {
|
||||
...doc.attributes,
|
||||
panels: outPanels,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
export const dashboardVisualization: SavedObjectsType = {
|
||||
name: 'dashboard_visualization', [1]
|
||||
/** ... */
|
||||
migrations: {
|
||||
// Takes a pre 1.1.0 doc, and converts it to 1.1.0
|
||||
'1.1.0': migrateDashboardVisualization110,
|
||||
|
||||
// Takes a 1.1.0 doc, and converts it to 1.4.0
|
||||
'1.4.0': migrateDashboardVisualization140, [3]
|
||||
},
|
||||
};
|
||||
```
|
||||
[1] It is useful to define an interface for each version of the schema. This allows TypeScript to ensure that you are properly handling the input and output
|
||||
types correctly as the schema evolves.
|
||||
|
||||
[2] Returning a shallow copy is necessary to avoid type errors when using different types for the input and output shape.
|
||||
|
||||
[3] Migrations do not have to be defined for every version. The version number of a migration must always be the earliest Kibana version
|
||||
in which this migration was released. So if you are creating a migration which will
|
||||
be part of the v7.10.0 release, but will also be backported and released as v7.9.3, the migration version should be: 7.9.3.
|
||||
|
||||
Migrations should be written defensively, an exception in a migration function will prevent a Kibana upgrade from succeeding and will cause downtime for our users.
|
||||
Having said that, if a
|
||||
document is encountered that is not in the expected shape, migrations are encouraged to throw an exception to abort the upgrade. In most scenarios, it is better to
|
||||
fail an upgrade than to silently ignore a corrupt document which can cause unexpected behaviour at some future point in time.
|
||||
|
||||
It is critical that you have extensive tests to ensure that migrations behave as expected with all possible input documents. Given how simple it is to test all the branch
|
||||
conditions in a migration function and the high impact of a bug in this code, there’s really no reason not to aim for 100% test code coverage.
|
||||
Loading…
Add table
Add a link
Reference in a new issue