kibana/docs/developer/advanced/sharing-saved-objects.asciidoc

[[sharing-saved-objects]]
== Sharing Saved Objects

This guide describes the Sharing Saved Objects effort, and the breaking changes that plugin developers need to be aware of for the planned
8.0 release of {kib}.

[[sharing-saved-objects-overview]]
=== Overview

<<saved-objects-service, Saved objects>> (hereinafter "objects") are used to store all sorts of things in {kib}, from Dashboards to Index
Patterns to Machine Learning Jobs. The effort to make objects shareable can be summarized in a single picture:

image::images/sharing-saved-objects-overview.png["Sharing Saved Objects overview"]

Each plugin can register different object types to be used in {kib}. Historically, objects could be _isolated_ (existing in a single
<<xpack-spaces, space>>) or _global_ (existing in all spaces), there was no in-between. As of the 7.12 release, {kib} now supports two
additional types of objects:

|======================================================================================================
|                     | *Where it exists* | *Object IDs*         | *Registered as:*
| Global              | All spaces        | Globally unique      | `namespaceType: 'agnostic'`
| Isolated            | 1 space           | Unique in each space | `namespaceType: 'single'`
| (NEW) Share-capable | 1 space           | Globally unique      | `namespaceType: 'multiple-isolated'`
| (NEW) Shareable     | 1 or more spaces  | Globally unique      | `namespaceType: 'multiple'`
|======================================================================================================

Ideally, most types of objects in {kib} will eventually be _shareable_; however, we have also introduced
<<sharing-saved-objects-faq-share-capable-vs-shareable,_share-capable_ objects>> as a stepping stone for plugin developers to fully support
this feature.

[[sharing-saved-objects-breaking-changes]]
=== Breaking changes

To implement this feature, we had to make a key change to how objects are serialized into raw {es} documents. As a result,
<<sharing-saved-objects-faq-changing-object-ids,some existing object IDs need to be changed>>, and this will cause some breaking changes to
the way that consumers (plugin developers) interact with objects. We have implemented mitigations so that *these changes will not affect
end-users _if_ consumers implement the required steps below.*

Existing, isolated object types will need to go through a special _conversion process_ to become share-capable upon upgrading {kib} to
version 8.0. Once objects are converted, they can easily be switched to become fully shareable in any future release. This conversion will
change the IDs of any existing objects that are not in the Default space. Changing object IDs itself has several knock-on effects:

* Nonstandard links to other objects can break - _mitigated by <<sharing-saved-objects-step-1>>_
* "Deep link" pages (URLs) to objects can break - _mitigated by <<sharing-saved-objects-step-2>> and <<sharing-saved-objects-step-3>>_
* Encrypted objects may not be able to be decrypted - _mitigated by <<sharing-saved-objects-step-5>>_

*To be perfectly clear: these effects will all be mitigated _if and only if_ you follow the steps below!*

TIP: External plugins can also convert their objects, but <<sharing-saved-objects-faq-external-plugins,they don't have to do so before the
8.0 release>>.

[[sharing-saved-objects-dev-flowchart]]
=== Developer Flowchart

If you're still reading this page, you're probably developing a {kib} plugin that registers an object type, and you want to know what steps
you need to take to prepare for the 8.0 release and mitigate any breaking changes! Depending on how you are using saved objects, you may
need to take up to 5 steps, which are detailed in separate sections below. Refer to this flowchart:

image::images/sharing-saved-objects-dev-flowchart.png["Sharing Saved Objects developer flowchart"]

TIP: There is a proof-of-concept (POC) pull request to demonstrate these changes. It first adds a simple test plugin that allows users to
create and view notes. Then, it goes through the steps of the flowchart to convert the isolated "note" objects to become share-capable. As
you read this guide, you can https://github.com/elastic/kibana/pull/107256[follow along in the POC] to see exactly how to take these steps.

[[sharing-saved-objects-q1]]
=== Question 1

> *Do these objects contain links to other objects?*

If your objects store _any_ links to other objects (with an object type/ID), you need to take specific steps to ensure that these links
continue functioning after the 8.0 upgrade.

[[sharing-saved-objects-step-1]]
=== Step 1

⚠️ This step *must* be completed no later than the 7.16 release. ⚠️

> *Ensure all object links use the root-level `references` field*

If you answered "Yes" to <<sharing-saved-objects-q1>>, you need to make sure that your object links are _only_ stored in the root-level
`references` field. When a given object's ID is changed, this field will be updated accordingly for other objects.

The image below shows two different examples of object links from a "case" object to an "action" object. The top shows the incorrect way to
link to another object, and the bottom shows the correct way.

image::images/sharing-saved-objects-step-1.png["Sharing Saved Objects step 1"]

If your objects _do not_ use the root-level `references` field, you'll need to <<saved-objects-service-writing-migrations,add a migration>>
_before the 8.0 release_ to fix that. Here's a migration function for the example above:

```ts
function migrateCaseToV716(
  doc: SavedObjectUnsanitizedDoc<{ connector: { type: string; id: string } }>
): SavedObjectSanitizedDoc<unknown> {
  const {
    connector: { type: connectorType, id: connectorId, ...otherConnectorAttrs },
  } = doc.attributes;
  const { references = [] } = doc;
  return {
    ...doc,
    attributes: {
      ...doc.attributes,
      connector: otherConnectorAttrs,
    },
    references: [...references, { type: connectorType, id: connectorId, name: 'connector' }],
  };
}

...

// Use this migration function where the "case" object type is registered
migrations: {
  '7.16.0': migrateCaseToV716,
},
```

NOTE: Reminder, don't forget to add unit tests and integration tests!

[[sharing-saved-objects-q2]]
=== Question 2

> *Are there any "deep links" to these objects?*

A deep link is a URL to a page that shows a specific object. End-users may bookmark these URLs or schedule reports with them, so it is
critical to ensure that these URLs continue working. The image below shows an example of a deep link to a Canvas workpad object:

image::images/sharing-saved-objects-q2.png["Sharing Saved Objects deep link example"]

Note that some URLs may contain <<sharing-saved-objects-faq-multiple-deep-link-objects,deep links to multiple objects>>, for example, a
Dashboard _and_ a filter for an Index Pattern.

[[sharing-saved-objects-step-2]]
=== Step 2

⚠️ This step will preferably be completed in the 7.16 release; it *must* be completed no later than the 8.0 release. ⚠️

> *Update your code to use the new SavedObjectsClient `resolve()` method instead of `get()`*

If you answered "Yes" to <<sharing-saved-objects-q2>>, you need to make sure that when you use the SavedObjectsClient to fetch an object
using its ID, you use a different API to do so. The existing `get()` function will only find an object using its current ID. To make sure
your existing deep link URLs don't break, you should use the new `resolve()` function; <<sharing-saved-objects-faq-legacy-url-alias,this
attempts to find an object using its old ID _and_ its current ID>>.

In a nutshell, if your deep link page had something like this before:

```ts
const savedObject = savedObjectsClient.get(objType, objId);
```

You'll need to change it to this:

```ts
const resolveResult = savedObjectsClient.resolve(objType, objId);
const savedObject = resolveResult.saved_object;
```

TIP: See an example of this in https://github.com/elastic/kibana/pull/107256#user-content-example-steps[step 2 of the POC]!

The
https://github.com/elastic/kibana/blob/main/docs/development/core/server/kibana-plugin-core-server.savedobjectsresolveresponse.md[SavedObjectsResolveResponse
interface] has three fields, summarized below:

* `saved_object` - The saved object that was found.
* `outcome` - One of the following values: `'exactMatch' | 'aliasMatch' | 'conflict'`
* `alias_target_id` - This is defined if the outcome is `'aliasMatch'` or `'conflict'`. It means that a legacy URL alias with this ID points
  to an object with a _different_ ID.

The SavedObjectsClient is available both on the server-side and the client-side. You may be fetching the object on the server-side via a
custom HTTP route, or you may be fetching it on the client-side directly. Either way, the `outcome` and `alias_target_id` fields need to be
passed to your client-side code, and you should update your UI accordingly in the next step.

NOTE: You don't need to use `resolve()` everywhere, <<sharing-saved-objects-faq-resolve-instead-of-get,you should only use it for deep
links>>!

[[sharing-saved-objects-step-3]]
=== Step 3

⚠️ This step will preferably be completed in the 7.16 release; it *must* be completed no later than the 8.0 release. ⚠️

> *Update your _client-side code_ to correctly handle the three different `resolve()` outcomes*

The Spaces plugin API exposes React components and functions that you should use to render your UI in a consistent manner for end-users.
Your UI will need to use the Core HTTP service and the Spaces plugin API to do this.

Your page should change <<sharing-saved-objects-faq-resolve-outcomes,according to the outcome>>:

image::images/sharing-saved-objects-step-3.png["Sharing Saved Objects resolve outcomes overview"]

TIP: See an example of this in https://github.com/elastic/kibana/pull/107256#user-content-example-steps[step 3 of the POC]!

1. Update your plugin's `kibana.json` to add a dependency on the Spaces plugin:
+
```ts
...
"optionalPlugins": ["spaces"]
```

2. Update your plugin's `tsconfig.json` to add a dependency to the Space's plugin's type definitions:
+
```ts
...
"references": [
  ...
  { "path": "../spaces/tsconfig.json" },
]
```

3. Update your Plugin class implementation to depend on the Core HTTP service and Spaces plugin API:
+
```ts
interface PluginStartDeps {
  spaces?: SpacesPluginStart;
}

export class MyPlugin implements Plugin<{}, {}, {}, PluginStartDeps> {
  public setup(core: CoreSetup<PluginStartDeps>) {
    core.application.register({
      ...
      async mount(appMountParams: AppMountParameters) {
        const [coreStart, pluginStartDeps] = await core.getStartServices();
        const { http } = coreStart;
        const { spaces: spacesApi } = pluginStartDeps;
        ...
        // pass `http` and `spacesApi` to your app when you render it
      },
    });
    ...
  }
}
```

4. In your deep link page, add a check for the `'aliasMatch'` outcome:
+
```ts
if (spacesApi && resolveResult.outcome === 'aliasMatch') {
  // We found this object by a legacy URL alias from its old ID; redirect the user to the page with its new ID, preserving any URL hash
  const newObjectId = resolveResult.alias_target_id!; // This is always defined if outcome === 'aliasMatch'
  const newPath = `/this/page/${newObjectId}${window.location.hash}`; // Use the *local* path within this app (do not include the "/app/appId" prefix)
  await spacesApi.ui.redirectLegacyUrl(newPath, OBJECT_NOUN);
  return;
}
```
_Note that `OBJECT_NOUN` is optional, it just changes "object" in the toast to whatever you specify -- you may want the toast to say
"dashboard" or "index pattern" instead!_

5. And finally, in your deep link page, add a function that will create a callout in the case of a `'conflict'` outcome:
+
```tsx
const getLegacyUrlConflictCallout = () => {
  // This function returns a callout component *if* we have encountered a "legacy URL conflict" scenario
  if (spacesApi && resolveResult.outcome === 'conflict') {
    // We have resolved to one object, but another object has a legacy URL alias associated with this ID/page. We should display a
    // callout with a warning for the user, and provide a way for them to navigate to the other object.
    const currentObjectId = savedObject.id;
    const otherObjectId = resolveResult.alias_target_id!; // This is always defined if outcome === 'conflict'
    const otherObjectPath = `/this/page/${otherObjectId}${window.location.hash}`; // Use the *local* path within this app (do not include the "/app/appId" prefix)
    return (
      <>
        {spacesApi.ui.components.getLegacyUrlConflict({
          objectNoun: OBJECT_NOUN,
          currentObjectId,
          otherObjectId,
          otherObjectPath,
        })}
        <EuiSpacer />
      </>
    );
  }
  return null;
};
...
return (
  <EuiPage>
    <EuiPageBody>
      <EuiPageContent>
        {/* If we have a legacy URL conflict callout to display, show it at the top of the page */}
        {getLegacyUrlConflictCallout()}
        <EuiPageContentHeader>
...
);
```

6. https://github.com/elastic/kibana/pull/107099#issuecomment-891147792[Generate staging data and test your page's behavior with the
different outcomes.]

NOTE: Reminder, don't forget to add unit tests and functional tests!

[[sharing-saved-objects-step-4]]
=== Step 4

⚠️ This step *must* be completed in the 8.0 release (no earlier and no later). ⚠️

> *Update your _server-side code_ to convert these objects to become "share-capable"*

After <<sharing-saved-objects-step-3>> is complete, you can add the code to convert your objects.

WARNING: The previous steps can be backported to the 7.x branch, but this step -- the conversion itself -- can only take place in 8.0!
You should use a separate pull request for this.

When you register your object, you need to change the `namespaceType` and also add a `convertToMultiNamespaceTypeVersion` field. This
special field will trigger the actual conversion that will take place during the Core migration upgrade process when a user installs the
Kibana 8.0 release:

image::images/sharing-saved-objects-step-4.png["Sharing Saved Objects conversion code"]

TIP: See an example of this in https://github.com/elastic/kibana/pull/107256#user-content-example-steps[step 4 of the POC]!

NOTE: Reminder, don't forget to add integration tests!

[[sharing-saved-objects-q3]]
=== Question 3

> *Are these objects encrypted?*

Saved objects can optionally be <<xpack-security-secure-saved-objects,encrypted>> by using the Encrypted Saved Objects plugin. Very few
object types are encrypted, so most plugin developers will not be affected.

[[sharing-saved-objects-step-5]]
=== Step 5

⚠️ This step *must* be completed in the 8.0 release (no earlier and no later). ⚠️

> *Update your _server-side code_ to add an Encrypted Saved Object (ESO) migration for these objects*

If you answered "Yes" to <<sharing-saved-objects-q3>>, you need to take additional steps to make sure that your objects can still be
decrypted after the conversion process. Encrypted saved objects use some fields as part of "additionally authenticated data" (AAD) to defend
against different types of cryptographic attacks. The object ID is part of this AAD, and so it follows that the after the object's ID is
changed, the object will not be able to be decrypted with the standard process.

To mitigate this, you need to add a "no-op" ESO migration that will be applied immediately after the object is converted during the 8.0
upgrade process. This will decrypt the object using its old ID and then re-encrypt it using its new ID:

image::images/sharing-saved-objects-step-5.png["Sharing Saved Objects ESO migration"]

NOTE: Reminder, don't forget to add unit tests and integration tests!

[[sharing-saved-objects-step-6]]
=== Step 6

> *Update your code to make your objects shareable*

_This is not required for the 8.0 release; this additional information will be added in the near future!_

[[sharing-saved-objects-faq]]
=== Frequently asked questions (FAQ)

[[sharing-saved-objects-faq-share-capable-vs-shareable]]
==== 1. Why are there both "share-capable" and "shareable" object types?

We implemented the share-capable object type as an intermediate step for consumers who currently have isolated objects, but are not yet
ready to support fully shareable objects. This is primarily because we want to make sure all object types are converted at the same time in
the 8.0 release to minimize confusion and disruption for the end-user experience.

We realize that the conversion process and all that it entails can be a not-insignificant amount of work for some Kibana teams to prepare
for by the 8.0 release. As long as an object is made share-capable, that ensures that its ID will be globally unique, so it will be trivial
to make that object shareable later on when the time is right.

A developer can easily flip a switch to make a share-capable object into a shareable one, since these are both serialized the same way.
However, we envision that each consumer will need to enact their own plan and make additional UI changes when making an object shareable.
For example, some users may not have access to the Saved Objects Management page, but we still want those users to be able to see what
space(s) their objects exist in and share them to other spaces. Each application should add the appropriate UI controls to handle this.


[[sharing-saved-objects-faq-changing-object-ids]]
==== 2. Why do object IDs need to be changed?

This is because of how isolated objects are serialized to raw Elasticsearch documents. Each raw document ID today contains its space ID
(_namespace_) as a prefix. When objects are copied or imported to other spaces, they keep the same object ID, they just have a different
prefix when they are serialized to Elasticsearch. This has resulted in a situation where many Kibana installations have saved objects in
different spaces with the same object ID:

image::images/sharing-saved-objects-faq-changing-object-ids-1.png["Sharing Saved Objects object ID diagram (before conversion)"]

Once an object is converted, we need to remove this prefix. Because of limitations with our migration process, we cannot actively check if
this would result in a conflict. Therefore, we decided to pre-emptively regenerate the object ID for every object in a non-Default space to
ensure that every object ID becomes globally unique:

image::images/sharing-saved-objects-faq-changing-object-ids-2.png["Sharing Saved Objects object ID diagram (after conversion)"]

[[sharing-saved-objects-faq-multiple-deep-link-objects]]
==== 3. What if one page has deep links to multiple objects?

As mentioned in <<sharing-saved-objects-q2>>, some URLs may contain multiple object IDs, effectively deep linking to multiple objects.
These should be handled on a case-by-case basis at the plugin owner's discretion. A good rule of thumb is:

* The "primary" object on the page should always handle the three `resolve()` outcomes as described in <<sharing-saved-objects-step-3>>.
* Any "secondary" objects on the page may handle the outcomes differently. If the secondary object ID is not important (for example, it just
  functions as a page anchor), it may make more sense to ignore the different outcomes. If the secondary object _is_ important but it is not
  directly represented in the UI, it may make more sense to throw a descriptive error when a `'conflict'` outcome is encountered.
  - Embeddables should use `spacesApi.ui.components.getEmbeddableLegacyUrlConflict` to render conflict errors:
+
image::images/sharing-saved-objects-faq-multiple-deep-link-objects-1.png["Sharing Saved Objects embeddable legacy URL conflict"]
Viewing details shows the user how to disable the alias and fix the problem using the
<<spaces-api-disable-legacy-url-aliases,_disable_legacy_url_aliases API>>:
+
image::images/sharing-saved-objects-faq-multiple-deep-link-objects-2.png["Sharing Saved Objects embeddable legacy URL conflict (showing details)"]
  - If the secondary object is resolved by an external service (such as the index pattern service), the service should simply make the full
  outcome available to consumers.

Ideally, if a secondary object on a deep link page resolves to an `'aliasMatch'` outcome, the consumer should redirect the user to a URL
with the new ID and display a toast message. The reason for this is that we don't want users relying on legacy URL aliases more often than
necessary. However, such handling of secondary objects is not considered critical for the 8.0 release.

[[sharing-saved-objects-faq-legacy-url-alias]]
==== 4. What is a "legacy URL alias"?

As depicted above, when an object is converted to become share-capable, if it exists in a non-Default space, its ID gets changed. To
preserve its old ID, we also create a special object called a <<legacy-url-aliases,_legacy URL alias_>> ("alias" for short); this alias
retains the target object's old ID (_sourceId_), and it contains a pointer to the target object's new ID (_targetId_).

Aliases are meant to be mostly invisible to end-users by design. There is no UI to manage them directly. Our vision is that aliases will be
used as a stop-gap to help us through the 8.0 upgrade process, but we will nudge users away from relying on aliases so we can eventually
deprecate and remove them.

[[sharing-saved-objects-faq-resolve-outcomes]]
==== 5. Why are there three different resolve outcomes?

The `resolve()` function checks both if an object with the given ID exists, _and_ if an object has an alias with the given ID.

1. If only the former is true, the outcome is an `'exactMatch'` -- we found the exact object we were looking for.
2. If only the latter is true, the outcome is an `'aliasMatch'` -- we found an alias with this ID, that pointed us to an object with a
different ID.
3. Finally, if _both conditions_ are true, the outcome is a `'conflict'` -- we found two objects using this ID. Instead of returning an
error in this situation, in the interest of usability, we decided to return the _most correct match_, which is the exact match. By informing
the consumer that this is a conflict, the consumer can render an appropriate UI to the end-user explaining that this might not be the object
they are actually looking for.

*Outcome 1*

When you resolve an object with its current ID, the outcome is an `'exactMatch'`:

image::images/sharing-saved-objects-faq-resolve-outcomes-1.png["Sharing Saved Objects resolve outcome 1 (exactMatch)"]

This can happen in the Default space _and_ in non-Default spaces.

*Outcome 2*

When you resolve an object with its old ID (the ID of its alias), the outcome is an `'aliasMatch'`:

image::images/sharing-saved-objects-faq-resolve-outcomes-2.png["Sharing Saved Objects resolve outcome 2 (aliasMatch)"]

This outcome can only happen in non-Default spaces.

*Outcome 3*

The third outcome is an edge case that is a combination of the others. If you resolve an object ID and two objects are found -- one as an
exact match, the other as an alias match -- the outcome is a `'conflict'`:

image::images/sharing-saved-objects-faq-resolve-outcomes-3.png["Sharing Saved Objects resolve outcome 3 (conflict)"]

We actually have controls in place to prevent this scenario from happening when you share, import, or copy
objects. However, this scenario _could_ still happen in a few different situations, if objects are created a certain way or if a user
tampers with an object's raw ES document. Since we can't 100% rule out this scenario, we must handle it gracefully, but we do expect this
will be a rare occurrence.

It is important to note that when a `'conflict'` occurs, the object that is returned is the "most correct" match -- the one with the ID that
exactly matches.

[[sharing-saved-objects-faq-resolve-instead-of-get]]
==== 6. Should I always use resolve instead of get?

Reading through this guide, you may think it is safer or better to use `resolve()` everywhere instead of `get()`. Actually, we made an
explicit design decision to add a separate `resolve()` function because we want to limit the affects of and reliance upon legacy URL
aliases. To that end, we collect anonymous usage data based on how many times `resolve()` is used and the different outcomes are
encountered. That usage data is less useful is `resolve()` is used more often than necessary.

Ultimately, `resolve()` should _only_ be used for data flows that involve a user-controlled deep link to an object. There is no reason to
change any other data flows to use `resolve()`.

[[sharing-saved-objects-faq-external-plugins]]
==== 7. What about external plugins?

External plugins (those not shipped with {kib}) can use this guide to convert any isolated objects to become share-capable or fully
shareable! If you are an external plugin developer, the steps are the same, but you don't need to worry about getting anything done before a
specific release. The only thing you need to know is that your plugin cannot convert your objects until the 8.0 release.

==== 8. How will users be impacted?

Refer to <<saved-object-ids,Saved Object IDs>> documentation for more details how users should expect to be impacted.