This PR introduces an Inference plugin.
## Goals
- Provide a single place for all interactions with large language models
and other generative AI adjacent tasks.
- Abstract away differences between different LLM providers like OpenAI,
Bedrock and Gemini
- Host commonly used LLM-based tasks like generating ES|QL from natural
language and knowledge base recall.
- Allow us to move gradually to the _inference endpoint without
disrupting engineers.
## Architecture and examples
## Terminology
The following concepts are referenced throughout this POC:
- **chat completion**: the process in which the LLM generates the next
message in the conversation. This is sometimes referred to as inference,
text completion, text generation or content generation.
- **tasks**: higher level tasks that, based on its input, use the LLM in
conjunction with other services like Elasticsearch to achieve a result.
The example in this POC is natural language to ES|QL.
- **tools**: a set of tools that the LLM can choose to use when
generating the next message. In essence, it allows the consumer of the
API to define a schema for structured output instead of plain text, and
having the LLM select the most appropriate one.
- **tool call**: when the LLM has chosen a tool (schema) to use for its
output, and returns a document that matches the schema, this is referred
to as a tool call.
## Usage examples
```ts
class MyPlugin {
setup(coreSetup, pluginsSetup) {
const router = coreSetup.http.createRouter();
router.post(
{
path: '/internal/my_plugin/do_something',
validate: {
body: schema.object({
connectorId: schema.string(),
}),
},
},
async (context, request, response) => {
const [coreStart, pluginsStart] = await coreSetup.getStartServices();
const inferenceClient = pluginsSetup.inference.getClient({ request });
const chatComplete$ = inferenceClient.chatComplete({
connectorId: request.body.connectorId,
system: `Here is my system message`,
messages: [
{
role: MessageRole.User,
content: 'Do something',
},
],
});
const message = await lastValueFrom(
chatComplete$.pipe(withoutTokenCountEvents(), withoutChunkEvents())
);
return response.ok({
body: {
message,
},
});
}
);
}
}
```
## Implementation
The bulk of the work here is implementing a `chatComplete` API. Here's
what it does:
- Formats the request for the specific LLM that is being called (all
have different API specifications).
- Executes the specified connector with the formatted request.
- Creates and returns an Observable, and starts reading from the stream.
- Every event in the stream is normalized to a format that is close to
(but not exactly the same) as OpenAI's format, and emitted as a value
from the Observable.
- When the stream ends, the individual events (chunks) are concatenated
into a single message.
- If the LLM has called any tools, the tool call is validated according
to its schema.
- After emitting the message, the Observable completes
There's also a thin wrapper around this API, which is called the
`output` API. It simplifies a few things:
- It doesn't require a conversation (list of messages), a simple `input`
string suffices.
- You can define a schema for the output of the LLM.
- It drops the token count events that are emitted
- It simplifies the event format (update & complete)
### Observable event streams
These APIs, both on the client and the server, return Observables that
emit events. When converting the Observable into a stream, the following
things happen:
- Errors are caught and serialized as events sent over the stream (after
an error, the stream ends).
- The response stream outputs data as [server-sent
events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events)
- The client that reads the stream, parses the event source as an
Observable, and if it encounters a serialized error, it deserializes it
and throws an error in the Observable.
### Errors
All known errors are instances, and not extensions, from the
`InferenceTaskError` base class, which has a `code`, a `message`, and
`meta` information about the error. This allows us to serialize and
deserialize errors over the wire without a complicated factory pattern.
### Tools
Tools are defined as a record, with a `description` and optionally a
`schema`. The reason why it's a record is because of type-safety. This
allows us to have fully typed tool calls (e.g. when the name of the tool
being called is `x`, its arguments are typed as the schema of `x`).
## Notes for reviewers
- I've only added one reference implementation for a connector adapter,
which is OpenAI. Adding more would create noise in the PR, but I can add
them as well. Bedrock would need simulated function calling, which I
would also expect to be handled by this plugin.
- Similarly, the natural language to ES|QL task just creates dummy
steps, as moving the entire implementation would mean 1000s of
additional LOC due to it needing the documentation, for instance.
- Observables over promises/iterators: Observables are a well-defined
and widely-adopted solution for async programming. Promises are not
suitable for streamed/chunked responses because there are no
intermediate values. Async iterators are not widely adopted for Kibana
engineers.
- JSON Schema over Zod: I've tried using Zod, because I like its
ergonomics over plain JSON Schema, but we need to convert it to JSON
Schema at some point, which is a lossy conversion, creating a risk of
using features that we cannot convert to JSON Schema. Additionally,
tools for converting Zod to and [from JSON Schema are not always
suitable
](https://github.com/StefanTerdell/json-schema-to-zod#use-at-runtime).
I've implemented my own JSON Schema to type definition, as
[json-schema-to-ts](https://github.com/ThomasAribart/json-schema-to-ts)
is very slow.
- There's no option for raw input or output. There could be, but it
would defeat the purpose of the normalization that the `chatComplete`
API handles. At that point it might be better to use the connector
directly.
- That also means that for LangChain, something would be needed to
convert the Observable into an async iterator that returns
OpenAI-compatible output. This is doable, although it would be nice if
we could just use the output from the OpenAI API in that case.
- I have not made room for any vendor-specific parameters in the
`chatComplete` API. We might need it, but hopefully not.
- I think type safety is critical here, so there is some TypeScript
voodoo in some places to make that happen.
- `system` is not a message in the conversation, but a separate
property. Given the semantics of a system message (there can only be
one, and only at the beginning of the conversation), I think it's easier
to make it a top-level property than a message type.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## 📓 Summary
Closes#188443
Adding a static source repository for [metadata
fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html#_indexing_metadata_fields)
in the resolution chain, so that it's now possible to retrieve metadata
info for them too.
**GET /internal/fields_metadata?fieldNames=_index,_source**
```json
{
"fields": {
"_index": {
"dashed_name": "index",
"description": "The index to which the document belongs. This metadata field specifies the exact index name in which the document is stored.",
"example": "index_1",
"flat_name": "_index",
"name": "_index",
"short": "The index to which the document belongs.",
"type": "keyword",
"documentation_url": "https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index-field.html",
"source": "metadata",
"normalize": []
},
"_source": {
"dashed_name": "source",
"description": "The original JSON representing the body of the document. This field contains all the source data that was provided at the time of indexing.",
"example": "{\"user\": \"John Doe\", \"message\": \"Hello\"}",
"flat_name": "_source",
"name": "_source",
"short": "The original JSON representing the body of the document.",
"documentation_url": "https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html",
"source": "metadata",
"normalize": [],
"type": "unknown"
}
}
}
```
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Renames `@kbn/text-based-languages` plugin to `@kbn/esql` plugin. This
has been discussed internally, the rationale is that now there will be
only one language: ES|QL; and we may use this plugin for ES|QL related
HTTP routes.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Renames the experimental asset_manager plugin (never
documented/officially released) into entity_manager. I've used `node
scripts/lint_ts_projects --fix` and `node scripts/lint_packages.js
--fix` to help with the procedure and also renamed manually the
asset_manager references left.
The change also removes the deprecated asset_manager code, including the
`assetManager.alphaEnabled` plugin configuration. This means
entityManager plugin will be enabled by default.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Introducing the `search_homepage` plugin along with integration into
`enterprise_search` and `serverless_search` behind a feature flag. This
will allow implementing the feature gated behind the feature flag.
To test these changes you can enable the feature flag with the Kibana
Dev Console using the following command:
```
POST kbn:/internal/kibana/settings/searchHomepage:homepageEnabled
{"value": true}
```
You can then disable the feature flag with the following command:
```
DELETE kbn:/internal/kibana/settings/searchHomepage:homepageEnabled
```
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This is a PR to add a new backend plugin (frontend will be done in
separate [PR](https://github.com/elastic/kibana/pull/184546)).
The purpose of the plugin is to provide a set of API routes that is used
to perform a variety of GenAI workflows to generate new integrations
based on provided inputs.
It reuses the existing GenAI connectors for its LLM communication, and
provides a set of API's to create ECS mapping, Categorization, Related
Fields and an API to generate the actual integration package zip, which
is forwarded to the UI component.
### Planned follow-up changes:
As the PR is getting way too large, some planned changes would be added
in much smaller follow-ups. This includes mostly more improved try/catch
for certain routes, adding debug/error log entries where relevant,
especially for the API endpoints themself, some more unit and end2end
tests.
- OpenAPI spec for the API will be handled in a separate PR
- All the missing unit tests will be added as a followup PR
### Testing
The `integration_assistant` plugin will be disabled by default while
it's being implemented so we can iterate and merge partial PRs without
interfering with the releases. This config will work as our feature
flag:
6aefd4ff7b/x-pack/plugins/integration_assistant/server/config.ts (L11-L13)
To test it add this to your _kibana.dev.yml_:
```
xpack.integration_assistant.enabled: true
```
### Checklist
Delete any items that are not applicable to this PR.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Bharat Pasupula <saibharatchandra.pasupula@elastic.co>
Co-authored-by: Bharat Pasupula <123897612+bhapas@users.noreply.github.com>
## Description
In this PR, we implemented a view for managing inference endpoints. The
changes include the following items for both **Serverless** and
**Stack**.
- A blank page will be displayed if no inference endpoints are
available.
- A page displaying a list of inference endpoints. The user can view
various details about each endpoint, such as the endpoint itself, the
provider, and the type. The table supports pagination and sorting.
- Users can add a new inference endpoint using Elasticsearch models and
third-party APIs, including Hugging Face, Cohere, and OpenAI.
To keep the changes in this PR manageable, the following items are **out
of scope** but will be added in subsequent PRs
- Option to delete an inference endpoint
- Filtering and Search bar
- Information about allocations, thread.
- Icons for **Provider**
- Deployment status of underlying trained models
## Empty page in Stack Management
e2064ee8-3623-457f-8a04-19603e97e815
## Page with all inference endpoints in Stack Management
89bec450-1569-4425-b013-5058b577b95a
## Inference Endpoints Management in Serverless
bd8b6b71-0e09-49f4-aa9a-19338a1da225
---------
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
The PR changes the app's title from "Datasets" to "Data Set Quality".
Note that the changed title "Data Set Quality" will also be used as a
side nav menu item under Stack Management (in followups). The PR also
adds an app description as suggested in the parent issue.
The link leads to Doc's ECS page: [Data Stream
Fields](https://www.elastic.co/guide/en/ecs/current/ecs-data_stream.html).
The PR also changes the breadcrumb and quick search entry to use "Data
Set Quality" instead of "Data quality" or "Logs data quality".
As per the [writing style
guidelines](https://brand.elastic.co/302f66895/p/194a3b-writing-style-guide/t/889c93),
any instance of "Dataest" is changed to either "Data Set" or "Data set".
Create the Investigate plugin (naming TBD). Part of
https://github.com/elastic/kibana/pull/183293, splitting up the work in
several PRs.
The investigate plugin is mostly a registry to allow plugins to register
their widgets without creating dependency issues.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Closes https://github.com/elastic/observability-dev/issues/3331
Given the needs described in the linked issue about having a centralized
and async way to consume field metadata across Kibana, this work focuses
on providing server/client services to consume field metadata on demand
from static ECS definition and integration manifests, with the chance to
extend further the possible resolution sources.
3b2d9027-5c29-4081-ab17-1b43618c62a7
## 💡 Reviewers hints
This PR got quite long as it involves and touches different parts of the
codebase, so I'll break down the interesting parts for an easier review.
More details, code examples and mechanics description can be found in
the README file for the plugin.
### `@kbn/fields-metadata-plugin`
To avoid bundling and consuming the whole ECS static definition
client-side, a new plugin `@kbn/fields-metadata-plugin` is created to
expose the server/client services which enable retrieving only the
fields needed on a use-case basis.
### FieldsMetadataService server side
A `FieldsMetadataService` is instantiated on the plugin setup/start
server lifecycle, exposing a client to consume the fields and setup
tools for registering external dependencies.
The start contract exposes a `FieldsMetadataClient` instance. With this,
any application in Kibana can query for some fields using the available
methods, currently:
- `FieldsMetadataClient.prototype.getByName()`: retrieves a single
`FieldMetadata` instance.
- `FieldsMetadataClient.prototype.find()`: retrieves a record of
matching `FieldMetadata` instances.
`FieldsMetadataClient` is instantiated with the source repositories
dependencies. They act as encapsulated sources which are responsible for
fetching fields from their related source. Currently, there are 2 field
repository sources used in the resolution step, but we can use this
concept to extend the resolution step in future with more sources (LLM,
OTel, ...).
The currently used sources are:
- `EcsFieldsRepository`: allows fetching static ECS field metadata.
- `IntegrationFieldsRepository`: allows fetching fields from an
integration package from EPR, where the fields metadata are stored. To
correctly consume these fields, the `fleet` plugin must be enabled,
otherwise, the service won't be able to access the registered fields
extractor implemented with the fleet services.
As this service performs a more expensive retrieval process than the
`EcsFieldsRepository` constant complexity access, a caching layer is
applied to the retrieved results from the external source to minimize
latency.
### Fields metadata API
To expose this service to the client, a first API endpoint is created to
find field metadata and filter the results to minimize the served
payload.
- `GET /internal/fields_metadata/find` supports some initial query
parameters to narrow the fields' search.
### FieldsMetadataService client side
As we have a server-side `FieldsMetadataService`, we need a client
counterpart to consume the exposed API safely and go through the
validation steps.
The client `FieldsMetadataService` works similarly to the server-side
one, exposing a client which is returned by the public start contract of
the plugin, allowing any other to directly use fields metadata
client-side.
This client would work well with existing state management solutions, as
it's not decoupled from any library.
### useFieldsMetadata
For simpler use cases where we need a quick and easy way to consume
fields metadata client-side, the plugin start contract also exposes a
`useFieldsMetadata` react custom hook, which is pre-created accessing
the FieldsMetadataService client described above. It is important to
retrieve the hook from the start contract of this plugin, as it already
gets all the required dependencies injected minimizing the effort on the
consumer side.
The `UnifiedDocViewer` plugin changes exemplify how we can use this hook
to access and filter fields' metadata quickly.
### `registerIntegrationFieldsExtractor` (@elastic/fleet)
Getting fields from an integration dataset is more complex than
accessing a static dictionary of ECS fields, and to achieve that we need
access to the PackageService implemented by the fleet team.
To get access to the package, maintain a proper separation of concerns
and avoid a direct dependency on the fleet plugin, some actions were
taken:
- the `PackageService.prototype.getPackageFieldsMetadata()` method is
implemented to keep the knowledge about retrieving package details on
this service instead of mixing it on parallel services.
- a fleet `registerIntegrationFieldsExtractor` service is created and
used during the fleet plugin setup to register a callback that accesses
the service as an internal user and retrieves the fields by the given
parameters.
- the fields metadata plugin returns a
`registerIntegrationFieldsExtractor` function from its server setup so
that we can use it to register the above-mentioned callback that
retrieves fields from an integration.
This inverts the dependency between `fields_metadata` and `fleet`
plugins so that the `fields_metadata` plugin keeps zero dependencies on
external apps.
## Adoption
We currently have places where the `@elastic/ecs` package is directly
accessed and where we might be able to refactor the codebase to consume
this service.
**[EcsFlat usages in
Kibana](https://github.com/search?q=repo%3Aelastic%2Fkibana%20EcsFlat&type=code)**
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Relates to https://github.com/elastic/kibana/issues/183406.
## 📝 Summary
This PR creates a new plugin `data_quality` in order to register dataset
quality as a Stack management page under data section. For now there is
no reference to this new page in the sideNav in stateful or serverless.
In order to navigate to this new page you can use the url
`/app/management/data/data_quality`
Changes included in this PR:
- New plugin created
- Plugin registered in stack management, data section
- Dataset quality plugin is instantiated and the state is in sync with
URL
- Removed references to dataset quality in Logs explorer
## 🎥 Demo
501c9c47-4a1b-4f91-9be6-d022a821e88e
## 🙅🏼 Missing
- Dataset quality locator
- There are still references to logs explorer (table and flyout) that
will be handled in a follow up PR.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Paired with @ThomThomson to expand Embeddable documentation with
"Guiding principles" and "Best practices"
PR also moves overview to src/plugins/embeddables/README.md. Then, this
markdown is displayed in the embeddable example application as well.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Devon Thomson <devon.thomson@elastic.co>
Having the same thing in multiple places is confusing and hard to
maintain. Embeddable documentation is exposed via developer examples.
This PR removes embeddable documentation in README and dev-docs and
points those locations to the single location for embeddable
documenation.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Closes#181528Closes#181976
**N.B.** This work was initially reviewed on a separate PR at
https://github.com/tonyghiani/kibana/pull/1.
This implementation aims to have a stateful layer that allows the
management of dependencies between Discover and other plugins, reducing
the need for a direct dependency.
Although the initial thought was to have a plugin to register features
for Discover, there might be other cases in future where we need to
prevent cyclic dependencies.
With this in mind, I created the plugin as a more generic solution to
hold stateful logic as a communication layer for Discover <-> Plugins.
## Discover Shared
Based on some recurring naming in the Kibana codebase, `discover_shared`
felt like the right place for owning these dependencies and exposing
Discover functionalities to the external world.
It is initially pretty simple and only exposes a registry for the
Discover features, but might be a good place to implement other upcoming
concepts related to Discover.
Also, this plugin should ideally never depend on other solution plugins
and keep its dependencies to a bare minimum of packages and core/data
services.
```mermaid
flowchart TD
A(Discover) -- Get --> E[DiscoverShared]
B(Logs Explorer) -- Set --> E[DiscoverShared]
C(Security) -- Set --> E[DiscoverShared]
D(Any app) -- Set --> E[DiscoverShared]
```
## DiscoverFeaturesService
This service initializes and exposes a strictly typed registry to allow
consumer apps to register additional features and Discover and retrieve
them.
The **README** file explains a real use case of when we'd need to use it
and how to do that step-by-step.
Although it introduces a more nested folder structure, I decided to
implement the service as a client-service and expose it through the
plugin lifecycle methods to provide the necessary flexibility we might
need:
- We don't know yet if any of the features we register will be done on
the setup/start steps, so having the registry available in both places
opens the door to any case we face.
- The service is client-only on purpose. My opinion is that if we ever
need to register features such as server services or anything else, it
should be scoped to a similar service dedicated for the server lifecycle
and its environment.
It should never be possible to register the ObsAIAssistant
presentational component from the server, as it should not be permitted
to register a server service in the client registry.
A server DiscoverFeaturesService is not required yet for any feature, so
I left it out to avoid overcomplicating the implementation.
## FeaturesRegistry
To have a strictly typed utility that suggests the available features on
a registry and adheres to a base contract, the registry exposed on the
DiscoverFeaturesService is an instance of the `FeaturesRegistry` class,
which implements the registration/retrieval logic such that:
- If a feature is already registered, is not possible to override it and
an error is thrown to notify the user of the collision.
- In case we need to react to registry changes, is possible to subscribe
to the registry or obtain it as an observable for more complex
scenarios.
The FeaturesRegistry already takes care of the required logic for the
registry, so that `DiscoverFeaturesService`is left with the
responsibility of instantiating/exposing an instance and provide the set
of allowed features.
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
Create an empty assets data access plugin so we can start registering
the services(APIs) that will be used to fetch assets type documents.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Introduced the search_notebooks plugin which has a set of default python
notebooks that can be served to the frontend via API endpoints.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR moves the AI Assistant Management plugin into x-pack to
co-locate it with the other assistant plugins and to make it possible to
statically import from the other assistant plugins. This is not
currently possible because the Management plugin is in OSS and the other
plugins are in xpack.
Fixes https://github.com/elastic/kibana/issues/176420
## 🍒 Summary
This PR copies the SLO code that was inside the Observability app into
its own app under `observability-solution/slo` folder.
4f6b8dfb-9612-4d30-ad50-4ee5c55a9c32
## ✔️ Acceptance criteria
- URL of new app: `app/slos`
- Design and functionality are not changed.
- Git history has been retained for all files in
`x-pack/plugins/observability_solution/slo`.
- SLO should appear on server less
- SLO code inside `observability_solution/observability` code has been
removed. A new clean up round might be needed though for possible
leftovers.
- Burn rate rule is registered within the new slo app
- SLO embeddables are moved inside the new slo app
- overview
- alerts embeddable
- error budget burn down
- Alerts table configuration registration for slo details page and
alerts table embeddable is still done in the observability app. Response
Ops team is working on removing the need to register the alert table
anyway
- Slo app is wrapped into `ApplicationUsageTrackingProvider` which will
send slo `Application usage` information tracked by the `slo` appId
- Redirect old `app/observability/slos` route to `app/slos`
- Rename old `xpack.observability.slo` keys to `xpack.slo` in the
translation files
## 🌮 How to test
Design and functionality didn't change, so simply navigate to existing
slo pages and try to break it
- Slo list page
- group by
- unified search
- toggle buttons
- actions
- Slo creation
- try group by as well
- Slo detail page
- Actions on top
- navigate to overview and alerts tabs
- Create SLO flyout in Logs Explorer
- Create burn rate rules and verify they appear on rules page
- Verify SLO alerts appear on Alerts page and slo details page
- Embeddables
- Through the dashboard app
- Using the attach to dashboard action on the slo card item on slo list
page and the error budget burn down chart on the slo detail page
- SLOs only for platinum users
- Permissions
- Spaces
## TODO
- [x] Move slo stuff from observability folder to new slo plugin
- [x] Remove old slo stuff from observability folder
- [x] Update references
- [x] Fix typescript and eslint errors
- [x] Paths
- [x] Locators
- [x] Burn rate rule registration
- [x] Embeddable Alerts table configuration registration
- [x] Embeddables
- [x] Translations
- [x] Verify plugin.ts files contain all registration logic
- [x] public
- [x] server
- [x] Final cleanup for observability folder
- [x] Run tests
- [x] Application Usage (Telemetry)
- [x] Permissions
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: shahzad31 <shahzad31comp@gmail.com>
Co-authored-by: Coen Warmer <coen.warmer@gmail.com>
Splits up the Observability AI Assistant plugin so it can be used
outside of the context of the Observability apps. Additionally, the
following changes were made:
- Add the AI Assistant button to the top nav, instead of the header
menu. This prevents unmounts and remounts (and makes it much easier to
use everywhere).
- Contextual messages now use a function request/response to inject the
data of the insight. This allows us to remove `startedFrom`.
- ML is now an runtime dependency only (via `core.plugins.onStart`).
With a static dependency, we'll run into circular dependency issues.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/172922
## Summary
This PR:
- Integrates the new Settings application
(`packages/kbn-management/settings/application`) into stateful Kibana
and removes the old `management_app` from the
`src/plugins/advanced_settings` plugin.
- Adds support for section registry in the new Settings application, so
that other plugins can add their own sections to the Advanced settings
app.
- Adds functionality for disabling saving of settings based on the
provided capabilities of the current user.
<img width="1352" alt="Screenshot 2024-01-23 at 16 46 03"
src="1f3b7088-58e2-46e8-a7dd-ae0fc346b4ba">
<br><br>
"Usage collection" section in Global settings:
<img width="1099" alt="Screenshot 2024-01-23 at 16 48 24"
src="ebc54ad5-348b-46dd-a047-b418ddc7ba4f">
### How to test
**Testing Advanced settings in stateful Kibana:**
1. Start Es with `yarn es snapshot` and Kibana with `yarn start`
2. Go to Stack Management -> Advanced Settings
3. Verify that the app functions correctly. Both tabs (for space and
global settings) should be displayed, setting fields should be editable
and saveable, etc.
**Testing the section registry**
Currently, `telemetry_management_section` is the only plugin that
registers a section - the "Usage collection" section under the "Global
settings" tab. This should work correctly in stateful Kibana.
1. Start Es with `yarn es snapshot --license=trial` and Kibana with
`yarn start`
2. Go to Stack Management -> Advanced Settings and select the "Global
settings" tab
3. Scroll down and verify that the "Usage collection" section is
displayed and works as expected.
**Testing with different capabilities:**
1. Start Es with `yarn es snapshot` and Kibana with `yarn start`
2. Go to Stack Management -> Roles
3. Create a role that has "Read" access to Advanced settings and one
that doesn't have any access.
4. Create users with each of these two roles.
5. Log in with these users and verify that the user with "Read" access
can see the app but cannot edit it, and the user with no privileges
cannot access the app.
**Testing Advanced settings in serverless Kibana:**
The Advanced settings app in serverless shouldn't be affected by these
changes.
1. Start Es with `yarn es serverless` and Kibana with `yarn
serverless-{es/oblt/security}`
2. Go to Management -> Advanced Settings
3. Verify that the app functions correctly. There shouldn't be any tabs
as there are no spaces.
<!--
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Renames test subjects and page objects
- Renames test folders from `observability_log_explorer` to
`observability_logs_explorer`
- Changes app url from `observability-log-explorer` to
`observability-logs-explorer` and adds another app for redirects
Related to https://github.com/elastic/kibana/issues/171991
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
- Rename `log_explorer` to `logs_explorer` and move it to an
`observability_solution` subfolder
- Rename `observability_log_explorer` to `observability_logs_explorer`
and move it to an `observability_solution` subfolder
- Use `renameFromRoot` to rename old configs
Related to https://github.com/elastic/kibana/issues/171991 and
https://github.com/elastic/kibana/pull/170759
This PR is mostly focused on renaming the plugins, follow ups will be
created to change mentions of `log explorer`, `logExplorer` etc.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani01@gmail.com>
