mirror of
https://github.com/elastic/kibana.git
synced 2025-06-27 18:51:07 -04:00
47d308604f
## Summary This is a follow-up to https://github.com/elastic/kibana/pull/184974 that updates the KB Entries API's (and underlying schema) to support `IndexEntries` in addition to `DocumentEntries`. `IndexEntries` are entries in the Knowledge Base that are not backed by an embedded raw text source, but rather by an entire Index or Data Stream. The user can set the data source name, the specific field to query (must be ELSER embeddings in this initial implementation), and a description for when the assistant should search this data source for Knowledge Base content. This essentially enables the user to create custom retrieval tools backed by their own data. The changes in this PR, as with the other recent KB enhancements, are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` however as code change is required to test the new mappings. For this you can update the `knowledgeBaseDataStream` in `x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts` to ```ts this.knowledgeBaseDataStream = this.createDataStream({ resource: 'knowledgeBase', kibanaVersion: options.kibanaVersion, fieldMap: knowledgeBaseFieldMapV2, // Update this to the V2 mapping }); ``` Change set includes: - [X] ES Knowledge Base data stream schema and OAS has been updated to support `IndexEntries`. - [X] OAS schema files have been moved to the `/entries` sub-directory - [ ] Backend KB services have been updated to support `IndexEntries` - [X] Storage methods updated - [ ] Retrieval methods updated (will round out these endpoint when working the UI next) --- With these API changes, I've also introduced a few sample `*.http` files for easier development/testing. These files are supported out of the box in JetBrains IDE's or in VSCode with the [httpyac](https://httpyac.github.io/) (and many other) extensions. Since the configuration for these files includes a `-` in the name, that's why you'll see a few @elastic/kibana-operations files updated. You can read more about `http` files [here](https://www.jetbrains.com/help/webstorm/http-client-in-product-code-editor.html) and for the spec see this repo [here](https://github.com/JetBrains/http-request-in-editor-spec/blob/master/spec.md). If we find these useful, we could add support to our [OpenAPI Generator](https://openapi-generator.tech/docs/generators/jetbrains-http-client) to create these automatically. They currently live co-located next to the OAS and generated schema files here: ``` x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/bulk_crud_knowledge_base_entries_route.http x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http ``` and the main config here: ``` x-pack/packages/kbn-elastic-assistant-common/env/http-client.env.json ``` The `x-pack/packages/kbn-elastic-assistant-common/.gitignore` has been updated to ignore `http-client.private.env.json` files locally, which is how you can override the config as you'd like. This is helpful to add variables like `basePath` as below: ``` { "dev": { "basePath": "/kbn" } } ``` To use them, just open the corresponding `*.http` for the API you want to test, and click `Send`, and the response will open in another tab. Here is what that looks like for creating one of the new `IndexEntry` KB documents that have been introduced in this PR: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/c9e70d1a-28d2-4eb3-9853-ab6d8e1c7acf" /> </p> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in https://github.com/elastic/security-docs/issues/5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| src | ||
| index.ts | ||
| jest.config.js | ||
| jest.integration.config.js | ||
| kibana.jsonc | ||
| package.json | ||
| README.mdx | ||
| tsconfig.json | ||
--- id: kibDevDocsOpsCliDevMode slug: /kibana-dev-docs/ops/cli-dev-mode title: "@kbn/cli-dev-mode" description: A package to manage the Kibana cli behavior when in development date: 2022-05-24 tags: ['kibana', 'dev', 'contributor', 'operations', 'cli', 'dev', 'mode'] --- This package exposes a function that manages the alternate behavior of the Kibana cli when using the `--dev` flag. This mode provides several useful features in a single CLI for a nice developer experience: - automatic server restarts when code changes - runs the `@kbn/optimizer` to build browser bundles - runs a base path proxy which helps developers test that they are writing code which is compatible with custom basePath settings while they work - pauses requests when the server or optimizer are not ready to handle requests so that when users load Kibana in the browser it's always using the code as it exists on disk To accomplish this, and to make it easier to test, the `CliDevMode` class manages the following objects. ## `Watcher` The `Watcher` manages a [@parcel/watcher](https://github.com/parcel-bundler/watcher) instance to watch the server files, logs about file changes observed and provides an observable to the `DevServer` via its `serverShouldRestart$()` method. ## `DevServer` The `DevServer` object is responsible for everything related to running and restarting the Kibana server process: - listens to restart notifications from the `Watcher` object, sending `SIGKILL` to the existing server and launching a new instance with the current code - writes the stdout/stderr logs from the Kibana server to the parent process - gracefully kills the process if the SIGINT signal is sent - kills the server if the SIGTERM signal is sent, process.exit() is used, a second SIGINT is sent, or the graceful shutdown times out - proxies SIGHUP notifications to the child process, though the core team is working on migrating this functionality to the KP and making this unnecessary ## `Optimizer` The `Optimizer` object manages a `@kbn/optimizer` instance, adapting its configuration and logging to the data available to the CLI. ## `BasePathProxyServer` This proxy injects a random three character base path in the URL that Kibana is served from to help ensure that Kibana features are written to adapt to custom base path configurations from users. The basePathProxy also has another important job, ensuring that requests don't fail because the server is restarting and that the browser receives front-end assets containing all saved changes. We accomplish this by observing the ready state of the `Optimizer` and `DevServer` objects and pausing all requests through the proxy until both objects report that they aren't building/restarting based on recently saved changes.