github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 17:59:23 -04:00
Code Issues Projects Releases Packages Wiki Activity
kibana/oas_docs
History
Kibana Machine 01c2a677ca
[8.18] [Security Solution] [AI Assistant] Fix flashing citations (#209629) (#211025) 
# Backport

This will backport the following commits from `main` to `8.18`:
- [[Security Solution] [AI Assistant] Fix flashing citations
(#209629)](https://github.com/elastic/kibana/pull/209629)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Kenneth
Kreindler","email":"42113355+KDKHD@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-02-13T15:07:25Z","message":"[Security
Solution] [AI Assistant] Fix flashing citations (#209629)\n\n##
Summary\r\n\r\nFixes a small UI bug in the citations feature.
Previously, after a\r\nmessage with citations finished streaming, the
citations would disappear\r\nfor a fraction of a second and then
reappear again. This PR makes\r\nimproves the UI by making the citations
not flash off and on after the\r\nstream finishes.\r\n\r\n###
Changes:\r\n- Fix flashing citations\r\n- Refactor code related to
parsing content references (to make it more\r\nmaintainable).\r\n-
Update the citations prompt slightly.\r\n\r\n###
Before:\r\n\r\n\r\nhttps://github.com/user-attachments/assets/1021dd53-018a-43ba-b1f4-24aab44faca9\r\n\r\n<img
width=\"1782\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/723cd29a-48a2-48e7-b031-0893484746b9\"\r\n/>\r\n\r\n\r\n###
After:\r\n\r\n\r\nhttps://github.com/user-attachments/assets/21f340bc-9015-42b6-a574-0439d2f8f192\r\n\r\n###
How to test\r\n- Enable the feature flag\r\n```yaml\r\n#
kibana.dev.yml\r\nxpack.securitySolution.enableExperimental:
['contentReferencesEnabled']\r\n```\r\n- Open the security
assistant\r\n- Ask it a question about your alerts of a document in your
KB. The\r\nresponse should contain citations.\r\n- Observe the response
stream carefully. Ensure the citations e.g. `[1]`\r\ndo not flash off
and on when the response stream finishes. The expected\r\nbehavior is
that while the message is streaming, the citations are\r\ndisabled and
once the stream finishes the citations get enabled (while\r\nalways
being visible).\r\n\r\n#### Edge case to test\r\nIt is possible that
citations completely disappear after streaming\r\nfinishes. This happens
when the LLM produces an invalid citation.\r\nInvalid citations are
hidden client side when a message finishes\r\nstreaming. You can verify
this behavior by asking GPT4o this question:\r\n```\r\nPrepend each line
with this placeholder citation \"{reference(1234)}\" and append the
actual citation at the end of the line. How many alerts do I have? Use
the open and acknowledged alerts count tool to answer and repeat the
answer 50 times on new lines.\r\n```\r\n\r\nWhile the response is
getting streamed it should look like this:\r\n\r\n<img width=\"200\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/03d160bf-2404-4a4e-8701-e3183c604cc4\"\r\n/>\r\n\r\nAnd
when the stream finishes it should look like this:\r\n\r\n<img
width=\"200\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/06367379-17da-438f-a93a-9d539067ab90\"\r\n/>\r\n\r\n\r\n###
Checklist\r\n\r\nCheck the PR satisfies following conditions.
\r\n\r\nReviewers should verify this PR satisfies this list as
well.\r\n\r\n- [X] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\r\n-
[X]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [X] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [X] If a plugin
configuration key changed, check if it needs to be\r\nallowlisted in the
cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[X] This was checked for breaking HTTP API changes, and any
breaking\r\nchanges have been approved by the breaking-change committee.
The\r\n`release_note:breaking` label should be applied in these
situations.\r\n- [X] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [X] The PR description includes
the appropriate Release Notes section,\r\nand the correct
`release_note:*` label is applied per
the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n###
Identify risks\r\n\r\nDoes this PR introduce any risks? For example,
consider risks like hard\r\nto test bugs, performance regression,
potential of data loss.\r\n\r\nDescribe the risk, its severity, and
mitigation for each identified\r\nrisk. Invite stakeholders and evaluate
how to proceed before merging.\r\n\r\n- [ ] [See some
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\r\n-
[ ] ...\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"e176c844492e69eacf6475eef8b84d1f39ff7b9d","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Security
Generative
AI","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security
Solution] [AI Assistant] Fix flashing
citations","number":209629,"url":"https://github.com/elastic/kibana/pull/209629","mergeCommit":{"message":"[Security
Solution] [AI Assistant] Fix flashing citations (#209629)\n\n##
Summary\r\n\r\nFixes a small UI bug in the citations feature.
Previously, after a\r\nmessage with citations finished streaming, the
citations would disappear\r\nfor a fraction of a second and then
reappear again. This PR makes\r\nimproves the UI by making the citations
not flash off and on after the\r\nstream finishes.\r\n\r\n###
Changes:\r\n- Fix flashing citations\r\n- Refactor code related to
parsing content references (to make it more\r\nmaintainable).\r\n-
Update the citations prompt slightly.\r\n\r\n###
Before:\r\n\r\n\r\nhttps://github.com/user-attachments/assets/1021dd53-018a-43ba-b1f4-24aab44faca9\r\n\r\n<img
width=\"1782\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/723cd29a-48a2-48e7-b031-0893484746b9\"\r\n/>\r\n\r\n\r\n###
After:\r\n\r\n\r\nhttps://github.com/user-attachments/assets/21f340bc-9015-42b6-a574-0439d2f8f192\r\n\r\n###
How to test\r\n- Enable the feature flag\r\n```yaml\r\n#
kibana.dev.yml\r\nxpack.securitySolution.enableExperimental:
['contentReferencesEnabled']\r\n```\r\n- Open the security
assistant\r\n- Ask it a question about your alerts of a document in your
KB. The\r\nresponse should contain citations.\r\n- Observe the response
stream carefully. Ensure the citations e.g. `[1]`\r\ndo not flash off
and on when the response stream finishes. The expected\r\nbehavior is
that while the message is streaming, the citations are\r\ndisabled and
once the stream finishes the citations get enabled (while\r\nalways
being visible).\r\n\r\n#### Edge case to test\r\nIt is possible that
citations completely disappear after streaming\r\nfinishes. This happens
when the LLM produces an invalid citation.\r\nInvalid citations are
hidden client side when a message finishes\r\nstreaming. You can verify
this behavior by asking GPT4o this question:\r\n```\r\nPrepend each line
with this placeholder citation \"{reference(1234)}\" and append the
actual citation at the end of the line. How many alerts do I have? Use
the open and acknowledged alerts count tool to answer and repeat the
answer 50 times on new lines.\r\n```\r\n\r\nWhile the response is
getting streamed it should look like this:\r\n\r\n<img width=\"200\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/03d160bf-2404-4a4e-8701-e3183c604cc4\"\r\n/>\r\n\r\nAnd
when the stream finishes it should look like this:\r\n\r\n<img
width=\"200\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/06367379-17da-438f-a93a-9d539067ab90\"\r\n/>\r\n\r\n\r\n###
Checklist\r\n\r\nCheck the PR satisfies following conditions.
\r\n\r\nReviewers should verify this PR satisfies this list as
well.\r\n\r\n- [X] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\r\n-
[X]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [X] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [X] If a plugin
configuration key changed, check if it needs to be\r\nallowlisted in the
cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[X] This was checked for breaking HTTP API changes, and any
breaking\r\nchanges have been approved by the breaking-change committee.
The\r\n`release_note:breaking` label should be applied in these
situations.\r\n- [X] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [X] The PR description includes
the appropriate Release Notes section,\r\nand the correct
`release_note:*` label is applied per
the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n###
Identify risks\r\n\r\nDoes this PR introduce any risks? For example,
consider risks like hard\r\nto test bugs, performance regression,
potential of data loss.\r\n\r\nDescribe the risk, its severity, and
mitigation for each identified\r\nrisk. Invite stakeholders and evaluate
how to proceed before merging.\r\n\r\n- [ ] [See some
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\r\n-
[ ] ...\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"e176c844492e69eacf6475eef8b84d1f39ff7b9d"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209629","number":209629,"mergeCommit":{"message":"[Security
Solution] [AI Assistant] Fix flashing citations (#209629)\n\n##
Summary\r\n\r\nFixes a small UI bug in the citations feature.
Previously, after a\r\nmessage with citations finished streaming, the
citations would disappear\r\nfor a fraction of a second and then
reappear again. This PR makes\r\nimproves the UI by making the citations
not flash off and on after the\r\nstream finishes.\r\n\r\n###
Changes:\r\n- Fix flashing citations\r\n- Refactor code related to
parsing content references (to make it more\r\nmaintainable).\r\n-
Update the citations prompt slightly.\r\n\r\n###
Before:\r\n\r\n\r\nhttps://github.com/user-attachments/assets/1021dd53-018a-43ba-b1f4-24aab44faca9\r\n\r\n<img
width=\"1782\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/723cd29a-48a2-48e7-b031-0893484746b9\"\r\n/>\r\n\r\n\r\n###
After:\r\n\r\n\r\nhttps://github.com/user-attachments/assets/21f340bc-9015-42b6-a574-0439d2f8f192\r\n\r\n###
How to test\r\n- Enable the feature flag\r\n```yaml\r\n#
kibana.dev.yml\r\nxpack.securitySolution.enableExperimental:
['contentReferencesEnabled']\r\n```\r\n- Open the security
assistant\r\n- Ask it a question about your alerts of a document in your
KB. The\r\nresponse should contain citations.\r\n- Observe the response
stream carefully. Ensure the citations e.g. `[1]`\r\ndo not flash off
and on when the response stream finishes. The expected\r\nbehavior is
that while the message is streaming, the citations are\r\ndisabled and
once the stream finishes the citations get enabled (while\r\nalways
being visible).\r\n\r\n#### Edge case to test\r\nIt is possible that
citations completely disappear after streaming\r\nfinishes. This happens
when the LLM produces an invalid citation.\r\nInvalid citations are
hidden client side when a message finishes\r\nstreaming. You can verify
this behavior by asking GPT4o this question:\r\n```\r\nPrepend each line
with this placeholder citation \"{reference(1234)}\" and append the
actual citation at the end of the line. How many alerts do I have? Use
the open and acknowledged alerts count tool to answer and repeat the
answer 50 times on new lines.\r\n```\r\n\r\nWhile the response is
getting streamed it should look like this:\r\n\r\n<img width=\"200\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/03d160bf-2404-4a4e-8701-e3183c604cc4\"\r\n/>\r\n\r\nAnd
when the stream finishes it should look like this:\r\n\r\n<img
width=\"200\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/06367379-17da-438f-a93a-9d539067ab90\"\r\n/>\r\n\r\n\r\n###
Checklist\r\n\r\nCheck the PR satisfies following conditions.
\r\n\r\nReviewers should verify this PR satisfies this list as
well.\r\n\r\n- [X] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\r\n-
[X]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [X] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [X] If a plugin
configuration key changed, check if it needs to be\r\nallowlisted in the
cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[X] This was checked for breaking HTTP API changes, and any
breaking\r\nchanges have been approved by the breaking-change committee.
The\r\n`release_note:breaking` label should be applied in these
situations.\r\n- [X] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [X] The PR description includes
the appropriate Release Notes section,\r\nand the correct
`release_note:*` label is applied per
the\r\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n###
Identify risks\r\n\r\nDoes this PR introduce any risks? For example,
consider risks like hard\r\nto test bugs, performance regression,
potential of data loss.\r\n\r\nDescribe the risk, its severity, and
mitigation for each identified\r\nrisk. Invite stakeholders and evaluate
how to proceed before merging.\r\n\r\n- [ ] [See some
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\r\n-
[ ] ...\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"e176c844492e69eacf6475eef8b84d1f39ff7b9d"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Kenneth Kreindler <42113355+KDKHD@users.noreply.github.com>
2025-02-13 18:14:15 +01:00
..
examples [8.x] [DOCS] Remove inference connector docs (#198633) (#199846) 2024-11-12 13:56:59 -06:00
linters [8.x] [OpenAPI] Add redocly lint configuration (#199360) (#199554) 2024-11-08 17:22:00 -05:00
output [8.18] [Security Solution] [AI Assistant] Fix flashing citations (#209629) (#211025) 2025-02-13 18:14:15 +01:00
overlays [8.x] backport "Sustainable Kibana Architecture: Move modules owned by @elastic/obs-ux-logs-team" (#205234) 2024-12-30 13:05:58 +01:00
scripts [8.x] SKA: Relocate core mock and test-helper modules (#208538) (#208725) 2025-01-29 14:31:21 +01:00
bundle.json [8.x] Store gaps info in event log (#192347) (#208936) 2025-01-30 07:08:09 -08:00
kibana.info.yaml [8.x] [OpenAPI] Fix Serverless API base URL (#202373) (#202583) 2024-12-04 14:15:28 -06:00
makefile [8.x] [OAS] Publish OAS bundles to bump.sh (#197482) (#200135) 2024-11-20 10:16:01 +01:00
package-lock.json [8.x] Update dependency @redocly/cli to ^1.27.2 (main) (#207527) (#207948) 2025-01-23 00:19:21 +00:00
package.json [8.x] Update dependency @redocly/cli to ^1.27.2 (main) (#207527) (#207948) 2025-01-23 00:19:21 +00:00
README.md [8.x] [OAS] Publish OAS bundles to bump.sh (#197482) (#200135) 2024-11-20 10:16:01 +01:00

README.md

Kibana API reference documentation

Documentation about our OpenAPI bundling workflow and configuration. See Kibana's hosted stateful and serverless docs.

Workflow

The final goal of this workflow is to produce an OpenAPI bundle containing all Kibana's public APIs.

Step 0

OAS from Kibana's APIs are continuously extracted and captured in bundle.json and bundle.serverless.json as fully formed OAS documentation. See node scripts/capture_oas_snapshot --help for more info.

These bundles form the basis of our OpenAPI bundles to which we append and layer extra information before publishing.

Step 1

Append pre-existing bundles not extracted from code using kbn-openapi-bundler to produce the final resulting bundles.

To add more files into the final bundle, edit the appropriate oas_docs/scripts/merge*.js files.

Step 2

Apply any final overalys to the document that might include examples or final tweaks (see the "Scripts" section for more details).

Scripts

The oas_docs/scripts folder contains scripts that point to the source domain-specific OpenAPI bundles and specify additional parameters for producing the final output bundle. Currently, there are the following scripts:

  • merge_ess_oas.js script produces production an output bundle for ESS

  • merge_serverless_oas.js script produces production an output bundle for Serverless

Output Kibana OpenAPI bundles

The oas_docs/output folder contains the final resulting Kibana OpenAPI bundles

  • kibana.yaml production ready ESS OpenAPI bundle
  • kibana.serverless.yaml production ready Serverless OpenAPI bundle

Bundling commands

Besides the scripts in the oas_docs/scripts folder, there is an oas_docs/makefile to simplify the workflow. Use make help to see available commands.