## Summary
Part of https://github.com/elastic/kibana/issues/206710
This PR introduces the `InferenceChatModel` class, which is a langchain
chatModel utilizing the inference APIs (`chatComplete`) under the hood.
Creating instances of `InferenceChatModel` can either be done by
manually importing the class from the new `@kbn/inference-langchain`
package, or by using the new `createChatModel` API exposes from the
inference plugin's start contract.
The main upside of using this chatModel is that the unification and
normalization layers are already being taken care of by the inference
plugin, making sure that the underlying models are being used with the
exact same capabilities. More details on the upsides and reasoning in
the associated issue.
### Usage
Usage is very straightforward
```ts
const chatModel = await inferenceStart.getChatModel({
request,
connectorId: myInferenceConnectorId,
chatModelOptions: {
temperature: 0.2,
},
});
// just use it as another langchain chatModel, e.g.
const response = await chatModel.stream('What is Kibana?');
for await (const chunk of response) {
// do something with the chunk
}
```
### Important
This PR is only adding the implementation, and not wiring it anywhere or
using it in any existing code. This is meant to be done in a later
stage. Merging that implementation first will allow to have distinct PRs
for the integration with search (playground) and security (assistant +
other workflows), with proper testing
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Often we've failures on the MKI FTR runs in Buildkite, that are due to
our roles files being out of sync.
This is an attempt to get ahead of the problem, or to at least be
notified that a change is coming.
## Details
⚠️ This PR was created by an automated tool. Please review the changes
carefully. ⚠️
Add a workflow that will comment on PRs with AsciiDoc changes.
## Why
During the migration to Elastic Docs v3, the Docs team will focus
exclusively on migrating content.
To maintain consistency, prevent conflicts, and ensure a smoother
transition we will freeze all AsciiDoc changes.
This means you will get a warning when you create AsciiDoc changes in
your PRs.
See https://github.com/elastic/docs-builder/issues/281 for details
If there are any questions, please reach out to the
@elastic/docs-engineering
---------
Co-authored-by: Brandon Morelli <brandon.morelli@elastic.co>
## Summary
Add MapsPage to the scout core to be re-used by others.
Add rudimentary docs to show how to run these tests.
Added a `waitForRender` method.
Add test id.
---------
Co-authored-by: Robert Oskamp <traeluki@gmail.com>
Co-authored-by: Nick Partridge <nicholas.partridge@elastic.co>
Co-authored-by: Nick Partridge <nick.ryan.partridge@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
A minimal version of the file upload tool which can be triggered via a
uiAction.
The trigger takes a callback to enable subsequent actions after the
upload. This callback receives information about the upload, the index
and data view created and information about the files:
```
{
"index": "test9",
"dataView": {
"id": "a870ef68-a624-4df1-9d5d-fa62b75dd297",
"title": ""
},
"files": [
{
"fileName": "farequote-tiny.csv",
"docCount": 20,
"fileFormat": "delimited"
},
{
"fileName": "farequote.csv",
"docCount": 86275,
"fileFormat": "delimited"
}
]
}
```
If `autoAddInference` is set with the name of an inference endpoint
(`autoAddInference: '.elser-2-elasticsearch'`) the tool with
automatically add a `semantic_text` to the mappings for tika files (pdf,
txt docx)
Currently embedded in the search app's home page and playground. In
playgroubnd, after upload is complete and the flyout closed, the newly
created index will be selected.
https://github.com/user-attachments/assets/0589fa02-fb0e-400b-8e74-1eb9a993c6ba
Multiple files can be uploaded at once. They must be of the same file
format and the mappings cannot clash.
If more than once file has the same field but they are of different
types, the files are considered incompatible.
<img width="612" alt="image"
src="https://github.com/user-attachments/assets/67307fd0-7d10-4eab-9e72-df133ebddcfe"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
1. Extends the server-side prototype pollution protections introduced in
https://github.com/elastic/kibana/pull/190716 to include
`Array.prototype`.
2. Applies the same prototype pollution protections to the client-side.
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] Sealing prototypes on the client can lead to failures in
third-party dependencies. I'm relying on sufficient functional test
coverage to detect issues here. As a result, these protections are
disabled by default for now, and can be controlled via setting
`server.prototypeHardening: true/false`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR introduces initial cypress test and also a basic test was
configured to test for the presence of the Asset Inventory page when the
Feature Flag `assetInventoryUXEnabled` is set.
This PR resolves#194605 and closes#170132 and brings the following
changes:
- changed ELU metrics evaluation used for autoscaling;
- a rate limiter to throttle incoming requests when under a high load;
- a configuration option to exclude some routes from the rate limiter.
## Summary
As K8S Dashboard is currently hidden on main , the code serves no
purpose other than potentially causing Tech debts whenever a refactor or
a migration happens. As such its better to remove it completely. In case
we want to bring it back later we will just pull it from git history
> [!CAUTION]
> **This should only affect Serverless and Main, 8.x.x should still be
able to see and access K8S Dashboard**
## Related Tickets
- https://github.com/elastic/security-team/issues/11418
- https://github.com/elastic/security-team/issues/10735
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Paulo Silva <paulo.henrique@elastic.co>
Co-authored-by: Maxim Kholod <maxim.kholod@elastic.co>
## Summary
Closes https://github.com/elastic/kibana/issues/203967
Supports dashboard variables in ES|QL charts.
This PR introduces the first phase of ES|QL controls. In this phase:
- the flow starts from Lens ES|QL editor (and no vice-versa, this will
happen on a later phase after we discuss some technical details with ES)
- it is only available for dashboards (we want to include them in other
apps as Discover but this is the next phase driven by the presentation
team)
- it supports variables for intervals, fields and values. I haven't
added support for functions. I am going to do it after this PR being
merged (there are some business questions I want to answer first)
For more info check this
[deck](https://docs.google.com/presentation/d/1qSbWLSoC5SseXuLix763vpp8sa7ikp3pQTbHImEHBoc)
### Implementation details
- There is a new service, the ESQLVariables service that is responsible
for ES|QL variables. I isolated this to a new plugin owned by the ES|QL
team for cleaner code and for avoiding circular dependencies
- A new ESQL_CONTROL type got created. It follows the exact same logic
as the rest controls. No changes in the architecture here.
- The creation of the controls (the control forms) have been added in
the esql plugin.
- Lens has small changes:
- The support of variables in the textBased datasource
- Two callbacks needed to be called after the creation / cancellation of
an ES|QL control
### Types of ES|QL variables
We have 2 types:
- Static Values (the user gives a list of values with his own
responsibility). As the flow starts from the editor we can identify what
they most possibly want to do and we give the user some options but they
have the freedom to do as they want. A basic validation has been added
too.
- Values from an ES|QL query (the user gives an ES|QL query that
generates the values). As the flow starts from the editor we can suggest
a query for the users but they can always change it as they wish.
<img width="1168" alt="image"
src="https://github.com/user-attachments/assets/cc28beb8-111c-43ad-9f26-865bc62ae512"
/>
### Example of a control creation from the editor
### Release note
ES|QL charts now allow the creation of controls in dashboards. You can
control a part of the query such as a field, an interval or a value.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
Co-authored-by: Devon Thomson <devon.thomson@elastic.co>
## Summary
Summarize your PR. If it involves visual changes include a screenshot or
gif.
https://github.com/user-attachments/assets/6d7527d1-dc8d-4f3a-9b03-cfd0022701d2
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
We have some functional/API tests that were attributed to the wrong
team. This was [called
out](https://github.com/elastic/kibana/pull/194819/files#r1792639304) in
the inciting PR, but was never addressed.
There are probably some other misattributions here, and likely more due
to the Security/Security Solution confusion, but these are the ones I
know about right now.
## Summary
Current PR creates a new platform shared plugin named
`inference_endpoint` to expose in Kibana the new internal API
`_inference/_services`, which returns the list of inference providers
with the configuration settings.
Changed `@kbn/inference_endpoint_ui_common` package to fetch dynamically
the list of providers by using the route introduced in
`inference_endpoint` plugin.
Added fields settings filter based on the selected task in the
`supported_task_types`.
Cleaned up the types consolidating all in the package
`@kbn/inference_endpoint_ui_common`.
Changed .inference connector to use `unified_completion` subAction for
selected `chat_completion` task type.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co>
## 📓 Summary
Adds a new section to the overview tab in the log details flyout in
Discover to display stacktrace information for logs and exceptions.
In a follow-up, the stacktrace could be moved to a new tab in the log
details flyout and actions can be added to the stacktrace (and quality)
icons in the document table to open the relevant sections in the flyout.
Closes https://github.com/elastic/kibana/issues/190460
### APM - Log stacktrace (library frames)
<img width="1470" alt="image"
src="https://github.com/user-attachments/assets/8991f882-d329-4bc5-aa37-424576bcee72"
/>
### APM - Exception (with cause)
<img width="1476" alt="image"
src="https://github.com/user-attachments/assets/cfbf24a7-6f82-48f1-b275-5aac977411ac"
/>
### APM - Exception (simple stacktrace)
<img width="1474" alt="image"
src="https://github.com/user-attachments/assets/fc0306c4-5fcd-4b74-bb0d-c1784a48d677"
/>
### Apache Tomcat Integration (Catalina) - Stacktrace
<img width="1472" alt="image"
src="https://github.com/user-attachments/assets/281f1822-faea-4e2d-9515-c11a9ee12f50"
/>
## 📝 Notes for reviewers
- The `@kbn/apm-types` package was marked as platform / shared as it's
being used by the
[unified_doc_viewer](https://github.com/elastic/kibana/blob/main/src/plugins/unified_doc_viewer/kibana.jsonc)
- The code used to render stacktraces in APM was moved into a new
`@kbn/event-stacktrace` package as it is reused in the
`unified_doc_viewer`
- The code used to render metadata table in APM was moved into a new
`@kbn/key-value-metadata-table` package
## 🧪 Testing instructions
The deployed environments have sample logs that can be used (time range:
Jan 1, 2025 - now). For a local setup, please follow the instructions
below:
1. Ingest sample logs with stacktraces
([gist](https://gist.github.com/gbamparop/0da21ca7f65b24c4a9c071ce9e9b97b0)).
Please note that these are test data and some fields that are not used
by stacktraces might not be consistent
2. View relevant logs in Discover (Query: `service.name: "synth-node-0"
OR apache_tomcat :*`, Time range: Jan 1, 2025 - now)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/206664
This PR moves Profiling Cypress tests to be run on the main pipeline
instead of the unsupported one.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Fixes: https://github.com/elastic/kibana/issues/202715**
**Fixes: https://github.com/elastic/kibana/issues/204714**
## Summary
This PR makes inconsistent/wrong rule's look-back duration prominent for
a user. It falls back to a default 1 minute value in rule upgrade
workflow.
## Details
### Negative/wrong `lookback` problem
There is a difference between rule schedule value in a saved object and
value represented to users
- Saved object (and rule management API) has `interval`, `from` and `to`
fields representing rule schedule. `interval` shows how often a rule
runs in task runner. `from` and `to` stored in date math format like
`now-10m` represent a date time range used to fetch source events. Task
manager strives to run rules exactly every `interval` but it's not
always possible due to multiple reasons like system load and various
delays. To avoid any gaps to appear `from` point in time usually stands
earlier than current time minus `interval`, for example `interval` is
`10 minutes` and `from` is `now-12m` meaning rule will analyze events
starting from 12 minutes old. `to` represents the latest point in time
source events will be analyzed.
- Diffable rule and UI represent rule schedule as `interval` and
`lookback`. Where `interval` is the same as above and `lookback` and a
time duration before current time minus `interval`. For example
`interval` is `10 minutes` and lookback is `2 minutes` it means a rule
will analyzing events starting with 12 minutes old until the current
moment in time.
Literally `interval`, `from` and `to` mean a rule runs every `interval`
and analyzes events starting from `from` until `to`. Technically `from`
and `to` may not have any correlation with `interval`, for example a
rule may analyze one year old events. While it's reasonable for manual
rule runs and gap remediation the same approach doesn't work well for
usual rule schedule. Transformation between `interval`/`from`/`to` and
`interval`/`lookback` works only when `to` is equal the current moment
in time i.e. `now`.
Rule management APIs allow to set any `from` and `to` values resulting
in inconsistent rule schedule. Transformed `interval`/`lookback` value
won't represent real time interval used to fetch source events for
analysis. On top of that negative `lookback` value may puzzle users on
the meaning of the negative sign.
### Prebuilt rules with `interval`/`from`/`to` resulting in negative
`lookback`
Some prebuilt rules have such `interval`, `from` and `to` field values
thatnegative `lookback` is expected, for example `Multiple Okta Sessions
Detected for a Single User`. It runs every `60 minutes` but has `from`
field set to `now-30m` and `to` equals `now`. In the end we have
`lookback` equals `to` - `from` - `interval` = `30 minutes` - `60
minutes` = `-30 minutes`.
Our UI doesn't handle negative `lookback` values. It simply discards a
negative sign and substitutes the rest for editing. In the case above
`30 minutes` will be suggested for editing. Saving the form will result
in changing `from` to `now-90m`
<img width="1712" alt="image"
src="https://github.com/user-attachments/assets/05519743-9562-4874-8a73-5596eeccacf2"
/>
### Changes in this PR
This PR mitigates rule schedule inconsistencies caused by `to` fields
not using the current point in time i.e. `now`. The following was done
- `DiffableRule`'s `rule_schedule` was changed to have `interval`,
`from` and `to` fields instead of `interval` and `lookback`
- `_perform` rule upgrade API endpoint was adapted to the new
`DIffableRule`'s `rule_schedule`
- Rule upgrade flyout calculates and shows `interval` and `lookback` in
Diff View, readonly view and field form when `lookback` is non-negative
and `to` equals `now`
- Rule upgrade flyout shows `interval`, `from` and `to` in Diff View,
readonly view and field form when `to` isn't equal `now` or calculated
`lookback` is negative
- Rule upgrade flyout shows a warning when `to` isn't equal `now` or
calculated `lookback` is negative
- Rule upgrade flyout's JSON Diff shows `interval` and `lookback` when
`lookback` is non-negative and `to` equals `now` and shows `interval`,
`from` and `to` in any other case
- Rule details page shows `interval`, `from` and `to` in Diff View,
readonly view and field form when `to` isn't equal `now` or calculated
`lookback` is negative
- `maxValue` was added to `ScheduleItemField` to have an ability to
restrict input at reasonable values
## Screenshots
- Rule upgrade workflow (negative look-back)
<img width="2558" alt="Screenshot 2025-01-02 at 13 16 59"
src="https://github.com/user-attachments/assets/b8bf727f-11ca-424f-892b-b024ba7f847a"
/>
<img width="2553" alt="Screenshot 2025-01-02 at 13 17 20"
src="https://github.com/user-attachments/assets/9f751ea4-0ce0-4a23-a3b7-0a16494d957e"
/>
<img width="2558" alt="Screenshot 2025-01-02 at 13 18 24"
src="https://github.com/user-attachments/assets/6908ab02-4011-4a6e-85ce-e60d5eac7993"
/>
- Rule upgrade workflow (positive look-back)
<img width="2555" alt="Screenshot 2025-01-02 at 13 19 12"
src="https://github.com/user-attachments/assets/06208210-c6cd-4842-8aef-6ade5d13bd36"
/>
<img width="2558" alt="Screenshot 2025-01-02 at 13 25 31"
src="https://github.com/user-attachments/assets/aed38bb0-ccfb-479a-bb3b-e5442c518e63"
/>
- JSON view
<img width="2559" alt="Screenshot 2025-01-02 at 13 31 37"
src="https://github.com/user-attachments/assets/07575a81-676f-418e-8b98-48eefe11ab00"
/>
- Rule details page
<img width="2555" alt="Screenshot 2025-01-02 at 13 13 16"
src="https://github.com/user-attachments/assets/e977b752-9d50-4049-917a-af2e8e3f0dfe"
/>
<img width="2558" alt="Screenshot 2025-01-02 at 13 14 10"
src="https://github.com/user-attachments/assets/06d6f477-5730-48ca-a240-b5e7592bf173"
/>
## How to test?
- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to
`kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web
browser)
- Install an outdated version of the `security_detection_engine` Fleet
package
```bash
curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```
- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```
- Set "inconsistent" rule schedule for `Suspicious File Creation via
Kworker` rule by running a query below
```bash
curl -X PATCH --user elastic:changeme -H "Content-Type: application/json" -H "elastic-api-version: 2023-10-31" -H "kbn-xsrf: 123" -d '{"rule_id":"ae343298-97bc-47bc-9ea2-5f2ad831c16e","interval":"10m","from":"now-5m","to":"now-2m"}' http://localhost:5601/kbn/api/detection_engine/rules
```
- Open rule upgrade flyout for `Suspicious File Creation via Kworker`
rule
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
[Internal link](https://github.com/elastic/security-team/issues/10820)
to the feature details
Set @elastic/security-threat-hunting as codewoners of the SIEM
Migrations integration tests folder.
> [!NOTE]
> This feature needs `siemMigrationsEnabled` experimental flag enabled
to work.
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 1 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/grid-layout` | `src/platform/packages/private/kbn-grid-layout` |
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./examples/grid_example/tsconfig.type_check.json
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-grid-layout/jest.config.js
./src/platform/packages/private/kbn-repo-packages/package-map.json
./tsconfig.base.json
./tsconfig.base.type_check.json
./tsconfig.refs.json
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/platform/packages/private/kbn-grid-layout/jest.config.js:12
src/platform/packages/private/kbn-grid-layout/tsconfig.json:2
src/platform/packages/private/kbn-grid-layout/tsconfig.type_check.json:2
```
</details>
Closes#192718
## Summary
This PR add a deployment-agnostic testing environment for Observability
AI Assistant tests by unifying the duplicated tests for stateful and
serverless environments. It create the ObservabilityAIAssistantApiClient
to work seamlessly in both environments, enabling a single test to run
across stateful, CI, and MKI.
Initial efforts focus on deduplicating the `conversations.spec.ts` and
`connectors.spec.ts` files, as these already run in all environments.
Move / dedup the tests that exist in stateful and serverless. They run
in serverless CI but not MKI and add the skipMki tag.
`chat.spec.ts`
`complete.spec.ts`
`elasticsearch.spec.ts`
`public_complete.spec.ts`
`alerts.spec.ts`
`knowledge_base_setup.spec.ts`
`knowledge_base_status.spec.ts`
`knowledge_base.spec.ts`
`summarize.ts`
`knowledge_base_user_instructions.spec.ts`
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 2 plugin(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/entities-data-access-plugin` |
`x-pack/solutions/observability/plugins/entities_data_access` |
| `@kbn/entityManager-app-plugin` |
`x-pack/solutions/observability/plugins/entity_manager_app` |
#### 10 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/core-user-settings-server` |
`src/core/packages/user-settings/server` |
| `@kbn/core-user-settings-server-internal` |
`src/core/packages/user-settings/server-internal` |
| `@kbn/core-user-settings-server-mocks` |
`src/core/packages/user-settings/server-mocks` |
| `@kbn/calculate-auto` |
`src/platform/packages/shared/kbn-calculate-auto` |
| `@kbn/charts-theme` | `src/platform/packages/shared/kbn-charts-theme`
|
| `@kbn/palettes` | `src/platform/packages/shared/kbn-palettes` |
| `@kbn/saved-search-component` |
`src/platform/packages/shared/kbn-saved-search-component` |
| `@kbn/use-tracked-promise` |
`src/platform/packages/shared/kbn-use-tracked-promise` |
| `@kbn/response-ops-rule-form` |
`src/platform/packages/shared/response-ops/rule_form` |
| `@kbn/streams-schema` |
`x-pack/solutions/observability/packages/kbn-streams-schema` |
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./docs/developer/plugin-list.asciidoc
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/core/packages/user-settings/server-internal/jest.config.js
./src/core/packages/user-settings/server-mocks/jest.config.js
./src/platform/packages/private/kbn-repo-packages/package-map.json
./src/platform/packages/shared/kbn-calculate-auto/jest.config.js
./src/platform/packages/shared/kbn-charts-theme/jest.config.js
./src/platform/packages/shared/kbn-palettes/jest.config.js
./src/platform/packages/shared/kbn-saved-search-component/jest.config.js
./src/platform/packages/shared/kbn-use-tracked-promise/jest.config.js
./src/platform/packages/shared/response-ops/rule_form/jest.config.js
./tsconfig.base.json
./x-pack/solutions/observability/packages/kbn-streams-schema/jest.config.js
./x-pack/solutions/observability/plugins/entities_data_access/jest.config.js
./x-pack/solutions/observability/plugins/entity_manager_app/jest.config.js
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/core/packages/user-settings/server-internal/jest.config.js:12
src/core/packages/user-settings/server-internal/tsconfig.json:2
src/core/packages/user-settings/server-mocks/jest.config.js:12
src/core/packages/user-settings/server-mocks/tsconfig.json:2
src/core/packages/user-settings/server/tsconfig.json:2
src/platform/packages/shared/kbn-calculate-auto/jest.config.js:12
src/platform/packages/shared/kbn-calculate-auto/tsconfig.json:2
src/platform/packages/shared/kbn-charts-theme/jest.config.js:12
src/platform/packages/shared/kbn-charts-theme/tsconfig.json:2
src/platform/packages/shared/kbn-palettes/jest.config.js:12
src/platform/packages/shared/kbn-palettes/tsconfig.json:2
src/platform/packages/shared/kbn-saved-search-component/jest.config.js:12
src/platform/packages/shared/kbn-saved-search-component/tsconfig.json:2
src/platform/packages/shared/kbn-use-tracked-promise/jest.config.js:12
src/platform/packages/shared/kbn-use-tracked-promise/tsconfig.json:2
src/platform/packages/shared/response-ops/rule_form/jest.config.js:12
src/platform/packages/shared/response-ops/rule_form/tsconfig.json:2
x-pack/solutions/observability/packages/kbn-streams-schema/jest.config.js:10
x-pack/solutions/observability/packages/kbn-streams-schema/tsconfig.json:2
x-pack/solutions/observability/plugins/entities_data_access/jest.config.js:12
x-pack/solutions/observability/plugins/entities_data_access/tsconfig.json:2
x-pack/solutions/observability/plugins/entity_manager_app/jest.config.js:12
x-pack/solutions/observability/plugins/entity_manager_app/tsconfig.json:2
x-pack/solutions/observability/plugins/entity_manager_app/tsconfig.json:7
```
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Part of #https://github.com/elastic/streams-program/issues/32
This work implements a UI for basic stream enrichment, supporting grok
and dissect processor + detected fields mapping.
The main features implemented in this PR consist of:
- **Sortable processors list**
- **Add new processor - Grok, Dissect**
- Ad-hoc forms for each processor
- Simulated document outcome with extracted fields
- Filter matching documents with parsed fields
- Mapping detected fields (only available for wired streams)
- **Edit processor**
- Change configuration only
- Delete processor CTA
As a side quest, I added a small package for object utils as
@simianhacker suggested.
`@kbn/object-utils` exposes `calculateObjectDiff` and `flattenObject` to
detect the changed fields in a simulation.
## 🔜 Follow-up work
I'll work on minor updates on top of this MVP to make this available for
further testing from the team.
The next steps will be:
- **Tests** for features that consolidate on the functional pov.
- Better field mapping detection and UI feedback (infer the type of the
detected field, currently always unmapped)
- Add better form validation and feedback for processor configuration.
As discussed offline, state management is purely based on the built-in
react APIs + react-hook-form. It could be improved with different
approaches, including a more solid state management library to make it
easier to maintain and bulletproof to race conditions. No state syncs
with the URL currently.
## 🎥 Demo
https://github.com/user-attachments/assets/a48fade9-f5aa-4270-bb19-d91d1eed822b
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This change converts Cypress tests for the custom logs flow into
Playwright using [the Scout
wrapper](https://github.com/elastic/kibana/tree/main/packages/kbn-scout).
> [!NOTE]
> As Scout package is still being developed, the PR pipeline configured
to runs Playwright tests only when code in certain plugins have been
changed and not on every PR.
### How to run tests locally
Start the Scout server
```bash
node scripts/scout.js start-server --stateful
```
In a separate terminal run the tests
```bash
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts
```
Playwright runs browsers in a headless mode by default, user `--headed`
option if needed
## Summary
Turn AssetInventory plugin into a simple Security solution page based on
this conversation:
- https://github.com/elastic/security-team/issues/10346
Follow-up of this work, where we initially developed the Asset Inventory
bare-bones as a plugin:
- https://github.com/elastic/kibana/issues/201704
Part of this work stream though not originally planned:
- https://github.com/elastic/security-team/issues/11247?reload=1
### Motivation
Reasoning behind is enabling fast development changes without any major
refactors in other packages/components that need to be reused, which
seems impossible as of now with the current codebase.
### Checklist
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
In the future, we'll probably have to turn AssetInventory into a
sub-plugin again. Doing so doesn't bring any risk in the present, and
should only involve the re-generation of the deleted files in this PR.
---------
Co-authored-by: Paulo Henrique <paulo.henrique@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Turns out, claude can in some situations (when returning both text and
toolcall in a single message) starts their toolcall index at `1` instead
of `0`, which introducing null values in the concatenated messages.
This fixes it, by removing null values from the tool calls when merging
the chunks.
Also remove the SKA codeowner override for the inference plugin to get
back the shared ownership
## Summary
* Delete unused package `@kbn/formatters`
* Delete unused package `@kbn/response-ops-feature-flag-service`
* Flag `@kbn/generate-console-definitions` as `devOnly: true`
* Flag `@kbn/plugin-check` as `devOnly: true`
* Flag `@kbn/set-map` as `devOnly: true`
* Flag `@kbn/synthetics-private-location` as `devOnly: true`
* Categorise `@kbn/calculate-auto` as `platform/shared`
* Categorise `@kbn/charts-theme` as `platform/shared`
* Categorise `@kbn/saved-search-component` as `platform/shared`
* Categorise `@kbn/use-tracked-promise` as `platform/shared`
* Categorise `@kbn/charts-theme` as `platform/shared`
* Categorise `@kbn/response-ops-rule-form` as `platform/shared`
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 3 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/code-editor` |
`src/platform/packages/shared/shared-ux/code_editor/impl` |
| `@kbn/code-editor-mock` |
`src/platform/packages/shared/shared-ux/code_editor/mocks` |
| `@kbn/monaco` | `src/platform/packages/shared/kbn-monaco` |
<details >
<summary>Updated relative paths</summary>
```
src/platform/packages/shared/kbn-monaco/jest.config.js:12
src/platform/packages/shared/kbn-monaco/tsconfig.json:2
src/platform/packages/shared/kbn-monaco/tsconfig.type_check.json:2
src/platform/packages/shared/shared-ux/code_editor/impl/jest.config.js:12
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.json:16
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.json:2
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:18
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:2
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:25
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:28
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:31
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:34
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:37
src/platform/packages/shared/shared-ux/code_editor/impl/tsconfig.type_check.json:40
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.json:16
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.json:2
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.type_check.json:18
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.type_check.json:2
src/platform/packages/shared/shared-ux/code_editor/mocks/tsconfig.type_check.json:25
```
</details>
Links dashboard to Streams.
Changes:
- Introduces `IndexStorageAdapter` to manage ES indices - see
https://github.com/dgieselaar/kibana/blob/streams-app-asset-linking/x-pack/solutions/observability/packages/utils_server/es/storage/README.md
for motivation
- Introduces `AssetClient` and `AssetService` to manage asset links with
`IndexStorageAdapter`
- `RepositorySupertestClient` to make it easier to use
`@kbn/server-route-repository` with FTR tests
- refactors related to above changes
---------
Co-authored-by: Chris Cowan <chris@elastic.co>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR:
- Updates the Kibana [Alert and action
settings](https://www.elastic.co/guide/en/kibana/current/alert-action-settings-kb.html)
page to be based off of a YAML source file
(`/docs/settings-gen/source/kibana-alert-action-settings.yml`) that is
manually converted to Asciidoc format
(`kibana-alert-action-settings.asciidoc`) by means of a Perl script
(`docs/settings-gen/parse-settings.pl`). A preview of the new, generated
page is
[here](https://kibana_bk_191787.docs-preview.app.elstc.co/guide/en/kibana/master/alert-action-settings-kb.html).
- Adds the `docs/settings-gen/parse-settings.pl` script which does the
YAML → Asciidoc conversion.
All new files are added to the `/docs/source-gen` folder.
This is a trial run updating only one page of settings in the docs.
Later, in separate PRs, we plan to convert other pages. After all Kibana
settings pages have been converted, we would ask that the Perl script be
run automatically as part of the CI whenever the YAML files in
`/docs/source-gen` are added or updated.
**Notes:**
- The Docs team is happy to own and maintain the Perl script (sorry to
use Perl - it's the only scripting language that I know).
- In time we also plan to convert all of these files from Asciidoc to
Markdown.
- When we eventually/hopefully get the rest of the Kibana settings files
converted, we will announce the settings doc process to the Kibana team
by email and/or in the Kibana newsletter.
Big thanks to the amazing @lukeelmers and @KOTungseth for guiding this!
---
Why are we doing this? We aim to:
- Create a more consistent appearance for settings across all of the
docs.
- Make it easier for people to contribute, since all Asciidoc/Markdown
formatting is handled by a script.
- Make it more apparent which settings may be missing info, such as the
default values, available options, etc.
---
P.S. I haven't worked in the Kibana repo very much and would appreciate
any help navigating the CI checks.
Rel: https://github.com/elastic/docs-projects/issues/239
## Summary
So that it can be used from `streams_app` (o11y).
See https://github.com/elastic/kibana/pull/204309
Steps to relocate:
1. Fetch latest `main`
2. Update the `group` and/or `visibility` in the module's manifest.
3. Run `node scripts/relocate --moveOnly
@kbn/saved-objects-tagging-plugin`
## Summary
This PR replaces the `useIsDarkTheme()` hook with EUI's `colorMode`.
Since this was the last hook in `@kbn/ml-kibana-theme` left this removes
the whole package too.
Note that the hook subscribed to an observable and was able to update
the theme in place. EUI's `colorMode` will only be updated after a page
refresh. Since updating the Kibana advanced setting to enable dark mode
requires a full page refresh too, I guess it's fair to remove this
behavior in favor of this simplification.
In the long run we should aim for getting rid of these checks altogether
and rely on dark-mode-aware EUI tokens.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
