## Summary
The intent is to have a centralised place to store the list of Kibana
solutions and serverless project types.
To that end, this PR creates a `@kbn/projects-solutions-groups` package.
It also adds the new solution type `'chat'`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR reworks how APM handles getting its sources data, elevating the
necessary code to a private shared plugin so that Discover for Traces
can access the data and handle user provided configuration. It also
removes the need for Discover for Traces to rely on the APM static data
view, so the Trace data source and document profile will work on any
compatible/configured index, even in ESQL mode.
Closes#211414
<img alt="ESQL Discover Traces Screenshot 2025-03-04 173032"
src="https://github.com/user-attachments/assets/f5bbb736-8b8b-45dc-ac23-4bf7083aa47e"
/>
## How to test
Test with olbt-cli instance for now, will post for doing with synthtrace
data. Ensure the following is added to your kibana.dev.yml:
```yaml
discover.experimental.enabledProfiles:
- observability-traces-data-source-profile
```
- Make sure your space has the Observability solution view configured
- Go to Discover page
- Select Data Views mode if required and create a view with a `traces`
specific index. Or use the APM static data view.
- The default columns on the page should show the summary column with
four of the following badges: `service.name`, `event.outcome`,
`transaction.name`, `transaction.duration.us`, `span.name`,
`span.duration.us`
- Go to ESQL mode with the query targetting a `traces` index
- The default columns should show the same as in Data View mode
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Irene Blanco <irene.blanco@elastic.co>
## Summary
This PR establishes the baseline to execute Cypress tests in the new
`search_ai_lake` tier.
## Changes Introduced
- All tests under
`x-pack/test/security_solution_cypress/cypress/e2e/ai4dsoc` will be
executed using the new tier by default.
- These tests will run as part of the PR process within the `Serverless
AI4DSOC - Security Solution Cypress Tests` execution.
## Adding a New Test
To add a new test, follow these guidelines:
- Read the
[README](x-pack/test/security_solution_cypress/cypress/e2e/ai4dsoc/README.md).
- Inside the `AI4DSOC` folder, we should have different subfolders
representing the various AI4DSOC functionalities.
- Each subfolder should have ownership by either an area team or the
developers actively working on it.
- Make sure that any functionality you want to be tested in the new tier
is added inside the `AI4DSOC` folder; otherwise, that functionality will
be tested using the complete tier.
## Running Tests Locally
Run the tests with the following Yarn scripts from
`x-pack/test/security_solution_cypress`:
```sh
yarn cypress:open:ai4dsoc:serverless
```
Opens the Cypress UI with all tests in the `e2e/ai4dsoc` directory. This
also runs a mocked serverless environment using the `ai_soc` product
line and `search_ai_lake` tier by default.
```sh
yarn cypress:run:ai4dsoc:serverless
```
Runs all tests tagged as @serverless in the e2e/ai4dsoc directory in
headless mode using the ai_soc product line and search_ai_lake tier by
default.
## Key Considerations
- All tests must have the `@serverless` tag to be executed as part of
the PR process.
- MKI is not yet supported for test execution.
- The AI4DSOC Cypress tests will be executed each time there is a change
in one of its
[dependencies](https://github.com/elastic/kibana/blob/main/.buildkite/scripts/pipelines/pull_request/pipeline.ts).
- All tests are executed by default using the `platform_engineer` role.
- Temporary Ownership: The Security Engineering Productivity team will
own the entire AI4DSOC testing folder initially to ensure structure and
best practices. Once all teams understand the workflow, this ownership
will be removed.
- Execution Time: If test execution in a PR takes more than 45 minutes,
parallelism should be increased in the new
`.buildkite/pipelines/pull_request/security_solution/ai4dsoc.yml` file.
## Security Engineering Productivity Codeownership Responsibilities
The Security Engineering Productivity team should ensure:
- Best practices are followed.
- All tests are placed inside a functionality subfolder.
- Each functionality subfolder has designated code owners.
- Tests include the `@serverless` label.
- The execution of AI4DSOC tests does not exceed 45 minutes.
## Follow-Up Tasks
- Remove the dummy test (@tomsonpl feel free to delete it when you need
to add new tests to the navigation).
- Integrate tests into the periodic pipeline.
- Add tests to the Kibana QA quality gate.
- Update the README with MKI instructions once tests are added to the
periodic pipeline and Kibana QA quality gate.
- Clarify which roles will be used for the AI4DSOC effort and update the
tests accordingly.
## Summary
Closes https://github.com/elastic/streams-program/issues/170 and
https://github.com/elastic/streams-program/issues/171
## Reviewer notes
It's worth reading the research in
https://github.com/elastic/streams-program/issues/168 to understand why
certain decisions were made. Some of the resolving code is inspired by
`grok-js`, unfortunately this wasn't something we could use directly.
The design / UX is **not** final. This is just a functional version. The
editor is currently on the streams > enrichment page so it can be played
with, this will **not** be merged.
There is pattern support for our ECS patterns, I have not added the
legacy ones.
There will almost certainly be some edge cases that don't work, every
repo for a Grok tool I looked at had their own 😅 I've tried to test this
with lots of varied examples though.
I'd recommend unticking "No extension" from the file filter to remove
the pattern files.
<img width="323" alt="Screenshot 2025-03-06 at 22 40 19"
src="https://github.com/user-attachments/assets/5b594f5f-0b0f-4ed0-ae10-2412fcf9e31a"
/>
## Possible improvements / followups
- We could in the future expand the UI to toggle on and off certain
pattern collections.
- Ability to add custom patterns (like in our Grok debugger). This can
still be achieved with `(?<queue_id>[0-9A-F]{10,11})` syntax.
- Point out when regex is invalid (right now it's silent).
- I've copied over the patterns from the ES repo for now, with a light
script to generate an object from them. There's a CLI skeleton in place
if we feel we want to actually pull these from the ES repo directly.
These patterns don't change often, and aren't heavy size wise.
- Debouncing etc for processing, but I'd like to see how the final UX
ends up.
## Media
Screenshot
Simple example
Complex example
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
More teams are adding Scout tests in their plugins, often as a PoC and
not stable yet for continuous execution.
We don't want to block it, but need a way to manage the scope of Scout
pipeline and be able to disable it quickly to unblock the Scout
development.
Since Scout is in active development and we need it to be simple and
quick as possible (we can iterate and improve later), we agreed with
Robert to disable tests by plugin:
```
ui_tests:
enabled:
- apm
- discover_enhanced
- maps
- observability_onboarding
disabled:
- *skipped_plugin*
```
When scout configuration is added to the new plugin, it will require to
update `.buildkite/scout_ci_config.yml` that is owned by `appex-qa`
team. If there is no intention to run Scout tests on CI, plugin name
should be added under `disabled` section.
**How to test locally:**
- Scout tests were added in `observability_onboarding` plugin, pipeline
will throw error
modify locally `.buildkite/scout_ci_config.yml`
```
ui_tests:
enabled:
- apm
- discover_enhanced
- maps
disabled:
```
run `node scripts/scout discover-playwright-configs --validate --save`
```
ERROR The following plugins are not registered in Scout CI config '.buildkite/scout_ci_config.yml'
- observability_onboarding
```
~~On CI annotation will be added to clarify the failure:~~
we decided to move validation to "Quick Checks", no need to annotate.
<img width="1583" alt="image"
src="https://github.com/user-attachments/assets/ed6b5778-74cb-4473-8218-b96239aab067"
/>
- `observability_onboarding` plugin is disabled, pipeline won't include
it (excluded in `scout_playwright_configs.json`)
modify locally `.buildkite/scout_ci_config.yml`
```
ui_tests:
enabled:
- apm
- discover_enhanced
- maps
disabled:
- observability_onboarding
```
run `node scripts/scout discover-playwright-configs --validate --save`
```
warn The following plugins are disabled in '.buildkite/scout_ci_config.yml' and will be excluded from CI run
- observability_onboarding
info Found Playwright config files in '4' plugins.
Saved '3' plugins to '/Users/dmle/github/kibana/.scout/test_configs/scout_playwright_configs.json'
```
Exposes an Inference (plugin) API client for scripts, that mimicks the
`chatComplete` and `output` APIs that are available on its start
contract. It depends on the KibanaClient that is exposed from the
`@kbn/kibana-api-cli` package. It automatically selects a connector if
available.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Changing code owners as discussed with @thomasneirynck and @davismcphee.
I also bumped CPU time limit as it fails few times passing over `1.55`
sec.
## Summary
Adds a basic api for finding suggested dashboards by alert.
This internal API is currently not in use anywhere.
This api finds suggested dashboards in two ways:
1. Finding dashboards with lens visualizations that query against the
same data view
2. Finding dashboards with lens visualizations that utilize fields in
the rule configuration, or alert data.
These are two naive approaches to finding suggested dashboards. These
heuristics will be improved over time and incorporate more sophisticated
approaches that have been explored by numerous engineers across
Observability.
## Testing
A basic api integration test is included that covers matching dashboards
by index and field, across spaces. As we develop this feature further
over time more fine grain tests will be added
### Manual Testing
1. Run
```
node x-pack/scripts/data_forge.js --events-per-cycle 200 --lookback now-1h --ephemeral-project-ids 10 --dataset fake_stack --install-kibana-assets --kibana-url http://localhost:5601 --event-template bad
```
2. Save the file as a .ndjson file and import it via saved objects
import
https://p.elstc.co/paste/3BWKIHLU#f0WfGYx7G9DhWy88yDXhaEXTX16Fn+ovDcomNIx6E3a
3. Navigate to the alert details page, you should start to see a lot of
alerts
4. Click on one, and copy the alert id
5. Navigate to console and paste `GET
kbn:/internal/observability/alerts/suggested_dashboards?alertId=[YOUR_ALERT_ID]`
to see the recommended dashboards
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Nick Peihl <nickpeihl@gmail.com>
## Summary
Closes https://github.com/elastic/kibana/issues/213216
This PR adds the functionality to properly highlight filtered values
within badges.
Previously, the content was treated as `text` instead of `html`, which
prevented the highlighted values from being displayed correctly.
The content is now rendered with the `<mark>` tag, allowing matching
values to be properly highlighted within the badges.
>[!NOTE]
>By looking at the code I assumed the `<mark>` tag is the only one we
introduce, so the proposed solution only handles that.
|Before|After|
|-|-|
|||
### How to test
- Make sure you are in a space with Observability as solution view
- Select the "All logs" data view
- Add any filter that matches the displayed badges value
## Summary
Depends on #191106Closes#171591
This PR migrates Storybook from `6.x` to `8.x`. Please see the
[migration
guide](https://storybook.js.org/docs/migration-guide/from-older-version)
for an overview of the changes because there are many breaking changes
which effect Kibana. The TODO list below is not inclusive of all the
changes.
## Reviewers
### Each commit contains all files changed for a specific codeowner,
please find your respective commit to make review easier.
A **first step before code review** should be checking the [`Storybooks
Preview`](https://ci-artifacts.kibana.dev/storybooks/pr-195148/index.html)
from CI for any runtime or style issues which were missed. The preview
can be compared to a build from `main`
[here](https://ci-artifacts.kibana.dev/storybooks/pr-212585/index.html).
It is worth noting that some stories have runtime issues which existed
before this migration.
Most stories appear to have been migrated properly, but the Operations
team does not have prior knowledge into every story. Some of the
migration was able to be automated through Storybook provided scripts.
It is possible this wasn't entirely correct due to the structure of some
stories. Additionally, part of this migration is moving Storybook to
Webpack 5 which changed how styles are being loaded.
#### TODO
- [x] Migrate `stories.mdx`
- [x] storyshots
- [x] [Migrate
packages](https://storybook.js.org/docs/migration-guide/from-older-version#package-structure-changes)
which were removed in `8.0`
- [x] `react-doc-gen` resolution
- [x] [Migrate
blocks](https://github.com/storybookjs/storybook/blob/next/MIGRATION.md#addon-docs-removed-deprecated-blocksjs-entry)
- [x] Migrate deprecated `addon-knobs` to
[addon-controls](https://www.npmjs.com/package/@storybook/addon-controls)
- [x] React Fast Refresh support
- [x] Watch flag callback
- [x] `canvas` webpack
- [x] Rerun CSF migrations for new stories
- [x] Handle ESM import for `addon-docs`
- [x] `'@storybook/addon-actions' should be listed in the project's
dependencies. Run 'npm i -S @storybook/addon-actions' to add
iteslint[import/no-extraneous-dependencies](https://github.com/import-js/eslint-plugin-import/blob/v2.28.0/docs/rules/no-extraneous-dependencies.md)`
- [x] `addDecorator` migration
- [x] `addParameter` migration
- [x] static build
- [ ] determine if #176500 is solved or push to followup PR
- This will need to be fixed separately
- [x] revert `.buildkite/pipelines/pull_request/base.yml` &
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to `main`
---------
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jacek Kolezynski <jacek.kolezynski@elastic.co>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
## Summary
Adds `@elastic/platform-analytics` as codeowners of the telemetry
schemas to raise awareness of any changes implemented in the snapshot
telemetry.
The end goal is to remove Kibana Core from owning them.
Closes#197953
## Summary
This PR replaces the alert status filter with filter controls. In this
PR, I also covered backward compatibility when we have a `status` URL
parameter by passing that value to filters, as shown below:
|State|Screenshot|
|---|---|
|Before||
|After||
https://github.com/user-attachments/assets/86e82a19-f68e-4127-9fd8-e0efe0d41ece
I checked in Serverless and we have access to controls in viewer mode as
well:
https://github.com/user-attachments/assets/2c90ba3a-7d95-4682-b722-e5b327f7334d
### 🐞 Known issue
1. Privilege
In Stateful, if a user has Kibana privilege but not the `.alert*` es
privilege, then the controls do not work as expected. This issue will be
tackled in a separate ticket:
<details>
<summary>This is the error that we show in this scenario</summary>
</details>
2. Initial load
Related ticket: https://github.com/elastic/kibana/issues/183412
### 🗒️ Tasks
- [x] ~~Solving the permission issue~~ This issue does not happen in
Serverless and for stateful, we will fix it in a separate ticket:
https://github.com/elastic/kibana/issues/208225
- The main issue will be fixed in this
[PR](https://github.com/elastic/kibana/pull/191110)
- In the above [PR](https://github.com/elastic/kibana/pull/191110), we
remove controls if the user does not have the privilege for alert
indices, but we need to figure out how to adjust filter controls to
access the data based on Kibana privileges.
- [x] We should configure the filters to allow the selection of one item
for alert status but still show the other options
- [x] We need to see how we can make this work with the current status
field. Ideally, if there is a status field, we would apply it and remove
it from the URL.
- Fixed in
[c6cad2d](c6cad2dbe1)
- [x] Changing the URL does not update the page filters correctly. ~~It
might be related to https://github.com/elastic/kibana/issues/183412.~~
- [x] We need to make sure these adjustments work as expected in APM as
they use the observability alert search bar.
- [x] Check if the tags filter can be improved, and if not, whether it
makes sense to keep it in its current form.
- It works based on how array filtering works in ES, which seems like a
good start to me.
- [x] Check with Maciej: Do we need to disable changing control configs?
- Checked with Maciej: it is fine to keep the option of editing
controls.
- [x] Do we need to have a different local storage item for each page
(apm/rule details/alert details/alerts)?
- How can we disable syncing with the local storage?
- Added the possibility of disabling sync in
[24bab21](24bab210b0)
and disabled it for the rule details and alert details pages.
- Also, disabled it for the APM alert search bar.
- [x] Setting default status as active on the related alerts tab
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Shahzad <shahzad31comp@gmail.com>
Replaces #213441
## Summary
@smith and I have agreed that it makes sense for the infra services team
to own the observability overview page within the obs plugin.
First PR didn't add the changes to the correct "overrides" section of
the CODEOWNERS file.
Co-authored-by: Maryam Saeidi <maryam.saeidi@elastic.co>
## Summary
Closes - https://github.com/elastic/kibana/issues/166679
## What's included ?
- The PR adds a feature in Logs View of Observability (to start with) to
hide the regular pagination toolbar from the footer and show Load More
only when the user has scrolled to the bottom of the page.
- The table would always load the items in batches of default set 500
- This PR also add 2 helper functions `useThrottleFn` and
`useDebounceFn`. Current React help library which KIbana uses called
-`react-use` does not have these and we cannot use Lodash variant of
these. We need such hooks which are React safe. Hence added these 2
## What's pending ?
- [x] Unit tests for the 2 new helper React hooks
- [x] Unit tests for data table footer component
- [x] Unit tests for Profile Resolution
- [x] Functional Serverless Tests
- [x] Functional Stateful Tests
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
Download, parse and replay loghub data with Synthtrace, for use in the
Streams project. In summary:
- adds a `@kbn/sample-log-parser` package which parses Loghub sample
data, creates valid parsers for extracting and replacing timestamps,
using the LLM
- add a `sample_logs` scenario which uses the parsed data sets to replay
Loghub data continuously as if it were live data
- refactor some parts of Synthtrace (follow-up work captured in
https://github.com/elastic/kibana/issues/212179)
## Synthtrace changes
- Replace custom Logger object with Kibana-standard ToolingLog
- Report progress and estimated time to completion for long-running jobs
- Simplify scenarioOpts (allow comma-separated key-value pairs instead
of just JSON)
- Simplify client initialization
- When using workers, only bootstrap once (in the main thread)
- Allow workers to gracefully shutdown
- Downgrade some logging levels for less noise
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves#212919
We noticed that setting the header `'Content-Type':
'text/event-stream',` didn't work as the browser's native EventSource
implementation.
```JS
return res.ok({
headers: {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
},
body: observableIntoEventSourceStream(events$ as unknown as Observable<ServerSentEvent>, {
signal: abortController.signal,
logger,
}),
});
```
The reason, apparently, is that we need to flush the compressor's buffer
negotiated in the HTTP request.
### How to test it:
Run Kibana with examples `yarn start --no-base-path --run-examples
--http2` and open the SSE example app in Kibana. You should see a clock
updating every second in the UI (the clock is coming from the server).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/211778 and
https://github.com/elastic/kibana/issues/211871
This PR is a follow-up to issue
[#208693](https://github.com/elastic/kibana/issues/208693) and
incorporates the suggestions from its associated PR
[#210612](https://github.com/elastic/kibana/pull/210612).
>[!NOTE]
> Please note that most of the changes in this PR involve moving files,
renaming `data-test-subj` attributes, updating imports, and not creating
the files from scratch.
**What's happening in this PR?**
- Transitioning from data source profile to document profile.
- Splitting the traces document viewer into separate span and
transaction viewers.
- Enhancing the `useTransaction` hook to abort requests on component
unmount.
As noted in the `TODOs`, a follow-up issue will focus on using for
APM-configured indexes to enable the experience and retrieve the
`transaction.name` for spans.
### How to test
Add this to your kibana.dev.yml:
```
discover.experimental.enabledProfiles:
- observability-traces-span-document-profile
- observability-traces-transaction-document-profile
`````
- Go to Discover page
- Select APM data view
- Check your traces data
- Spans and transactions should show the new overview tabs when opening
the flyout
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com>
Co-authored-by: Carlos Crespo <crespocarlos@users.noreply.github.com>
## Summary
Adds serverless security AI4SOC add-on metering.
- emits a ai4soc metering usage record with the corresponding tier every
~20 minutes
- records are unique to the project + hour
- records are backfilled up to a week
- adds usage record backfill functionality to the shared usage reporting
task
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Unified Search belongs to presentations team so I'm changing the
codeowners entry. There was a [previous
attempt](https://github.com/elastic/kibana/pull/208381/files) of doing
the same thing, but the auto-commit kicked in and reverted the change
for unified_search plugin, since we forgot to add the entry
in`kibana.jsonc`.
## Summary
This is intended to be a "minimal" migration for Defend Insights to
langgraph + output chunking. Other than the increased events due to the
context increase from output chunking, the functionality is unchanged.
* migrates defend insights to langgraph
* adds output chunking / refinement
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
The esql types are around many packages and plugins creating problems
either with the bundle size or cyclic dependencies.
This PR is moving some of them on the esql-types package. I am going to
move more there in follow up PRs.
(I decided to rename it as it makes more sense to have a generic one for
esql rather one only for the variables)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Adds the new docs CI checks. This will enable a preview to build our new
Markdown-based docs.
@florent-leborgne please tag reviewers, add the appropriate label(s),
and take this out of draft when you're ready.
---------
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
As a first step towards making parts of streams owned by different teams
working on it, this PR gives some of the things to the logs UX team.
These routes and views are clearly associated with the workstreams the
logs UX team moves forward. It's still expected that a lot of changes
(but probably not all) will have at least a portion outside of this code
area, but it's meant to put a starting point into place to make the
separation stronger over time.
For the API it's just about certain route handlers. For the UI, the
relevant components are pulled into a separate directory to encapsulate
them somewhat.
---------
Co-authored-by: Milton Hultgren <miltonhultgren@gmail.com>
**Fixes: https://github.com/elastic/kibana/issues/206527**
**Partially addresses: https://github.com/elastic/kibana/issues/209518**
## Summary
Adds a normalization to the `filters` field in the rule diffing
calculation that omits all filter fields other than the `query` field
and the `negate` and `disabled` fields within the `meta` object. This
makes our diffing logic much more robust and resilient as we only
compare data in the rule fields that have an impact on the query itself
and not the fields that relate to UI implementation (`alias`, `key`,
etc).
### To test
- Open a prebuilt rule with `filters` in the non-customized rule
parameters (e.g. `PowerShell Script with Discovery Capabilities`)
- Edit the rule and save without editing
- The rule should remain unmodified even though more fields have been
added to the rule's `filters` field
Unless the user adds or deletes a filter on the rule, the rule should
only be marked as customized under 3 circumstances:
- The user negates the filter (adds NOT to the beginning of the filter)
- The user disables the filter
- The user changes the filter query
All other scenarios (such as adding a custom name for the filter) should
not change the rule's customized status
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
* Remove some old paths pointing to `packages/kbn-pm` (no longer
exists).
* ~Fix group and visibility for `@kbn/streams-app-wrapper-plugin`~.
(done in https://github.com/elastic/kibana/pull/212210)
* Update `scripts/relocate` logic with latest enhancements.
* Convert `@kbn/observability-synthetics-test-data` folder name to
camel-case (messes up with pre-commit hook).
## Summary
The `/packages` folder at the root of the Kibana repository used to
contain a lot of packages.
In the context of SKA, they have been gradually moved to various
locations:
* `src/platform/packages`
* `x-pack/platform/packages`
* `src/core/packages`
Currently, only `devOnly: true` packages are left in this folder. This
comprises libraries for CLI scripts as well as testing utilities.
With this PR, we are moving ~half of these packages under
`src/platform/packages/(private|shared)/`.
In particular, we are moving those packages that are being used from
platform and/or solutions.
Since they are `"devOnly": true`, this means they are ONLY used from
tests, cypress tests, storybook configs, ./scripts/ folders inside some
modules, or other non-prod-time logic. Nonetheless, they are effectively
referenced from platform and/or solutions code, hence I decided they
should be placed under `platform` folders.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR moves the `streams` and `streams_app` plugins into platform so
they can be used in other solutions in the future. This PR is not
actually making it available in other solutions yet since we are still
discussing the release plans.
## Inlined helpers
As discussed before, this PR inlines a couple simple helper methods for
query building, time zone normalization, a header portal helper and a
data plugin timefilter state react integration hook as there is no good
place for these outside of the observability solution.
## streams_app plugin
The streams_app plugin is not actually registering anything, instead it
simply exports a component that renders the app which needs to be
consumed by another plugin to turn it into a registered app - for now,
`observability_streams_wrapper` takes over this job.
## observability_streams_wrapper plugin
While 99% of the streams logic is moved into the
`platform/shared/streams_app`, two bits are left behind in
`observability_streams_wrapper`:
* The actual app registration
* Integration with the observability_shared `PageTemplate` component
Once we decide streams should be displayed outside of the observability
solution, it's probably not necessary anymore to decouple app definition
and registration like this because it will always be visible no matter
the solution. Once this is the case, the navigation registration can be
moved into the central `observability` plugin, like it's handled with
other apps like infra.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
These where the only packages under `src/dev/packages`.
The goal long term is to refactor the code under `src/dev` and to
migrate it to one (or multiple) packages.
Thus, we're moving them to `src/core/test-helpers/`, which already
exists and contains a bunch of test-helpers.
#### 2 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/core-test-helpers-kbn-server` |
`src/core/test-helpers/kbn-server` |
| `@kbn/core-test-helpers-model-versions` |
`src/core/test-helpers/model-versions` |
<details >
<summary>Updated references</summary>
```
./.github/codeql/codeql-config.yml
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/core/system/tsconfig.type_check.json
./src/core/test-helpers/kbn-server/jest.config.js
./src/core/test-helpers/model-versions/jest.config.js
./src/dev/tsconfig.type_check.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./src/plugins/kibana_usage_collection/tsconfig.type_check.json
./src/plugins/usage_collection/tsconfig.type_check.json
./tsconfig.base.json
./tsconfig.base.type_check.json
./tsconfig.refs.json
./x-pack/platform/plugins/shared/fleet/tsconfig.type_check.json
./x-pack/platform/plugins/shared/spaces/tsconfig.type_check.json
./x-pack/plugins/actions/tsconfig.type_check.json
./x-pack/plugins/alerting/tsconfig.type_check.json
./x-pack/plugins/task_manager/tsconfig.type_check.json
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/core/test-helpers/kbn-server/jest.config.js:12
src/core/test-helpers/kbn-server/tsconfig.json:2
src/core/test-helpers/model-versions/jest.config.js:12
src/core/test-helpers/model-versions/tsconfig.json:2
```
</details>
The storage adapter helper is a very generic package. This PR moves it
out of the observability server utils into a dedicated package to better
reflect this and to be able to use it from non-observability contexts.
The same applies to the observability es client. This PR moves it as
well and renames it to `TracedEsClient` in the same way.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 3 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/securitysolution-data-table` |
`x-pack/solutions/security/packages/data-table` |
| `@kbn/ecs-data-quality-dashboard` |
`x-pack/solutions/security/packages/ecs-data-quality-dashboard` |
| `@kbn/security-solution-side-nav` |
`x-pack/solutions/security/packages/side-nav` |
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./tsconfig.base.json
./tsconfig.base.type_check.json
./tsconfig.refs.json
./x-pack/solutions/security/packages/data-table/jest.config.js
./x-pack/solutions/security/packages/ecs-data-quality-dashboard/jest.config.js
./x-pack/solutions/security/packages/side-nav/jest.config.js
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
x-pack/solutions/security/packages/data-table/jest.config.js:11
x-pack/solutions/security/packages/data-table/tsconfig.json:2
x-pack/solutions/security/packages/ecs-data-quality-dashboard/jest.config.js:24
x-pack/solutions/security/packages/ecs-data-quality-dashboard/tsconfig.json:10
x-pack/solutions/security/packages/ecs-data-quality-dashboard/tsconfig.json:2
x-pack/solutions/security/packages/side-nav/jest.config.js:10
x-pack/solutions/security/packages/side-nav/tsconfig.json:2
```
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
As part of the SKA, some groups of modules have been broken-down and
relocated in different folders, negatively impacting DX. This is the
case for `serverless` modules, that are spread across:
```
- packages/serverless # storybook config
- src/platform/packages/private/serverless
- src/platform/packages/shared/serverless
```
The goal of this PR is to unify them back under
`src/platform/packages/shared/severless`.
In this scenario, I believe the simplification is worth the trade off.
CC @elastic/kibana-management @elastic/appex-sharedux WDYT?
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
As part of the SKA, some groups of modules have been broken-down and
relocated in different folders, negatively impacting DX. This is the
case for `kbn-management` modules, that are spread across:
```
- packages/kbn-management (storybook config)
- src/platform/packages/private/kbn-management
- src/platform/packages/shared/kbn-management
```
The goal of this PR is to unify them back under
`src/platform/packages/shared/kbn-management`.
In this scenario, I believe the simplification is worth the trade off.
CC @elastic/kibana-management WDYT?
Co-authored-by: Matthew Kime <matt@mattki.me>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 4 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/alerting-rule-utils` |
`x-pack/platform/packages/shared/alerting-rule-utils` |
| `@kbn/observability-alert-details` |
`x-pack/solutions/observability/packages/alert-details` |
| `@kbn/observability-alerting-test-data` |
`x-pack/solutions/observability/packages/alerting-test-data` |
| `@kbn/observability-get-padded-alert-time-range-util` |
`x-pack/solutions/observability/packages/get-padded-alert-time-range-util`
|
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./tsconfig.base.json
./x-pack/platform/packages/shared/alerting-rule-utils/jest.config.js
./x-pack/solutions/observability/packages/alert-details/jest.config.js
./x-pack/solutions/observability/packages/alerting-test-data/jest.config.js
./x-pack/solutions/observability/packages/get-padded-alert-time-range-util/jest.config.js
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
x-pack/platform/packages/shared/alerting-rule-utils/jest.config.js:10
x-pack/platform/packages/shared/alerting-rule-utils/tsconfig.json:2
x-pack/solutions/observability/packages/alert-details/jest.config.js:10
x-pack/solutions/observability/packages/alert-details/tsconfig.json:2
x-pack/solutions/observability/packages/alerting-test-data/jest.config.js:10
x-pack/solutions/observability/packages/alerting-test-data/tsconfig.json:2
x-pack/solutions/observability/packages/get-padded-alert-time-range-util/jest.config.js:10
x-pack/solutions/observability/packages/get-padded-alert-time-range-util/tsconfig.json:2
```
</details>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 3 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/response-ops-alerts-apis` |
`src/platform/packages/shared/response-ops/alerts-apis` |
| `@kbn/response-ops-alerts-fields-browser` |
`src/platform/packages/shared/response-ops/alerts-fields-browser` |
| `@kbn/response-ops-alerts-table` |
`src/platform/packages/shared/response-ops/alerts-table` |
<details >
<summary>Updated references</summary>
```
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./src/platform/packages/shared/response-ops/alerts-apis/jest.config.js
./src/platform/packages/shared/response-ops/alerts-fields-browser/jest.config.js
./src/platform/packages/shared/response-ops/alerts-table/jest.config.js
./tsconfig.base.json
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/platform/packages/shared/response-ops/alerts-apis/jest.config.js:12
src/platform/packages/shared/response-ops/alerts-apis/tsconfig.json:2
src/platform/packages/shared/response-ops/alerts-fields-browser/jest.config.js:12
src/platform/packages/shared/response-ops/alerts-fields-browser/tsconfig.json:2
src/platform/packages/shared/response-ops/alerts-table/jest.config.js:12
src/platform/packages/shared/response-ops/alerts-table/tsconfig.json:2
```
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Umberto Pepato <umbopepato@users.noreply.github.com>
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 1 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/logs-overview` | `x-pack/platform/packages/shared/logs-overview`
|
<details >
<summary>Updated references</summary>
```
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./tsconfig.base.json
./x-pack/.i18nrc.json
./x-pack/platform/packages/shared/logs-overview/jest.config.js
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
x-pack/platform/packages/shared/logs-overview/jest.config.js:10
x-pack/platform/packages/shared/logs-overview/tsconfig.json:2
```
</details>
