## Summary
Remove run soon for sync private location task !!
It's not needed since it's a scheduled task running every 10 minutes .
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes#222112
## Summary
I used the Machine Learning section as a guide because it has a similar
mix of "items" and "groups" but is presented correctly in the UI, so by
making the "apm" section look the same, it is fixed.
<img width="496" alt="Screenshot 2025-05-30 at 4 08 24 PM"
src="https://github.com/user-attachments/assets/e6a9d659-df58-4ec5-bf2d-2584fa2fe0d2"
/>
## Context
Machine Learning nav:
<img width="515" alt="Screenshot 2025-05-30 at 3 48 17 PM"
src="https://github.com/user-attachments/assets/25b7c1e4-8a99-43a4-be75-3299bd5412bd"
/>
| Top-level Group | Items | Sub-groups |
| :-- | :-- | :-- |
| Machine Learning | Overview, Data Visualizer | Anomaly Detection, Data
Frame Analytics, AIOps Labs |
| Applications | Service Inventory, Traces, Dependencies, Settings |
Synthetics |
I noticed the structure of these two sections was slightly different.
The machine learning one looks like this (i18n functions removed for
readability):
```ts
{
id: 'machine_learning-landing',
renderAs: 'panelOpener',
title: 'Machine Learning',
children: [
{
children: [
{ link: 'ml:overview' },
{ link: 'ml:dataVisualizer' },
],
},
{
id: 'category-anomaly_detection',
title: 'Anomaly detection',
breadcrumbStatus: 'hidden',
children: [
{ link: 'ml:anomalyExplorer' },
{ link: 'ml:singleMetricViewer' },
],
},
]
}
```
whereas "Applications" included the synthetics sub-section further
nested in, as a sibling to the other items:
```ts
{
id: 'apm',
title: 'Applications',
renderAs: 'panelOpener',
children: [
{
children: [
{
link: 'apm:services',
title: 'Service Inventory',
},
{ link: 'apm:traces' },
{ link: 'apm:dependencies' },
{ link: 'apm:settings' },
{
id: 'synthetics',
title: 'Synthetics',
children: [
{
title: 'Overview',
id: 'synthetics-overview',
link: 'synthetics:overview',
breadcrumbStatus: 'hidden',
},
```
When I moved the Synthetics sub-section out of the children > children
section and into children, the nav rendered correctly.
I'm not sure if this is expected behavior, [there is some logic to
confirm](0fcc9a39ba/src/platform/packages/shared/shared-ux/chrome/navigation/src/ui/components/panel/default_content.tsx (L75))
sections are "all items" OR "all groups" but ours seemed to be a mix,
but didn't fail.
Currently we use a search with a `top_hits` aggregation to get the
alerts filtered with the scoped queries of the active maintenance
windows. But since `top_hits` aggregation has a fixed limit of 100 hits
we cannot use our max alerts limit to get all the alerts an execution
can generate.
This PR replaces `search` with `msearch` and removes the aggregation
from the search query so we can use the maxAlerts limit for the response
size.
Resolves https://github.com/elastic/kibana/issues/208459
## Summary
Unskips flaky test that was failing with the error below. This PR
updates the test to wait until the task is finished running before
checking that the status is marked as `unrecognized`.
```
Error: expect(received).toBe(expected) // Object.is equality
Expected: "unrecognized"
Received: "idle"
at toBe (/opt/buildkite-agent/
```
Closes https://github.com/elastic/kibana/issues/1547
Closes https://github.com/elastic/kibana/issues/190342
Closes https://github.com/elastic/kibana/issues/197716
## Summary
This PR adds the ability for collapsible sections to be created and
managed on Dashboards.
https://github.com/user-attachments/assets/c5c046d0-58f1-45e1-88b3-33421f3ec002
> [!NOTE]
> Most of the work for developing collapsible sections occurred in PRs
contained to the `kbn-grid-layout` package (see [this meta
issue](https://github.com/elastic/kibana/issues/190342) to track this
work) - this PR simply makes them available on Dashboards by adding them
as a widget that can be added through the "Add panel" menu. As a result
of this, most work is contained in the Dashboard plugin - changes made
to the `kbn-grid-layout` package only include adding IDs for additional
tests that were added for the Dashboard integration.
### Technical Details
#### Content Management Schema
The content management schema allows for panels and sections to be mixed
within the single `panels` key for a dashboard **without** worrying
about section IDs; for example:
```
{
"panels": [
{
// this is a simplified panel
"gridData": {
"x": 0,
"y": 0,
"w": 12,
"h": 8,
},
"panelConfig": { ... },
},
{
// this is a section
"gridData": {
"y": 9,
},
"collapsed": false,
"title": "Section title",
"panels": [
{
// this is a simplified panel
"gridData": {
"x": 0,
"y": 0,
"w": 24,
"h": 16,
},
"panelConfig": { ... },
},
],
},
]
}
```
#### Saved Object Schema
The dashboard saved object schema, on the other hand, separates out
sections and panels under different keys - this is because, while we are
stuck with panels being stored as `panelJSON`, I didn't want to add much
to this. So, under grid data for each panel, they have an optional
`sectionId` which then links to a section in the `sections` array in the
saved object:
```
{
"panelsJSON": "<...> \"gridData\":{\"i\":\"panelId\",\"y\":0,\"x\":0,\"w\":12,\"h\":8,\"sectionId\":\"someSectionId\"} <...>"
"sections": [
{
"collapsed": false,
"title": "Section title",
"gridData": {
"i": "someSectionId",
"y": 8.
}
}
],
}
```
This allows sections to be serialized **without** being stringified.
This storage also matches how we store this data in runtime using
`layout`.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Release note
Adds collapsible sections to Dashboard, which allow panels to grouped
into sections that will not load their contents when their assigned
section is collapsed.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Change are in support of Endpoint space awareness and most of them are
to Fleet services which are used from within the Security Solution
plugin.
### Fleet Changes
- Several modules were updated with `.debug()` statements in order to
better debug issues and understand code flow.
- Some modules where updated to ensure that calls to SO Client methods
handle errors better (using a new Error utility named
`catchAndWrapError`)
- `PackagePolicy` and `AgentPolicy` services `.get()` methods where
updated to internally call `getById()` in order to support retrieving
policies with an un-scoped SO clients (allows for use of `namespaces:
[]` when using SoClient.bulkGet()`)
### Security Solution Changes:
- Manifest Manager was updated to
- ensure it retrieves policies across all spaces
- Ensure a properly scoped SO client is used when making updates -
including ensuring that when calling bulk update type of methods, the
updates are first grouped by space ID and then each group is called with
an SO client scoped to that space.
- Fleet server extension point callbacks were updated to ensure an
un-scoped SO client is used when they are invoked, so that policies can
be found in any space
- Policy Watcher and Policy Telemetry Watcher (background tasks) were
updated to ensure they retrieve policies across all spaces and if
needing ot update policies, that is done using a properly scoped
SoCLient.
- Several `.debug()` statements were sprinkled through out in order to
better understand code flow and failures that are logged to the kibana
log
## Summary
Fixes#178571
This PR reverts #221248 adding the missing part on it to make the bug
found in #178571 disappear.
In fact there was just a single line missing who was meant to keep the
suggestion frame version up to date with the latest workspace state.
I've tried to add a unit test but it was so hard to actually follow the
code underneath, so I settled to add a FTR, which led to few more
extras:
* subtype selector now has test ids
* Lens page now has few more utility helpers for layer cloning and sub
type selector
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Gemini connector: makes `maxOutputTokens` an optional, configurable request parameter
This PR updates the Gemini connector to make the `maxOutputTokens` request parameter, which was previously included in all requests with a (non configurable) value of `8192`, an optional, configurable request parameter.
- `maxOutputTokens` is no longer included in requests to Gemini models by default
- It may be optionally included in requests by passing the new `maxOutputTokens` parameter to the Gemini connector's `invokeAI` and `invokeAIRaw`, and `invokeStream` APIs
The driver for this change was some requests to newer models, e.g. Gemini Pro 2.5 were failing, because previously all requests from the Gemini Connector were sent with the value `8192` [here](f3115c6746/x-pack/platform/plugins/shared/stack_connectors/server/connector_types/gemini/gemini.ts (L397)).
The previous, non-configurable limit was less than the output token limit of newer Gemini models, as illustrated by the following table:
| Model | Description |
|--------------------|------------------------------------|
| Gemini 1.5 Flash | 1 (inclusive) to 8193 (exclusive) |
| Gemini 1.5 Pro | 1 (inclusive) to 8193 (exclusive) |
| Gemini 1.5 Pro 002 | 1 (inclusive) to 32769 (exclusive) |
| Gemini 2.5 Pro | 1 (inclusive) to 65536 (exclusive) |
When newer Gemini models generated a response with more than `8192` output tokens, the API response from Gemini would:
- Not contain the generated output, (because the `maxOutputTokens` limit was reached)
- Include a finish reason of `MAX_TOKENS`,
- Fail response validation in the connector, because the `usageMetadata` response metadata was missing the required `candidatesTokenCount` property, per the _Details_ below.
#### Details
Attack discovery requests to Gemini 2.5 Pro were failing with errors like the following:
```
ActionsClientLlm: action result status is error: an error occurred while running the action - Response validation failed (Error: [usageMetadata.candidatesTokenCount]: expected value of type [number] but got [undefined])
```
A full example error trace appears in the _Details_ below (click to expand):
<details>
```
ActionsClientLlm: action result status is error: an error occurred while running the action - Response validation failed (Error: [usageMetadata.candidatesTokenCount]: expected value of type [number] but got [undefined])
Response validation failed (Error: [usageMetadata.candidatesTokenCount]: expected value of type [number] but got [undefined]): ActionsClientLlm: action result status is error: an error occurred while running the action - Response validation failed (Error: [usageMetadata.candidatesTokenCount]: expected value of type [number] but got [undefined])
at ActionsClientLlm._call (/Users/$USER/git/kibana/x-pack/platform/packages/shared/kbn-langchain/server/language_models/llm.ts:112:21)
at async ActionsClientLlm._generate (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:375:29)
at async ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:176:26)
at async ActionsClientLlm.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:34:24)
at async RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1291:27)
at async RunnableCallable.generate [as func] (/Users/$USER/git/kibana/x-pack/solutions/security/plugins/elastic_assistant/server/lib/attack_discovery/graphs/default_attack_discovery_graph/nodes/generate/index.ts:61:27)
at async RunnableCallable.invoke (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/utils.cjs:82:27)
at async RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1285:33)
at async _runWithRetry (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/retry.cjs:70:22)
at async PregelRunner._executeTasksWithRetry (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/runner.cjs:211:33)
at async PregelRunner.tick (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/runner.cjs:48:40)
at async CompiledStateGraph._runLoop (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/index.cjs:1018:17)
at async createAndRunLoop (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/index.cjs:907:17)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:176:37)
at ActionsClientLlm._generate (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:374:20)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:176:37)
at async ActionsClientLlm.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:34:24)
at async RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1291:27)
at async RunnableCallable.generate [as func] (/Users/$USER/git/kibana/x-pack/solutions/security/plugins/elastic_assistant/server/lib/attack_discovery/graphs/default_attack_discovery_graph/nodes/generate/index.ts:61:27)
at async RunnableCallable.invoke (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/utils.cjs:82:27)
at async RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1285:33)
at async _runWithRetry (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/retry.cjs:70:22)
at async PregelRunner._executeTasksWithRetry (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/runner.cjs:211:33)
at async PregelRunner.tick (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/runner.cjs:48:40)
at async CompiledStateGraph._runLoop (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/index.cjs:1018:17)
at async createAndRunLoop (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/index.cjs:907:17)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:142:51)
at CallbackManager.handleLLMStart (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/callbacks/manager.cjs:464:25)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:142:51)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:136:73)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:136:73)
at ActionsClientLlm._generateUncached (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:129:28)
at ActionsClientLlm.generate (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:281:25)
at ActionsClientLlm.generatePrompt (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:97:21)
at ActionsClientLlm.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/language_models/llms.cjs:34:35)
at RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1291:43)
at async RunnableCallable.generate [as func] (/Users/$USER/git/kibana/x-pack/solutions/security/plugins/elastic_assistant/server/lib/attack_discovery/graphs/default_attack_discovery_graph/nodes/generate/index.ts:61:27)
at async RunnableCallable.invoke (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/utils.cjs:82:27)
at async RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1285:33)
at async _runWithRetry (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/retry.cjs:70:22)
at async PregelRunner._executeTasksWithRetry (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/runner.cjs:211:33)
at async PregelRunner.tick (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/runner.cjs:48:40)
at async CompiledStateGraph._runLoop (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/index.cjs:1018:17)
at async createAndRunLoop (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/pregel/index.cjs:907:17)
at RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1285:70)
at raceWithSignal (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/utils/signal.cjs:4:30)
at RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1285:70)
at async RunnableCallable.generate [as func] (/Users/$USER/git/kibana/x-pack/solutions/security/plugins/elastic_assistant/server/lib/attack_discovery/graphs/default_attack_discovery_graph/nodes/generate/index.ts:61:27)
at async RunnableCallable.invoke (/Users/$USER/git/kibana/node_modules/@langchain/langgraph/dist/utils.cjs:82:27)
at async RunnableSequence.invoke (/Users/$USER/git/kibana/node_modules/@langchain/core/dist/runnables/base.cjs:1285:33)
```
</details>
The root cause of the validation error above was models like Gemini 2.5 Pro were generating responses with output tokens that reached the `maxOutputTokens` limit sent in the request.
When this happens, the response from the model:
- Includes a `finishReason` of `MAX_TOKENS`
- The `usageMetadata` does NOT include the `candidatesTokenCount`, which causes the validation error to occur
as illustrated by the following (partial) response:
```json
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"text": ""
}
]
},
"finishReason": "MAX_TOKENS",
// ...
}
],
"usageMetadata": {
"promptTokenCount": 82088,
"totalTokenCount": 90280,
"trafficType": "ON_DEMAND",
"promptTokensDetails": [
{
"modality": "TEXT",
"tokenCount": 82088
}
],
"thoughtsTokenCount": 8192
},
"modelVersion": "gemini-2.5-pro-preview-03-25",
"createTime": "2025-05-06T18:05:44.475008Z",
"responseId": "eE8aaID_HJXj-O4PifyReA"
}
```
To resolve this issue, the `maxOutputTokens` request parameter is only sent when it is specified via the Gemini connector's `invokeAI` and `invokeAIRaw` APIs.
#### Alternatives considered
All of the Gemini models tested will fail with errors like the following example when their (model-specific) output token limits are reached:
```
Status code: 400. Message: API Error: INVALID_ARGUMENT: Unable to submit request because it has a maxOutputTokens value of 4096000 but the supported range is from 1 (inclusive) to 65536 (exclusive). Update the value and try again.: ActionsClientLlm: action result status is error: an error occurred while running the action - Status code: 400. Message: API Error: INVALID_ARGUMENT: Unable to submit request because it has a maxOutputTokens value of 4096000 but the supported range is from 1 (inclusive) to 65536 (exclusive). Update the value and try again.
```
For example, passing a `maxOutputTokens` value of `32768` will work OK for `Gemini 1.5 Pro 002` and `Gemini 2.5 Pro`, but fail for `Gemini 1.5 Flash` and `Gemini 1.5 Pro`:
| Model | Description |
|--------------------|------------------------------------|
| Gemini 1.5 Flash | 1 (inclusive) to 8193 (exclusive) |
| Gemini 1.5 Pro | 1 (inclusive) to 8193 (exclusive) |
| Gemini 1.5 Pro 002 | 1 (inclusive) to 32769 (exclusive) |
| Gemini 2.5 Pro | 1 (inclusive) to 65536 (exclusive) |
As an alternative to removing `maxOutputTokens` from requests, we could have made it a required parameter to the `invokeAI`, `invokeAIRaw`, and `invokeStream` APIs, but that would require callers of these APIs to know the correct model-specific value to provide.
#### Desk testing
This PR was desk tested:
- Interactively in Attack discovery with the following models:
| Model |
|--------------------|
| Gemini 1.5 Flash |
| Gemini 1.5 Pro |
| Gemini 1.5 Pro 002 |
| Gemini 2.5 Pro |
- Interactively in Attack discovery with non-Gemini models, e.g. a subset of the GPT and Claude family of models (to test for unintended side effects)
- Via evaluations using `Eval AD: All Scenarios` for the Gemini connectors above, (and with the Claude / OpenAI models for regression testing)
- Interactively via the security assistant, to answer requests in tool calling, and non-tool calling scenarios
**Related Epic:** https://github.com/elastic/kibana/issues/179907
## Summary
This PR refactors prebuilt rules integration tests structure to streamline the implementation of test plans targeted [Prebuilt Rules Customization Milestone 4](https://github.com/elastic/kibana/issues/179907).
## Details
Existing integration tests structure have prebuilt rules related integration tests scattered around rules management area. Due to historical reasons and pace of the prebuilt rules customization development some old tests were updated, some new were added as random spots as well as some new tests structure was suggested.
This PR moves files and some tests around to the following structure
- `test_suites/detection_response/rules_management/prebuilt_rules` is the root folder for prebuilt rules related integration tests
- `customization_disabled` subfolder contains prebuilt rules integration tests covering scenarios when users have **insufficient** for customization license level (basic/essentials)
- `customization_enabled` subfolder contains prebuilt rules integration tests covering scenarios when users have **sufficient** for customization license level
- `customization_disabled` and `customization_enabled` subfoldera have test suites grouped by sub domains
- `prebuilt_rules_package` - contains integration tests related to detection rules Fleet package installation and updating, bootstrap is also belong to here
- `install_prebuilt_rules` - contains tests related to prebuilt rules installation from prebuilt rule assets
- `upgrade_prebuilt_rules` - contains tests related to prebuilt rules upgrade workflow
- `customization` - contains tests related to prebuilt rules customization including `is_customized` calculation
- `import_export` - contains tests related to exporting and importing customized and non-customized prebuilt rules
- `status` - contain status endpoints related tests
Closes https://github.com/elastic/kibana/issues/220278
## Summary
- implemented the change to hide `Observables` in the O11y UI. The
**features** property in the cases context provider now supports an
`observables : { enabled: boolean }` configuration, with the default set
to true. For Observability the `enabled` value will be passed as
`false`, and the observables feature will be rendered based on this.
---------
Co-authored-by: Christos Nasikas <xristosnasikas@gmail.com>
Fixes https://github.com/elastic/kibana/issues/209789
## Summary
For `inputs` type integrations
([docs](https://github.com/elastic/kibana/blob/main/x-pack/platform/plugins/shared/fleet/dev_docs/input_packages.md)),
we install all the assets when creating the integration policy and not
at integration install time (like for "regular": integrations).
However the clean up of assets doesn't happen when removing the
integration policy and this leaves around orphaned assets that are not
working anymore.
- This PR adds a new endpoint that removes the datastream assets
```
DELETE kbn:/api/fleet/epm/packages/{pkgName}/{pkgVersion}/datastream_assets?packagePolicyId={Id}
```
- The new endpoint is called by the UI when removing the integration
policy;
- Only the datastream assets that match exactly the dataset name are
removed; assets that are common across the integration are kept and can
only be deleted when the whole integration is uninstalled.
Additional changes:
- I did some light refactoring of the functions for inputs-type
integrations to make the code more readable
- Updated the dev_docs about input-type integrations that haven't been
touched for long time
### Testing
- Install an input-type package, for instance "custom logs"
- Check the assets created under the tab `assets`
- Check that the package has only this integration policy
- Remove the integration policy for the package - a warning is shown:
<img width="937" alt="Screenshot 2025-05-09 at 16 58 51"
src="https://github.com/user-attachments/assets/0f86ab38-e0a9-47f5-91f5-71b83e17f2e3"
/>
- Verify that the assets related to the package are cleaned up as well
- Try again but with several integration policies
- In this case the clean up doesn't happen
### Delete assets when there are two integration policies with different
dataset names
Dataset names are`udp.generic` and `udp.test` - in the video I deleted
policy `udp-2` having dataset name `udp.test` and the relative assets
are no longer present:
https://github.com/user-attachments/assets/23350051-1b26-4e52-914d-62f784809c80
### Delete assets when there are two integration policies with same
dataset names
In this case there are two different policies having the same dataset
name `udp.generic`, when deleting the policy there is no warning to
remove the assets. In fact they can be deleted only when there is only
one remaining policy using them:
https://github.com/user-attachments/assets/f75668dd-a4ce-4f5a-ba5d-c99911278dfc
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: jillguyonnet <jill.guyonnet@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Attempt to fix flakyness of the embeddable alerts panel functional test
by decreasing the interval of the rules that are used to generate alerts
to 5 seconds and parallelizing the execution of async tasks that are not
dependent on one another.
Not backporting to the 8.x track on purpose since the original PR
introducing this functional test hasn't been backported yet because of
this issue (will include the same fix in the backport branch).
## Summary 🌹Resolves#213024.
The frontend changes for
[#216377](https://github.com/elastic/kibana/pull/216377).
Depends on #216377 and https://github.com/elastic/kibana/pull/216292.
## Testing these changes 🌸
This adds frontend integration with the API changes we previously merged
in #216377. There is a new editor in the Rule Create/Edit Detail view,
below the pre-existing field for naming the rule.
To test that this feature is working you should:
- This is easiest to test if you have actual data that will trigger an
alert in your cluster. If you need some fake data, you can use the nifty
`data-forge` utility with a command like:
```shell
node x-pack/scripts/data_forge.js --events-per-cycle 200 --lookback now-1h --ephemeral-project-ids 10 --dataset fake_stack --install-kibana-assets --kibana-url http://localhost:5601 --event-template bad
```
- Create a rule with an investigation guide specified. This is easy.
Write some Markdown text into the editor and save the rule. My favorite
rule for testing the feature is Custom Threshold, because it's easy to
configure an alert that will fire. But this works for any rule.
<img width="1260" alt="image"
src="https://github.com/user-attachments/assets/bf9bf866-2439-456a-a700-1a93ae2e5dac"
/>
- After you create your rule, it should fire at some point, ideally.
Using the Observability -> Alerts view, drill into the Alert Details
page. There, you should find a spiffy new tab called _Investigation
Guide_. Confirm the contents on that tab are your markdown, properly
rendered.
<img width="1000" alt="image"
src="https://github.com/user-attachments/assets/bff19e48-da44-4886-bcf7-a296559c0aca"
/>
- Repeat step 1-2 as many times as you like with different rule types,
if you desire.
- Edit your rule, using the edit page or flyout.
<img width="606" alt="image"
src="https://github.com/user-attachments/assets/1a6149fe-016a-4b8c-9846-24cb2931aed3"
/>
- When you save the rule you should be able to refresh the alert details
page and see the modified Investigation Guide reflected in the tab.
---------
Co-authored-by: Panagiota Mitsopoulou <giota85@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/219488
## Summary
Add license gate around remote synced integrations - Only accounts
having Enterprise license will be able to use the new remote synced
integrations feature. This requirement depends on the ccr feature, which
works only on Enterprise licenses, this PR makes sure that this also
checked in Fleet.
### Testing
1. With a license different than enterprise
- Try to create an output that enables synced integrations - it should
fail with a 400:
```
POST kbn:/api/fleet/outputs
{
"name": "new 1",
"type": "remote_elasticsearch",
"hosts": [ "https://lskfgjojg" ],
"is_default": false,
"is_default_monitoring": false,
"sync_integrations": true
}
```
- Verify that the UI elements listed in
https://github.com/elastic/kibana/issues/219488 are not visible
2. With Enterprise license
- Check that the UI elements are visible and the remote sync
integrations features work as expected
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/streams-program/issues/173.
From a high level this takes the UI components / models in the Grok UI
package from a POC level to a polished level, and then integrates those
with the Enrichment / Extraction page in the Streams UI. The Grok UI
package is completely agnostic to it's consumer, so these package
resources should be ready to integrate with any part of Kibana (we want
to eventually bring this to Discover, for example).
## Notes for the reviewer
Quite a few notes, but hopefully they help navigate the code and
decisions made.
- The `PreviewTable` now allows a custom `renderCellValue` prop, this is
used when we are dealing with a **draft** Grok processor.
- With processors we have a saved state (these don't run in the
simulation), a staged state (configured and added but not saved, these
are used in the simulation), and a draft state (freshly added, but not
staged / saved yet, these are used in the simulation). The Grok
highlighting is only available in the simulation table when the Grok
processor if in a **draft** state, this is because we can't cleanly
differentiate between staged processors at the moment which one exactly
is being edited (multiple edit panels can be opened at once). I have
discussed this with @tonyghiani and @patpscal and we've agreed this is
fine to start with, with
https://github.com/elastic/kibana/issues/218955?reload=1?reload=1 and
moving to only one edit at a time we will be able to roll this out to
editing staged processors as well as the draft processor.
- Custom sample data is **always** available, so there is always a way
to test the patterns with highlights etc.
- In the Grok UI package there is a read only version of the sample
component, this is for things like cell rendering within a table and
provides it's tooltips via EUI. There is also an "input" version which
is backed by Monaco (tooltips also provided by Monaco). The styling
differs on the tooltips a bit between the two for this reason, I will
try and enhance / override the Monaco styles so they adhere to our
colour palette / themes etc in a followup.
- Giorgos had suggested using the `/_ingest/processor/grok` ES endpoint
to fetch the patterns, however I couldn't find a convenient way to do
this yet without adding an API endpoint somewhere (since this needs an
ES client). I've deferred this for now, but it's why I've preempted
`setup()` being modelled as `async`.
- I had to pass `formControl` around in a bit of an ugly way in the
patterns editor, this was purely due to the `useForm()` hook not working
as I expected in that part of the hierarchy.
- We only need a single `GrokCollection` instance, as such this is
stored in the top level state machine.
- We technically have two areas where state is managed. We have the
state machines and then we have the form state managed by React Hook
Form. The form state is synchronised back to the state machines when
processor changes are made. This Grok integration work introduces a new
need which is for resources to be shared between the processors panel
and the simulation panel, in this case the same `DraftGrokExpression`
instances. There are multiple ways this could have been modelled, with
various pros and cons, but I wanted to keep the paradigm we have now
which is that the processor definitions in the state machine are
"complete" (in the sense they're converted) and not partial form state.
The `DraftGrokExpressions` are integrated directly with the form state,
and synchronised back to a new `resources` property in the state
machine.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
### Summary
Sets the priority of the `@stream` templates to the highest possible
value which guarantees that no existing templates can take precedence if
it is successfully created. Since no overlapping templates can have the
same priority, the template will fail to create if there is an existing
template that covers the stream index pattern.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes#217617
Adds a new endpoint `POST kbn:/api/fleet/agents/{agentId}/migrate`
allowing a user to migrate a single agent to another cluster.
Required parameters are: `enrollment_token` and `uri`
- Adds `MIGRATE` as an action type and is reflected in UI
- Includes unit tests as well as integration tests
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
N/A
# Release Note
- Added endpoint allowing a user to migrate an individual agent to
another cluster by specifying the URL and Enrollment Token. Note: tamper
protected and fleet agents can not be migrated and attempting to do so
will return a `403` status code.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/217683
## Summary
Follow up of https://github.com/elastic/kibana/pull/217144
Handle errors occurring when `sync_uninstalled_integrations` is enabled
on remote outputs; these errors are now saved under
`latest_uninstall_failed_attempts` and will be reported by
`api/fleet/remote_synced_integrations/<output_id>/remote_status` and
`api/fleet/remote_synced_integrations/remote_status`.
- I added a new field in the response of these apis that allows to
understand at a glance the install status of an integration on both
cluster:
```
install_status: {
main: 'installed',
remote: 'not_installed',
}
```
- Added a new "warning" state for synced integrations
- Handled the case when an integration was successfully uninstalled from
both clusters (marked as complete)
- Removed the "throw error" for the case of `outputId` in favour of a
regular error in the response
### Testing
- Follow steps in https://github.com/elastic/kibana/pull/217144
- Check that the errors reported in `latest_uninstall_failed_attempts`
are now visible when querying
`api/fleet/remote_synced_integrations/<output_id>/remote_status` under
the new "warning" field. This can be done also from the UI, checking the
network tab
-
<img width="2111" alt="Screenshot 2025-05-20 at 11 17 13"
src="https://github.com/user-attachments/assets/80a077e7-8b1b-4d04-abe9-0ef0cc44def8"
/>
Response for the failed uninstalled integration:
```
{
"package_name": "akamai",
"package_version": "2.28.0",
"install_status": {
"main": "not_installed",
"remote": "installed"
},
"updated_at": "2025-05-21T09:34:34.492Z",
"sync_status": "warning",
"warning": "Unable to remove package akamai:2.28.0 with existing package policy(s) in use by agent(s) at Fri, 23 May 2025 07:54:41 GMT"
},
```
### UI changes
The integrations uninstalled from the main cluster are now shown with a
greyed out text and the warning is shown on screen as well:
<img width="703" alt="Screenshot 2025-05-23 at 10 37 57"
src="https://github.com/user-attachments/assets/a6900e0b-96cc-4bcc-8f16-db0001f55de3"
/>
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/220815
## Summary
This PR adds telemetry for _ignored fields. The goal is to have an alert
based on this field and get notified in case a user hits the mapping
limit, but by default, we don't expect this to happen.
This PR adds `count_ignored_fields_by_rule_type` field that counts the
number of _ignored fields per rule type.
In the future, we can extend the telemetry data to also include the
actual number of mappings over the limit (there is a [feature
request](https://github.com/elastic/elasticsearch/issues/68947) for
adding field count information to index API)
### How to test
- Add a lot of dynamic fields as mentioned here:
https://github.com/elastic/kibana/pull/216719
- Create a rule with a custom threshold rule with multiple group by
fields to generate an alert with _ignored field
- Run the following API and check the value of
`count_ignored_fields_by_rule_type`
```
POST kbn:/internal/telemetry/clusters/_stats?apiVersion=2
{
"unencrypted": true,
"refreshCache": true
}
```
<details>
<summary> Here is what it looks like:</summary>
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves#211243.
As noted in the linked ticket, Uptime rule types display on the Create
Rule modal, even if Uptime is disabled (which it is by default).
You can test this by going to Advanced Settings ->
`observability:enableLegacyUptimeApp` and setting it to `enabled`. When
it is enabled, you should see the Uptime rule types in the modal. When
the setting is turned off, they should be hidden.
*Note to reviewer:* the naming of the filtered var might be something we
want to change. It's also possible there's a better spot to perform this
filtering than in the component itself, like in one of the hooks it
calls or further up the component tree. I am open to any suggestions on
how to improve this.
### Uptime disabled
<img width="1168" alt="image"
src="https://github.com/user-attachments/assets/3a182c6f-add1-4be6-9df6-274c903d07b7"
/>
### Uptime enabled
<img width="1185" alt="image"
src="https://github.com/user-attachments/assets/b8f7ae68-2152-4f2a-83e4-88aed286059a"
/>
Closes#216514
## Summary
This PR skips the flaky Osquery tab test: I added some context in the
[issue
comment](https://github.com/elastic/kibana/issues/216514#issuecomment-2916281697).
I don't find this test super useful the way it is, so we should revisit
it in the future (I added a comment as a reminder). I am also fine to
remove it.
## Summary
Fix#220616
This PR introduces a new Lens API to help mounting a unified flyout
panel for a Lens chart.
The rest of the PR moves into the alerting codebase the entire logic
within the `initializeAlertRules` in the Lens embeddable area.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
