## Summary
Splitting long running config:
`x-pack/test/functional/apps/saved_query_management/config.ts` **~57
min** into
- x-pack/test/functional/apps/saved_query_management/config.ts 35m
- x-pack/test/functional/apps/saved_query_management/config.v2.ts 25m
17s
ideally we need to split both even more, but I will leave it for the
later (probably Data-Discovery Team have some ideas how to re-org it?)
## Summary
This PR implements the initial work to keep fetched results when
switching tabs:
- Avoid cancelling the current document request when switching tabs
(this still needs more work, especially migrating the `use_esql_mode`
hook to the central data fetching).
- Move `DiscoverStateContainer` and `DiscoverCustomizationService` to
`RuntimeStateManager` so they can be reused by tabs without
reinitializing.
- Re-add the current tab ID to `InternalStateStore` for high-level tab
management only (called `unsafeCurrentId` now to discourage misuse).
- Move `initializeAndSync` and initial `fetchData` call to the
`initializeSession` thunk to avoid calling it when switching back to
existing tabs.
- Move URL tracking directly into `DiscoverSavedSearchContainer` since
it previously used a hook which could now become out of sync because
`initializeAndSync` was moved (URL could update before the hook was
called).
- Support fully disconnecting tabs with a new `disconnectTab` thunk
(called on tab close and for all remaining tabs when leaving Discover).
- Sync global services to current tab state when switching tabs (this
should probably be cleaned up more, but it should work as a start).
- Basic implementation of `getPreviewData` (needs to be cleaned up).
- A couple of small misc changes (e.g. fixing scrollbar when tabs are
enabled).
Part of #216475.
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
### Fleet Changes
- Added a debug statement to the execution of external callbacks that
outputs `callback.toString()` whenever an external callback throws an
error
- The output of `callback.toString()` should help identify the code
behind the callback that threw an error and assist with debugging
### Security Solution
- Fixes the Endpoint package policy update callback that is registered
with Fleet to now throw an error while attempting to report policy
protection feature usage. (see error below)
- Fleet recently made a change to `PackagePolicy.bulkUpdate()` service
method that ensure registered callbacks for policy update is called,
which seems to be the reason why this started to happen recently. Its
unclear why this error is occurring being that Space Awareness features
are currently disabled by default.
Closes: #215457
**Description**
When user clicks on Add lens, maps or custom visualizations, Kibana
announces them as "Group" which doesn't give the user the context that
they are on create lens page or maps page or custom visualization
**Changes made:**
1. Added `role="menu"` and `role="menuitem"` to provide users with more
context.
2. The `Group` component was slightly updated: `EuiListGroupItem` and
`EuiTooltip` were removed in favor of using the native EuiListGroup API.
**Screens:**
https://github.com/user-attachments/assets/8587787c-6444-4d39-b499-5f1a3daefbde
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `29150cd` ->
`c56628d` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Jon <jon@elastic.co>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base-fips | digest | `60d2da3` ->
`22d75e2` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwiY2k6YnVpbGQtZG9ja2VyLWZpcHMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Jon <jon@elastic.co>
## Summary
This PR fixes the https://github.com/elastic/kibana/issues/215323 issue
and also provide small cleanup on the code near the edited file, not
directly correlated but I was already there and the changes are pretty
minimal.
Cleanups:
- the `SavedObjectReference` is deprecated and needs to be imported from
`core/server` instead
- Looking at the locator I've noticed a couple of non-null assertion
that wasn't necessary
- the locator readme was referencing the old `kibana app services` team
- cleaned up the `x-pack/platform/plugins/shared/lens/public/utils.ts`
to align all the method to use the same form (or all functions or all
arrow function, there was a pretty interesting mix there)
- replaced usages of exclusive OR with a more appropriate nullish
coalescing operator
## Summary
Removes unused code from the Investigate and Investigate app plugin.
Removes all references to those plugins in storybook, i18n, types, etc.
Removes codeowner requirements for those plugins
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/209386
## Summary
1. The option to archive conversations are enabled via the conversation
contextual menu.
2. Archived conversations can be viewed under the "Archived" section of
the conversation list.
3. Only the owner of the conversation can archive and unarchive.
4. Once archived, the conversation cannot be continued until unarchived.
5. If the archived conversation is shared, other users (who are not the
owner) can duplicate the conversation, if they wish to continue the
conversation.
6. The archived section of the conversation list is collapsed by
default.
7. Updating the conversation such as title updates, regenerating,
providing chat feedback are disabled for archived conversations
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Related: https://github.com/elastic/kibana/pull/216916
The flaky test runner is sometimes throwing this error:
```
└-: withLock API
--
| └-> "before all" hook: beforeTestSuite.trigger in "withLock API"
| └-> "before all" hook in "withLock API"
| └- ✖ fail: Stateful Observability - Deployment-agnostic API integration tests observability AI Assistant LockManager withLock API "before all" hook in "withLock API"
| │ ResponseError: {"took":1,"timed_out":false,"total":1,"deleted":0,"batches":1,"version_conflicts":1,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[{"index":".kibana_locks-000001","id":"my_lock_with_token_fencing","cause":{"type":"version_conflict_engine_exception","reason":"[my_lock_with_token_fencing]: version conflict, required seqNo [117], primary term[1]. but no document was found","index_uuid":"F_O5sNfQSLqtyPeLexG_Qw","shard":"0","index":".kibana_locks-000001"},"status":409}]}
| │ at SniffingTransport._request (node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:605:17)
| │ at processTicksAndRejections (node:internal/process/task_queues:95:5)
| │ at /opt/buildkite-agent/builds/bk-agent-prod-gcp-1744099176361402458/elastic/kibana-flaky-test-suite-runner/kibana/node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:711:22
| │ at SniffingTransport.request (node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:708:14)
| │ at Client.DeleteByQueryApi [as deleteByQuery] (node_modules/@elastic/src/api/api/delete_by_query.ts:143:10)
| │ at Context.<anonymous> (distributed_lock_manager.spec.ts:444:9)
| │ at Object.apply (wrap_function.js:74:16)
```
The error happens when `release` and `clearAllLocks` simultaneously
tries to delete the same document. This PR ensures that `clearAllLocks`
will not throw an exception on conflict
## Summary
A PR introduced into 8.18/9.0
(https://github.com/elastic/kibana/pull/205507) changed the way we
validate the log threshold rule type parameters. The validation happens
on rule params and changes a loose validation to a strict validation, so
those users who’ve inserted excess fields before 8.18/9.0 will see rules
starting to fail to run, their rule page failing to load and the API
starting to reject calls with excess fields.
Fixes: https://github.com/elastic/kibana/issues/217384
## Testing instructions
1. Start Kibana on 8.17 and create the following rule using the API. Let
the rule run.
<details><summary>Rule</summary>
```
{
"name": "[QAF] Observability rule 3",
"tags": [
"metrics",
"threshold",
"qaf"
],
"rule_type_id": "logs.alert.document.count",
"consumer": "alerts",
"schedule": {
"interval": "1m"
},
"actions": [],
"params": {
"timeSize": 8,
"timeUnit": "h",
"count": {
"value": 1,
"comparator": "more than"
},
"criteria": [
{
"field": "bytes",
"comparator": "more than",
"value": 1
}
],
"logView": {
"logViewId": "log-view-reference-0",
"type": "log-view-reference"
},
"groupBy": [
"geo.dest"
],
"outputIndex": ".alerts-observability.logs.alerts-default"
}
}
```
</details>
2. Start Kibana on 8.18. Verify that you cannot create the same rule and
the rule created in step 1 starts failing.
3. Start Kibana on this PR and that you can create the same rule and the
rule created in step 1 is working as expected.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Currently, the streams UI doesn't deal well with partial permissions.
This PR improves that. As a lot of things come together in streams, we
could do even better, but I think it's OK to draw a line somewhere.
The logic is now as follows:
When reading a stream, the privileges of the current user are returned
along with the stream itself. These are grouped like this:
```
interface IngestStreamPrivileges {
// User can change everything about the stream
manage: boolean;
// User can read stats (like size in bytes) about the stream
monitor: boolean;
// User can change the retention policy of the stream
lifecycle: boolean;
// User can simulate changes to the processing or the mapping of the stream
simulate: boolean;
}
```
This is part of the definition response and is passed around to the
components and disabled buttons and similar in the places where this is
necessary.
The "advanced" tab is only shown when full `manage` permissions are
present - there constellations of permissions that would allow some
access but not all (e.g. having `read_pipelines` but not
`manage_index_templates`), but these should be rather rare and not worth
the additional effort.
## Conditions
In the following places privileges are checked:
* Overview
* Without `monitor`, the overall stats are not shown
* Enrichment
* Without `manage`, you can't save changes
* Without `simulate`, the UI is readonly
* Partitioning
* Without `manage`, you can't save changes
* Without `simulate`, the UI is readonly
* Schema editor
* Without `manage`, the UI is readonly
* Retention
* Without `monitor`, the ingest stats are not shown
* Without `lifecycle`, the retention can't be changed and ILM breakdown
is not rendered
* Advanced
* Without `manage`, the tab is hidden completely
## Drive-by fix
I noticed that we still register the app header action menu which adds
an empty bar on serverless, removed that code.
## Testing
Check
https://github.com/elastic/kibana/pull/217353/files#diff-d8f33d7021058bf90cbeea908bf399da2af50d8b8bfac8a07f160ddc0cdff12bR747
for which Elasticsearch level privileges you need for different
permutations. Then set up a role and a user and log in as that user.
Also test the different pre-defined roles on serverless.
## Summary
### Lists plugin:
- Add the ability to pass a KQL `filter` to the
exportExceptionListAndItems()` server-side service
- NOTE: this `filter` property is NOT exposed via the public API. Only
used internally
- Fixes the `getExcetionList()` service method to ensure that if a list
if not found using the `id`, that it attempts to then find it using
`list_id` if that was provided on input to the method.
### Security Solution:
- Export for endpoint artifacts was updated with additional filtering
criteria to ensure that only artifact accessible in active space are
included in the export
## Summary
Splitting the following config:
-
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
**~61 min**
by moving `ai_assistant`, `synthetics` and `streams` tests in its own
configs
-
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.ai_assistant.serverless.config.ts
~11m 30s
-
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.synthetics.serverless.config.ts
~21m 30s
-
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.streams.serverless.config.ts
~21m 43s
original config with less tests:
-
x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
~17 min
### Background
This PR is a proposal for a different way to structure the Streams code
flow based on some challenges faced while working on
https://github.com/elastic/streams-program/issues/26 and discussed
[here](https://github.com/elastic/streams-program/discussions/147) and
[here](https://github.com/elastic/streams-program/discussions/55),
mainly around finding it difficult to decide where to place certain
validations that need access to the state as a whole.
It is also in response to some expressed difficulty about how to add new
stream types into the code base.
It aims to achieve 3 goals:
1. It is easy to add new stream types and there is a clear place where
changes (new validation, new logic) for existing stream types happen,
making the code easier to evolve over time
2. It is easier to improve the robustness of the system because there
are clear phases where problems can be caught, fixed and rolled back
3. It lays some ground work for features such as [bulk
changes](https://github.com/elastic/streams-program/issues/125), [dry
runs](https://github.com/elastic/streams-program/discussions/138) and a
[health
endpoint](https://github.com/elastic/streams-program/discussions/139)
In the future, this will most likely be handled by Elasticsearch to a
large degree, as imagined in
https://github.com/elastic/streams-program/discussions/30
The solution takes inspiration from the reconciliation / controller
pattern that Kubernetes uses, where users specify a desired state and
the system takes action towards reaching that step. But it is also
somewhat more similar to how React's Virtual DOM works in that it
happens in a single iteration.
Another key pattern is the [Active Record
pattern](https://www.martinfowler.com/eaaCatalog/activeRecord.html), we
let each stream class contain all the logic for how to validate and
modify that stream in Elasticsearch. The client and `State` class simply
orchestrate the flow but defer all actual work and decision making to
the stream classes.
**Note:** This PoC ignores the management of assets
### Summary
The process takes the following steps:
1. A route accepts a request (upsert / delete) and translates it into
one or more (for bulk) `StreamChange` objects before passing these to
`State.applyChanges` method (which also takes a toggle for dry runs)
2. The current state of Streams is loaded by using the `State` class
3. The changes are then applied to the current state to derive the
desired state [1]
4. The desired state is then validated, this is done by asking each
individual stream if given the desired state and starting state, from
the perspective of that individual stream, is it in a valid state
(upserted or deleted correctly)
5. If the state is invalid, we return those errors and stop
6. Else we continue, if it's a dry run, we ask the desired state object
for what has changed and report that in the shape of the Elasticsearch
actions that would be attempted
7. Else we proceed to commit the changes to Elasticsearch by asking each
changed stream to determine which Elasticsearch actions need to be
performed to reach the desired state
8. These actions are then combined and sent to the `ExecutionPlan` class
which does planning (mainly for actions around Unwired streams) and then
handles executing the actions in the most parallel way but in the safe
order
9. If any error happens, we attempt to revert back to the starting state
by taking the changed streams and marking each stream as created based
on the starting state and then getting the Elasticsearch actions for
that and applying those
This PR also changes our `resync` endpoint to make use of the same rough
strategy (load current state, mark all as created, get Elasticsearch
actions and apply).
[1] Applying changes:
1. The current state is first cloned
2. Then for each change we see if it is a deletion or an upsert
3. Based on this we either mark existing streams for deletion or
create/update existing streams
10. When creating a new stream instance we use the helper
`streamFromDefinition` which is the only mapping between the definition
documents and the Active Record-style stream type classes
11. As part of this, each stream that changes is marked in the desired
state
12. The stream is passed the desired and current state and should update
itself based on the change
13. The stream can return a set of cascading changes (taking the same
format as the requested changes) which are executed directly after but
we have a limit for how many rounds of cascading changes can happen to
avoid infinite loops
### Adding new stream types
Key in all of this is that the client and `State` classes don't know
anything about any of the specific stream types, they know only of the
`StreamActiveRecord` interface.
When adding a new stream type you need to implement this interface and
update `streamFromDefinition` to create the right class for your new
definition. Streams of different types should only interact with each
other by creating cascading changes.
### Possible follow up tasks
- Introduce a lazy Elasticsearch cluster state cache because multiple
places in the code access the same stuff over and over again
- Make API endpoints the consume `attemptChanges` pass back the
`DesiredState` and planned `ElasticsearchActions` as debug information
based on a flag (maybe also all cascading changes)
- Don't run cascading changes by default but run them if _some_ flag is
submitted based on
https://github.com/elastic/streams-program/discussions/230
- Wrap `attemptChanges` and `resync` with the new LockManager
https://github.com/elastic/kibana/pull/216397
- Unit test WiredStream, UnwiredStream and GroupStream
- Clean up old sync helpers
- Wrap ES calls to get better stack traces for errors
### Out of scope
- Asset linking and content pack installation (it's probably okay for
these to continue to use the asset client directly since there is less
domain logic and no cascading changes involved)
---------
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Added `DisabledAuthz` utility class, this will address the current
repetition of the reason string `'This route delegates authorization to
the ES/SO client` and other common scenarios.
__Closes: https://github.com/elastic/kibana/issues/216632__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jeramy Soucy <jeramy.soucy@elastic.co>
* Preview tables for enrichment and partitioning showed header actions
which were not actually taking effect (now disabled)
* Ingest pipeline was linking to "not found" flyout if data stream
doesn't have a default ingest pipeline
* Index pattern for discover link and histogram was built the wrong way
(shows child streams for classic streams instead of wired streams)
## Summary
Feature description: [internal
link](https://github.com/elastic/security-team/issues/10142)
Part of: [internal
link](https://github.com/elastic/security-team/issues/12005)
These changes we switched to `core.services.featureFlags` to manage the
`assistantAttackDiscoverySchedulingEnabled` feature flag to control
visibility of the Attack Discovery Scheduling feature.
To enable the flag locally, add this in the `kibana.dev.yml`:
```
feature_flags.overrides:
securitySolution.assistantAttackDiscoverySchedulingEnabled: true
```
This PR replaces usage of the Kibana Task Manager with the [LockManager
utility](https://github.com/elastic/kibana/pull/216397) that ensures
strict mutual exclusion when running operations such as re-indexing or
data migrations.
---------
Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR updates the rendering of multi value fields - vulnerability.id,
package.name, package.version and package.fixed_version in the
vulnerabilities data-grid page and alerts insights vulnerabilities tab
data grid.
It also updates the rendering of package.* fields in the vulnerabilities
flyout and both flyout and data grids are re using the same kbn package
component to display it.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Make CVSS column to be displayed first in the data grid.
- [x] if there is a single CVE display its value across the data grid.
- [x] data-grid if there is more than a single CVE show it as
<first_cve> <+x more> badge indicating the number of CVES left. Clicking
on the badge should open a Popver where all CVEs are displayed as badges
- clicking on the value will add it to the search bar filters, each
batch should have a copy icon as well.
- [x] insights tab data-grid should have similar logic to display multi
value fields but without adding it to the filters logic since there are
no filters in that page.
- [x] logic of displaying multiple CVEs should be applied to
package.name, package.version and package.fixed_version fields in both
data grids.
- [x] arrays in package-related vulnerability fields are rendered
correctly in the flyout header and footer.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Screen recording
https://github.com/user-attachments/assets/208f8445-83c1-4e8f-a490-85ec48830fae
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR enhances support for otel data in Synthtrace. It introduces the
ability to generate otel-sdk data and transforms APM Server data into
otel format.
Key Changes
- Added a pipeline for processing otel traces.
- Add a pipeline to convert APM server traces into exported otel traces
- Removed `OtelSynthtraceEsClient` in favor of `ApmSynthtraceEsClient`.
### Examples
Reproduces the otlp traces -> APM Server/tracesexporter -> output case
```ts
// this needs to be set in the synthtrace scenario's `bootstrap` to run the correct pipeline
apmEsClient.pipeline(apmEsClient.getPipeline('otelToApm'));
// scenario
apm.otelService({
name: 'sendotlp-otel-native-synth',
sdkName: 'otlp',
sdkLanguage: 'nodejs',
})
.instance('otel-native-instance')
// this interface doesn't provide a `transaction` function
.span({
name: transactionName,
kind: 'Server',
})
.timestamp(timestamp)
.duration(1000)
.success()
.children(
instance
.dbExitSpan({
name: 'GET /',
type: 'elasticsearch',
})
.duration(1000)
.success()
.timestamp(timestamp)
)
```
APM Server -> otel output. This can be useful for reusing existing
synthtrace scenarios.
```ts
// this needs to be set in the synthtrace scenario's `bootstrap` to run the correct pipeline
apmEsClient.pipeline(apmEsClient.getPipeline('apmToOtel'));
// scenario
apm.service({
name: 'apmserver-otel-synth',
environment: 'prod',
agentName: 'opentelemetry/java',
})
.instance('otel-apmserver-instance')
.transaction({ transactionName })
.timestamp(timestamp)
.defaults({
'url.domain': 'foo.bar',
})
.duration(1000)
.success()
.children(
otelApmServerInstace
.span({
spanName: 'GET apm-*/_search',
spanType: 'db',
spanSubtype: 'elasticsearch',
})
.duration(1000)
.success()
.destination('elasticsearch')
.timestamp(timestamp)
)
```
### How to test
otel -> APM Server/tracesexporter -> output
Run `node scripts/synthtrace otel_simple_trace.ts --live --uniqueIds
--clean`
<img width="800" alt="image"
src="https://github.com/user-attachments/assets/e237e506-1c0d-4851-9053-0f1e2fe554db"
/>
APM Server -> otel
Run `node scripts/synthtrace simple_trace.ts --scenarioOpts
pipeline=apmToOtel --live --uniqueIds --clean`
<img width="800" alt="image"
src="https://github.com/user-attachments/assets/1bd577be-ded6-44ad-a54d-c5bb4e5ad59d"
/>
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[@launchdarkly/node-server-sdk](https://redirect.github.com/launchdarkly/js-core/tree/main/packages/sdk/server-node)
([source](https://redirect.github.com/launchdarkly/js-core)) |
dependencies | patch | [`^9.7.6` ->
`^9.7.7`](https://renovatebot.com/diffs/npm/@launchdarkly%2fnode-server-sdk/9.7.6/9.7.7)
|
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJUZWFtOlNlY3VyaXR5IiwiYmFja3BvcnQ6cHJldi1taW5vciIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/188856
## Summary
Currently ES query rule types link back to the alert details in the
`context.link` variable, except for the search source type. This PR
updates the rest of the ES query rule types to links to the relevant
docs in Discover.
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To verify
1. Set `server.publicBaseUrl` in kibana.yml
2. Create an ES|QL rule and include `context.link` in your action
message. Verify that when you go to the link it goes to Discover with
the correct query and timestamp
3. Create a DSL ES query rule and include `context.link` in your action
message. Verify that when you go to the link it goes to Discover and
works as expected
4. For your DSL rule also add some filters and verify that when you go
to Discover your filters are correctly applied.
Resolves https://github.com/elastic/kibana/issues/216319
## Summary
Adds an internal reporting health API to return whether conditions are
sufficient to support scheduled reports. For scheduled reporting, we
need for security and API keys to be enabled in Elasticsearch and for a
permanent encryption key to be set for the encrypted saved objects
plugin.
```
GET kbn:/internal/reporting/_health
Response
{
"has_permanent_encryption_key": true,
"is_sufficiently_secure": true
}
```
The issue also mentions returning whether a preconfigured email service
is configured, but that will be done as part of the main scheduled
reporting task.
## To Verify
1. Run kibana and ES with no special flags, both flags should be `true`
2. Run ES with `-E xpack.security.enabled=false`.
`is_sufficiently_secure` should be set to `false`
3. Run ES With `-E xpack.security.authc.api_key.enabled=false`.
`is_sufficient_secure` should be set to `false`
Note that in dev mode, an encryption key is auto-set if not defined in
the Kibana yml so `has_permanent_encryption_key` will always return
`true` in dev mode.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[@types/semver](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/semver)
([source](https://redirect.github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/semver))
| devDependencies | minor | [`^7.5.8` ->
`^7.7.0`](https://renovatebot.com/diffs/npm/@types%2fsemver/7.5.8/7.7.0)
|
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDphbGwtb3BlbiIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Jean-Louis Leysens <jeanlouis.leysens@elastic.co>
