## Summary
Resolves#180590
Since SLO instanceId is provided in all APIs, it will be made required
for sloWithDataResponseSchema and the SLOWithSummaryResponse type.
Checks for the existence of instanceId have either been removed or
changed to check for ALL_VALUE (`*`)
---------
Co-authored-by: Kevin Delemme <kevin.delemme@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
**Fixes: https://github.com/elastic/kibana/issues/202966**
**Fixes: https://github.com/elastic/kibana/issues/206527**
## Summary
The issue that causes the overarching problem mentioned in the ticket is
that we add an extra `alias: null` property to the filter via the kibana
filter utils instead of keeping the `alias` field unset. This is
functionally the same rule but since the prebuilt rule objects are
technically different (`alias` is set to `undefined` instead of `null`),
we mark these rules as customized and causes the query fields to show as
a modified field on update.
To address this, since changing the kibana util filter would be very
invasive and touching a lot of code, we instead normalize the field on
our side before version comparison. This fixes the bug reported and
improves resiliency of rule upgrades in the future.
### Testing (copied from ticket)
- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to
`kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web
browser)
- Install an outdated version of the `security_detection_engine` Fleet
package
```bash
curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```
- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```
- Open a `threat_match` rule for editing. For example `Threat Intel Hash
Indicator Match` with rule_id `aab184d3-72b3-4639-b242-6597c99d8bca`.
With this fix, users should **NOT** see any extra fields in the rule
upgrade flyout, nor should the rule be marked as "Modified" if opened
and saved with no other modifications
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR updates the ES|QL grammars (lexer and parser) to match the
latest version in Elasticsearch.
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
Make the Security Solution `endpoint-meta-telemetry` task send the new
Endpoint field `Endpoint.metrics.top_process_trees`
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Improved handling for operator-defined role mappings:
- Detail page with operator-defined role mappings is now marked with
lock icon and tooltip
- Operator-defined role mappings have a read-only experience.
<img width="1256" alt="Screenshot 2025-01-29 at 11 45 27"
src="https://github.com/user-attachments/assets/911dd2cd-4fe8-4141-8d8a-ffde974884d7"
/>
<img width="1234" alt="Screenshot 2025-01-28 at 15 21 44"
src="https://github.com/user-attachments/assets/d9a03431-f8da-471e-8c94-f504aa00623d"
/>
### How to test
1. Create a `settings.json` file in
`$ES_HOME/config/operator/settings.json`, and define some role mappings
there. Here's an example that will create 2 operator mappings:
```json
{
"metadata": {
"version": "1",
"compatibility": "8.4.0"
},
"state": {
"role_mappings": {
"operator_role_mapping_1": {
"enabled": true,
"roles": [
"kibana_user"
],
"metadata": {
"from_file": true
},
"rules": {
"field": {
"username": "role-mapping-test-user"
}
}
},
"operator_role_mapping_2": {
"enabled": true,
"roles": [
"fleet_user"
],
"metadata": {
"from_file": true
},
"rules": {
"field": {
"username": "role-mapping-test-user"
}
}
}
}
}
}
```
2. Navigate to `Role Mappings` page and check the UI has a read only
view.
3. Navigate to `Role Mappings Details` page and check the UI has a read
only view.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
__Closes: https://github.com/elastic/kibana/issues/194635__
### Release Notes
Improved handling for operator-defined role mappings
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## 🍒 Summary
This PR adds support to `/api/streams` endpoints for the
`GroupStreamDefinition` type. Group streams are simply a list of member
streams along with dashboards. An example of the definition looks like:
```JSON
{
"name": "nginx-logs",
"stream": {
"grouped": {
"description": "A collection of streams for Nginx",
"members": [
"logs",
"logs.nginx"
]
}
},
"dashboards": []
}
```
The following APIs support `GroupStreamDefinition`:
- `GET /api/streams`
- `GET /api/streams/{id}`
- `PUT /api/streams/{id}`
- `DELETE /api/streams/{id}`
- `GET /api/streams/{id}/_details`
This PR only includes the support to the APIs, I will submit a follow PR
for the UI.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
## Summary
While working on 9.x Upgrade testing, i found that when running
synthtrace scenarios pointing to a 7.x cluster, it tries to fetch the
latest version of the APM package. In 7.x, the `latestVersion` item was
present under a different node in the API response. Hence adding this
fix.
## Summary
This package relies on devOnly packages, thus it must be devOnly too.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#184582
## Summary
This PR fixes an issue with processes missing from the processes table
and not being searchable.
- Additional fixes: The chart size was not set correctly and they were
not visible so it is fixed now:
| Before | After |
| ---- | ------ |
|
|
|
⚠️ _UPDATE_ I also updated the archive used in the API test with the one
we have for the e2e name (and matched the name with the old one) so we
have consistent results and also the correct fields / mappings
### Solution details
After some digging into the fields I ended up **not** using the
suggested `process.name` because it gave us too generic information for
the table (we should still use the command there) also tried the process
id but then the search by command won't work properly. So I found a
field that is not ignored and contains the same information called
`process.command_line`and used it:
<img width="538" alt="image"
src="https://github.com/user-attachments/assets/ce4d35a6-e912-4656-bee1-8137d5635432"
/>
## Testing
- Follow the steps in the issue inside `How to reproduce` to create the
two processes
- Note: in order to see the process summary, you need to enable it in
the config ( ⚠️ for the oblt hosts I checked it is not enabled and the
summary section showed N/A ) :
https://github.com/user-attachments/assets/41bdcc8d-6b87-4351-b866-2e706dd11dfd
- The metrics charts should be visible
- The search should work as expected:
- And the missing.... process should be visible
Fixes https://github.com/elastic/kibana/issues/187065
## Summary
In the repository creation endpoint, when the validation failed, we were
showing duplicated errors. This was caused by how the validation schema
was build. This PR refactors the duplication schema so it doesn't
duplicate the nama field.
### How to test
* RunES with yarn es snapshot --license=trial -E
path.repo=/tmp/es-backups
* Navigate to repos list and try creating a new repo
* Fill the name input with an string longer than 1000 chars.
* Verify the error message when saving the repo is not duplicated
### Screenshot
<img width="1032" alt="Screenshot 2025-02-03 at 15 24 27"
src="https://github.com/user-attachments/assets/75231f68-a13e-44bf-8d8a-be488c9f25e5"
/>
As titled. The Project Switcher was created in #155582 as a means of
allowing a developer to switch between project types without having to
restart Kibana. Since that first PR, Kibana *must* be started with a
particular project configuration in mind-- the switcher now does
nothing. This PR removes it and deprecates the config keys.
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[react-reverse-portal](https://togithub.com/httptoolkit/react-reverse-portal)
| dependencies | minor | [`^2.1.2` ->
`^2.2.0`](https://renovatebot.com/diffs/npm/react-reverse-portal/2.1.2/2.2.0)
|
---
### Release Notes
<details>
<summary>httptoolkit/react-reverse-portal
(react-reverse-portal)</summary>
###
[`v2.2.0`](https://togithub.com/httptoolkit/react-reverse-portal/compare/v2.1.2...v2.2.0)
[Compare
Source](https://togithub.com/httptoolkit/react-reverse-portal/compare/v2.1.2...v2.2.0)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkRhdGFEaXNjb3ZlcnkiLCJiYWNrcG9ydDphbGwtb3BlbiIsInJlbGVhc2Vfbm90ZTpza2lwIl19-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/196584
## Summary
This PR updates the task poll interval logic for projects using the mget
strategy to optimize request loads to Elasticsearch, particularly for
smaller projects with low utilization. When task manager (TM)
utilization is below 25%, the poll interval will be set to 3 seconds
instead of the current 500 milliseconds. This change does not affect
projects utilizing `update_by_query`.
The existing backpressure logic remains unchanged for handling errors.
The only adjustment occurs in scenarios where there are no errors, the
TM utilization is below 25%, and the poll interval is less than 3
seconds. In such cases, the poll interval will increase to 3 seconds,
even if the backpressure logic has not fully reset the interval to its
original value.
I just chose 25%, but I am definitely open to other ideas.
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To verify
- Start Kibana and go to
`http://localhost:5601/api/task_manager/_health` and verify the poll
interval is 3s
- Create some alerting rules scheduled to run every second, and let them
run. I created 4 rules.
- Check `http://localhost:5601/api/task_manager/_health` again to verify
that with rules running the poll interval is back to 500ms. (It may take
a couple refreshes for the health api to reflect the changes)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
These tests are no longer needed with the removal of enterprise search
apps
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The other day when the https://github.com/elastic/kibana/pull/208546 was
opened, no required review was triggered for the changes in
`x-pack/test/security_solution_cypress/cypress/tasks/api_calls/common.ts`.
This is a problem, since any change in that file can have a huge impact
in the overall Cypress test suite.
In this PR we are adding security engineering productivity as owners of
that file to make sure that changes there are properly reviewed.
Resolves https://github.com/elastic/response-ops-team/issues/221
## Summary
Looking through the logs for this, the majority of these errors have the
following stack trace (14,000+ in the last 7 days):
```
ResponseError: search_phase_execution_exception
Root causes:
no_shard_available_action_exception: [es-es-search-686bf6b747-xldl8][100.65.77.183:9300][indices:data/read/search[phase/query]]
no_shard_available_action_exception: null
at KibanaTransport._request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:543:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at /usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:641:32
at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:637:20)
at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:60:16)
at ClientTraced.SearchApi [as search] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/search.js:72:12)
at getTotalAlertsCountAggregations (/usr/share/kibana/node_modules/@kbn/alerting-plugin/server/usage/lib/get_telemetry_from_alerts.js:42:21)
at async Promise.all (index 6)
at TaskManagerRunner.run (/usr/share/kibana/node_modules/@kbn/task-manager-plugin/server/task_running/task_runner.js:325:22)
```
Looking through the code, we are already catching these errors and
returning a default response for that telemetry object. The
`no_shard_available_action_exception` is not an issue with Kibana, it's
an ES issue, so this PR catches these types of errors and log them at a
debug level instead of a warn level to avoid polluting the logs with
errors we have no control over.
Excluding those results, we see a different and less frequent stack
trace (100+ in the last 15 days):
```
TypeError: Cannot read properties of undefined (reading 'by_rule_type_id')
at getTotalAlertsCountAggregations (/usr/share/kibana/node_modules/@kbn/alerting-plugin/server/usage/lib/get_telemetry_from_alerts.js:49:109)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Promise.all (index 6)
at TaskManagerRunner.run (/usr/share/kibana/node_modules/@kbn/task-manager-plugin/server/task_running/task_runner.js:325:22)
```
For actions telemetry, the volume of these errors is much lower.
Unfortunately, we are not logging the stack trace for these errors so
it's harder to track down the source. I've updated the code to store the
stack trace as well as adding the same handling for
`no_shard_available_action_exception` under the assumption that it could
also be getting these errors.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
As part of our effort to harden API action definitions and enforce
standards this PR adds an utility `ApiPrivileges` class.
It is supposed to be used for both feature registration and API route
definition to construct the privilege name.
```ts
plugins.features.registerKibanaFeature({
privileges: {
all: {
app: [...],
catalogue: [...],
api: [ApiPrivileges.manage('subject_name')],
...
},
read: {
...
api: [ApiPrivileges.read('subject_name')],
...
},
},
})
....
// route definition
router.get(
{
path: 'api_path',
security: {
authz: {
requiredPrivileges: [ApiPrivileges.manage('subject_name')],
},
},
},
async (ctx, req, res) => {}
);
```
`require_kibana_feature_privileges_naming` eslint rule has been added to
show warning if the API privilege name doesn't satisfy the naming
convention.
### Naming convention
- API privilege should start with valid `ApiOperation`: `manage`,
`read`, `update`, `delete`, `create`
- API privilege should use `_` as separator
❌ `read-entity-a`
❌ `delete_entity-a`
❌ `entity_manage`
✅ `read_entity_a`
✅ `delete_entity_a`
✅ `manage_entity`
> [!IMPORTANT]
> Serverless ZDT update scenario:
>
> - version N has an endpoint protected with the `old_privilege_read`.
> - version N+1 has the same endpoint protected with a new
`read_privilege`.
>
> There might be a short period between the time the UI pod N+1 passes
SO migrations and updates privileges and the time it's marked as
ready-to-handle-requests by k8s, and when UI pod N is terminated.
>
> After discussion with @legrego and @azasypkin we decided to ignore it
due to the perceived risk-to-cost ratio:
> 1. The time window users might be affected is very narrow because we
register privileges late in the Kibana startup flow (e.g., after SO
migrations).
> 2. The transient 403 errors users might get won't result in session
termination and shouldn't lead to data loss.
> 3. The roll-out will be performed in batches over the course of
multiple weeks and implemented by different teams. This means the impact
per release shouldn't be significant.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Relates: https://github.com/elastic/kibana/issues/198716__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [moment-timezone](http://momentjs.com/timezone/)
([source](https://togithub.com/moment/moment-timezone)) | dependencies |
patch | [`^0.5.46` ->
`^0.5.47`](https://renovatebot.com/diffs/npm/moment-timezone/0.5.46/0.5.47)
|
---
### Release Notes
<details>
<summary>moment/moment-timezone (moment-timezone)</summary>
###
[`v0.5.47`](https://togithub.com/moment/moment-timezone/blob/HEAD/changelog.md#0547-2025-01-28)
[Compare
Source](https://togithub.com/moment/moment-timezone/compare/0.5.46...0.5.47)
- Updated data to IANA TZDB `2025a`.
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOlNoYXJlZFVYIiwiYmFja3BvcnQ6YWxsLW9wZW4iLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR fixes the missing highlight theme for HJSON specs in Vega.
The issue https://github.com/elastic/kibana/issues/205711 was caused by
https://github.com/elastic/kibana/pull/182348 that inadvertently
specified the code-editor package as sideEffect free. This cause the
compiler to exclude every import without exported and used methods.
This was the case for the code-editor that registered some language
highlighters in that way.
The solution adopted here is to mark the register_language.ts file as a
file with side effects, the alternative solution can be to register
these directly from within the code_editor component.
A third option is to move these registration within the monaco package
where other languages are also registered.
I'd like to leave to @elastic/appex-sharedux the preference to followup
with a better fix for the future.
I'd also like to have the opinion from @elastic/kibana-operations
because the misconfigured package issue can be seen only in production
and not in development mode. Is it possible that webpack doesn't apply
treeshaking when in development mode?
fix https://github.com/elastic/kibana/issues/205711
Should also fix the same issue but for TSVB Markdown
The fix was tested on CI by running at first only the CI FT with the
`sideEffects:false` to verify the failure (see
[build](https://buildkite.com/elastic/kibana-pull-request/builds/272375))
and then with the fix specifying the actual file that contains side
effects.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes the bug of the invalid event filter created automatically when
creating a cloud workloads endpoint integration. The issue was a type
issue: `undefined` or an object is expected, instead an array was
passed.
To make sure this does not happen again, the type for the `meta` field
was updated from the deprecated `t.object` to `t.UnknownRecord`, which
is able to catch similar issues as a type error:
ca0c01b63b
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
