## Summary
This PR adds some basic integration tests. I also had to modify the
Entity schemas to allow for parsing the dynamic metadata fields. I don't
love the type solution and would ideally like to have something like:
```typescript
const entitySchema = createEntitySchemaFromDefintion(entityDefinintion);
```
which would dynamically generate an entity schema from the definition
based on the identity and metadata fields. I have a prototype that sort
of works but still needs a lot of work and started to become a huge
"time suck".
Closes [139](https://github.com/elastic/elastic-entity-model/issues/139)
Prior to introducing node variants, we were cleaning up unused files
included with Node.js. The node paths changed, and the cleanup paths
were not. This fixes the issue.
## Testing
`$KIBANA_HOME/node/*` should be cleaned of unused files, see the
patterns in the diff for exact files.
This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [terser](https://terser.org)
([source](https://togithub.com/terser/terser)) | devDependencies | patch
| [`^5.31.3` ->
`^5.31.4`](https://renovatebot.com/diffs/npm/terser/5.31.3/5.31.4) |
`5.31.6` (+1) |
---
### Release Notes
<details>
<summary>terser/terser (terser)</summary>
###
[`v5.31.4`](https://togithub.com/terser/terser/blob/HEAD/CHANGELOG.md#v5314-reverted)
[Compare
Source](https://togithub.com/terser/terser/compare/v5.31.3...v5.31.4)
- drop_unused: drop classes which only have side effects in the
`extends` part
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJyZWxlYXNlX25vdGU6c2tpcCJdfQ==-->
---------
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: Jonathan Budzenski <jon@elastic.co>
Adds the GitHub action `launchdarkly/find-code-references` to the
LaunchDarkly group of dependencies.
A pull request will be opened shortly after introducing sha pinning to
the dependency.
## Summary
Adding new service that acts as a wrapper of `supertestWithoutAuth`
service authenticated with role-based API key and enriched with request
headers.
The proposed change streamlines test design by centralizing the
management of API key and internal/common headers, eliminating the need
to pass these arguments individually in every API call. This approach
reduces code duplication and enhances maintainability
Before:
```ts
const samlAuth = getService('samlAuth');
const supertestWithoutAuth = getService('supertestWithoutAuth');
const roleAuthc = await samlAuth.createM2mApiKeyWithRoleScope('admin');
const internalHeaders = samlAuth.getInternalRequestHeader();
await supertestWithoutAuth
.get('/api/console/api_server')
.set(roleAuthc.apiKeyHeader)
.set(internalHeaders)
.set('kbn-xsrf', 'true')
.expect(200);
```
After:
```ts
const roleScopedSupertest = getService('roleScopedSupertest');
const supertestWithAdminScope = await roleScopedSupertest.getSupertestWithRoleScope('admin', {
withInternalHeaders: true,
withCustomHeaders: {'kbn-xsrf': 'true'},
});
await supertestWithAdminScope
.get('/api/console/api_server')
.expect(200);
```
Use this service to easily test API endpoints with role-specific
authorization and custom headers, both in serverless and stateful
environments.
closes#190228
---------
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
This PR adds support for using `zod` as the validation library alongside
of `io-ts`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
i noticed few extra calls which can be prevented during package policy
updates/creation while investigating synthetics project monitor API call
!!
---------
Co-authored-by: Nicolas Chaulet <n.chaulet@gmail.com>
## Summary
Adds documentation for how we currently test our validation and
autocomplete engines.
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/190320 and
https://github.com/elastic/kibana/issues/190396
- Start investigation from Custom threshold alert details page
- Go to ongoing investigation instead of creating new one if one already
exists
- Initial investigation status is set as `ongoing`
- Investigation origin is set as `alert`
"Start investigation" is hidden for other alert types and when
investigate plugin is disabled.
### Testing
- Add the following in `kibana.dev.yml`
```
xpack.investigate.enabled: true
xpack.investigateApp.enabled: true
```
- Create Custom threshold rule
- Open Custom threshold alert details page
- Click on "Start investigation"
- Verify that a new saved object is created for the investigation
https://github.com/user-attachments/assets/6dfe8a5f-287b-4cc5-92ae-e4c315c7420b
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kevin Delemme <kdelemme@gmail.com>
Closes:
https://github.com/elastic/observability-accessibility/issues/134
## Description
`Home` > `Cases` - Dropdown label does not inform user of related
results
### Steps to recreate
Pre-requisitie:
Screen reader is activated
1. Navigate to the `Serverless Observability Solution `instance.
2. Click `Cases` in Left Nav
3. Keyboard navigate to 'Severity', 'Status', et al. dropdowns
Expected Result:
-Screen reader reads dropdown header as well as number of entries in the
dropdown, indicated by the number in the header
Actual Result:
-Screen reader reads only the category label
### What was changed:
1. Removed extra `aria-label` attribute. `EuiFilterButton` component is
responsible for setting correct `aria-label` based on internal state.
### Screen
<img width="1296" alt="image"
src="https://github.com/user-attachments/assets/3051e71d-5890-44c4-a2b1-bd47a71edade">
## [Security Solution] Updates the `test:generate:serverless-dev` script to align with the latest serverless defaults
This PR updates the `test:generate:serverless-dev` script, (which runs `scripts/endpoint/resolver_generator`), to align with the latest serverless defaults.
After this change, running the following command:
```sh
yarn test:generate:serverless-dev
```
populates a local serverless project with generated alerts and events (for testing).
### Desk testing
1) Start a new (development) instance of Elasticsearch:
```sh
yarn es serverless --clean --projectType security
```
2) Start a local (development) instance of Kibana:
```sh
yarn start --serverless=security --no-base-path
```
3) Login to the project as the `admin` user
4) Install the default detection rules
5) Change to the `$KIBANA_HOME/x-pack/plugins/security_solution` directory:
```sh
cd $KIBANA_HOME/x-pack/plugins/security_solution
```
6) Run the following command:
```sh
yarn test:generate:serverless-dev
```
**Expected result**
- The `resolver_generator` script populates the serverless project with alerts and events, as illustrated by the output below:
```
info Build flavor: serverless
info Indexing host and alerts...
info Creating and indexing documents took: 15098ms
✨ Done in 17.37s.
```
**Actual result**
- The `resolver_generator` script does NOT populate the serverless project with alerts and events, as illustrated by the output below:
```
info Build flavor: serverless
info Indexing host and alerts...
Unhandled Promise rejection detected:
EndpointDataLoadingError: other side closed - Local: 127.0.0.1:63688, Remote: 127.0.0.1:9200
at wrapErrorIfNeeded (utils.ts:29:7)
at wrapErrorAndRejectPromise (utils.ts:32:75)
at index_endpoint_hosts.ts:259:26 {
meta: ConnectionError: other side closed - Local: 127.0.0.1:63688, Remote: 127.0.0.1:9200
at SniffingTransport.request (/Users/$USER/Projects/forks/$USER/kibana/node_modules/@elastic/transport/src/Transport.ts:630:17)
at Client.BulkApi [as bulk] (/Users/$USER/Projects/forks/$USER/kibana/node_modules/@elastic/elasticsearch/src/api/api/bulk.ts:75:10)
at index_endpoint_hosts.ts:259:26 {
options: { redaction: [Object] },
meta: {
body: undefined,
statusCode: 0,
headers: {},
meta: [Object],
warnings: null
}
}
}
Terminating process...
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
```
## Summary
Relates to - https://github.com/elastic/kibana/issues/184572
Figma Design -
https://www.figma.com/design/8WVWLeVn8mvoUm0VGgbSbB/Data-set-quality-V2?node-id=3564-73485&t=KADTdNFiiOBJ7rOS-0
**NOTE: This PR is part of a multi series PRs. Hence expect it to not do
everything.**
### What are we going to do?
1. The content of the flyout, henceforth will be known as Dataset
Quality Details is being copied to a component with the same name.
2. This component can be initialised as page, like in Management app or
used as an individual component in a flyout in Unified Doc Viewer for
example. As scope of this PR, a page in Management app has been created
which will load this detailed component. A new route will be created
with breadcrumb.
3. This page will co-live with the Flyout for now, accessible only via
direct URL. In subsequent PR, when we remove the Flyout completely, we
will change the action in Dataset Quality to instead of opening a Flyout
to navigating to this new page.
### What's in this PR ?
1. As part of this change, i have created a complete new State Machine,
Controller for Dataset Quality Details component which is responsible
for replicating the Flyout.
2. A dedicated route registered under `/details` where this component
will live at the moment. Sample URL will look like this
`http://localhost:5601/pfd/app/management/data/data_quality/details?pageState=(dataStream:logs-synth.1-default,v:1)`
3. The individual components which currently load inside the flyout may
be duplicated for time being.
4. Validation when no data stream provided.
5. Breadcrumb for the Management page
### What's not in this PR
1. Tests needs to be migrated, they will be done as part of the Next PR
2. Telemetry for Flyout has been removed. It will be added as part of
next PR.
3. Existing Flyout code has not be removed. That needs to be removed and
the old state machine needs to be meticulously cleaned.
6. Swapping the Click to Open Flyout to Page needs to be done when the
above 3 are ready.
## Screenshot
### Good scenario
<img width="1482" alt="image"
src="https://github.com/user-attachments/assets/4409eb57-89d5-477c-a946-1b7a45df074c">
### When datastream does not exist
<img width="1527" alt="image"
src="https://github.com/user-attachments/assets/66d735aa-8f0f-4fb8-b57c-4d22cecad2c7">
### When invalid state is provided by the page, it redirects to parent
Dataset Quality Page
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/176445
## Summary
Flaky test runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6679
This PR fixes the advanced settings API integration tests that seem to
be flaky. The reason for the occasional failures is most likely a
`version_conflict_engine_exception` which is thrown when another node
indexes the same documents. This can happen when we save an advanced
setting since the settings API uses saved objects under the hood, and in
CI, multiple nodes can try to save an advanced setting simultaneously.
The solution in this PR is to retry the request if we encounter a 409
error. This is adapted from the solution in
https://github.com/elastic/kibana/pull/174185 which resolves a similar
failure.
## Summary
Runtime mappings should be obtained from data view spec, we are removing
this field from the sourcerer model and all the uses will depend on the
embedded spec from now on.
Closes:
https://github.com/elastic/observability-accessibility/issues/137
## Description
`Home` > `Project Settings` > `Integrations`- Radio buttons
(Recommended, Elastic Agent, Beats) do not have context conveyed via
screen reader
### Steps to recreate
Pre-requisite:
Screen reader is activated
1. Navigate to the `Serverless Observability Solution `instance.
2. Click `Project Settings` in lower Left Nav
3. Select `Integrations`
4. Keyboard navigate to the Recommended, Elastic Agent, Beats radio
buttons
5. Tab through the list
Expected Result:
-Screen reader user understands the meaning and context of the radio
button options.
Actual Result:
-Screen reader reads radio buttons but is not provided the context of
what they relate to.
### Screens
<img width="1296" alt="image"
src="https://github.com/user-attachments/assets/fdca5a9b-46d9-44a2-a12f-f50134439333">
Closes https://github.com/elastic/kibana/issues/187921
The Obs AI Assistant calls the endpoint `GET _connector` to get a list
of connector indices. This works for admin users but not for users with
limited privileges like users with the `editor` role.
Currently an error is thrown but never caught. This causes the kibana
server to crash during development. In prod this problem means that
users cannot retrieve connector indices, and thus fallback to querying
`search-*`.
This PR fixes it by using the internal user to call `GET _connector`.
Additional context:
https://elastic.slack.com/archives/C05J7LXR5DE/p1723560268104559
## Summary
It closes https://github.com/elastic/security-team/issues/10104
This PR addresses the necessary changes in Kibana to reflect the updated
agentless API settings in kibana.yml settings:
- **xpack.fleet.agentless.enabled: true**
Changing to use `xpack.fleet.agentless.enabled` as a Feature flag for
enabling agentless in ESS environments. (Serverless changes was out of
scope, so it is still using `xpack.fleet.enableExperimental` for now).
Also, this value will now be exposed to the client side instead of the
API URL (It closes
https://github.com/elastic/security-team/issues/10019).
- **xpack.fleet.agentless.api.url**
This setting will now include only the URL (without the basePath), so I
added a `prependAgentlessApiBasePathToEndpoint` helper function to
append the basePath with type support. Also this value will no longer be
exposed to the client side
### Additional changes
- This PR also modified the Agentless settings schema validation to be
optional (so it will only validate the types), as because of the nature
of how the `kibana.yml` will set the configuration as of
[this](https://github.com/elastic/cloud/pull/130314) PR, it might happen
that we will have partial settings, so changing the schema to optional
will prevent Kibana failing from starting due to incomplete agentless
settings. (It closes
https://github.com/elastic/security-team/issues/10189)
- Updated unit tests and FTR e2e config files to reflect the latest
changes
