## Summary
Functional tests and API integration tests that trigger failures are
problematic. These have the potential to run in a monitored environment.
Engineers that are on-call and watching for failures in HTTP responses
could see these as a false positive of an actual problem.
1. Remove Reporting API integration that generally tests for error
codes. These tests are suitably covered in jest integration tests.
2. Reorganize Screenshotting tests that test for error codes. WIP: find
out if these can be covered in jest integration tests and if so, remove
them
3. Reorganize tests of export types by creating `csv` and `screenshot`
directories. This change facilitates finer-grained code ownership.
## Summary
Resolves#143905. This PR adds support for integration-level outputs.
This means that different integrations within the same agent policy can
now be configured to send data to different locations. This feature is
gated behind `enterprise` level subscription.
For each input, the agent policy will configure sending data to the
following outputs in decreasing order of priority:
1. Output set specifically on the integration policy
2. Output set specifically on the integration's parent agent policy
(including the case where an integration policy belongs to multiple
agent policies)
3. Global default data output set via Fleet Settings
Integration-level outputs will respect the same rules as agent
policy-level outputs:
- Certain integrations are disallowed from using certain output types,
attempting to add them to each other via creation, updating, or
"defaulting", will fail
- `fleet-server`, `synthetics`, and `apm` can only use same-cluster
Elasticsearch output
- When an output is deleted, any integrations that were specifically
using it will "clear" their output configuration and revert back to
either `#2` or `#3` in the above list
- When an output is edited, all agent policies across all spaces that
use it will be bumped to a new revision, this includes:
- Agent policies that have that output specifically set in their
settings (existing behavior)
- Agent policies that contain integrations which specifically has that
output set (new behavior)
- When a proxy is edited, the same new revision bump above will apply
for any outputs using that proxy
The final agent policy YAML that is generated will have:
- `outputs` block that includes:
- Data and monitoring outputs set at the agent policy level (existing
behavior)
- Any additional outputs set at the integration level, if they differ
from the above
- `outputs_permissions` block that includes permissions for each
Elasticsearch output depending on which integrations and/or agent
monitoring are assigned to it
Integration policies table now includes `Output` column. If the output
is defaulting to agent policy-level output, or global setting output, a
tooltip is shown:
<img width="1392" alt="image"
src="https://github.com/user-attachments/assets/5534716b-49b5-402a-aa4a-4ba6533e0ca8">
Configuring an integration-level output is done under Advanced options
in the policy editor. Setting to the blank value will "clear" the output
configuration. The list of available outputs is filtered by what outputs
are available for that integration (see above):
<img width="799" alt="image"
src="https://github.com/user-attachments/assets/617af6f4-e8f8-40b1-b476-848f8ac96e76">
An example of failure: ES output cannot be changed to Kafka while there
is an integration
<img width="1289" alt="image"
src="https://github.com/user-attachments/assets/11847eb5-fd5d-4271-8464-983d7ab39218">
## TODO
- [x] Adjust side effects of editing/deleting output when policies use
it across different spaces
- [x] Add API integration tests
- [x] Update OpenAPI spec
- [x] Create doc issue
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## [Security Solution] [Attack discovery] Attack Discovery RBAC / Display an upgrade CTA for the serverless essentials product tier
### Summary
This PR adds Role Based Access Control (RBAC) to Attack discovery.
Security users may enable or disable the new `Attack Discovery` RBAC feature shown in the figure below:
_Above: The new `Attack discovery` RBAC feature_
It is possible to for example, configure a custom role that enables Attack discovery, but disables the assistant, as illustrated by the table below:
| Role | License | Navigation visible | Show upsell | Upsell has actions | View in assistant enabled |
|-------------------------------------------|------------|--------------------|-------------|--------------------|---------------------------|
| `has_attack_discovery_all_assistant_none` | Basic | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_all_assistant_none` | Trial | ✅ | ❌ | ❌ | ❌ |
| `has_attack_discovery_all_assistant_none` | Platinum | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_all_assistant_none` | Enterprise | ✅ | ❌ | ❌ | ❌ |
_Above: An example role that enables Attack discovery, but disables the assistant_
See the `Desk Testing` section of this PR for details.
This PR also fixes an issue where Attack discovery does not display an upgrade call to action (CTA) for the serverless _essentials_ product tier, per the before and after screenshots below:
#### Before
_Above: Before the fix, an upgrade CTA is NOT displayed for the serverless essentials product tier_
#### After
_Above: After the fix, an upgrade CTA is displayed for the serverless essentials product tier_
The fix above is implemented by adopting the upselling framework.
### New Feature ID
This PR adds a new Feature ID for attack discovery:
```typescript
export const ATTACK_DISCOVERY_FEATURE_ID = 'securitySolutionAttackDiscovery' as const;
```
in `x-pack/packages/security-solution/features/src/constants.ts`
### Upselling framework usage
This PR updates the Attack discovery page to use the upselling framework via the following summarized steps:
1. Removed the branching logic from `x-pack/plugins/security_solution/public/attack_discovery/pages/upgrade/index.tsx`, and moved the component to an upselling `section` component in `x-pack/packages/security-solution/upselling/sections/attack_discovery/index.tsx`, where the component was renamed to `AttackDiscoveryUpsellingSection`.
This `section` component handles (just) the styling of the upselling message and actions (by itself, without the page wrapper), and receives the following props:
```typescript
interface Props {
actions?: React.ReactNode;
availabilityMessage: string;
upgradeMessage: string;
}
```
The self managed and serverless-specific actions and `i18n` messages are passed down via the components described in the later steps below.
2. Removed all previous references to the `Upgrade` component (and references to `useProductTypes`) from the Attack discovery page in `x-pack/plugins/security_solution/public/attack_discovery/pages/index.tsx`, because the framework manages the upgrade case instead of the page itself.
3. Created an upselling `page` component `AttackDiscoveryUpsellingPage` in `x-pack/packages/security-solution/upselling/pages/attack_discovery/index.tsx`.
This component handles (just) the styling of the _page_ that wraps the Attack discovery `section`. It passes the same props to the previously described `AttackDiscoveryUpsellingSection` component.
4. Created a self-managed-specific `AttackDiscoveryUpsellingPageESS` component in `x-pack/plugins/security_solution_ess/public/upselling/pages/attack_discovery/index.tsx`
This component passes self-managed-specific upgrade action buttons / links and `i18n` strings to the previously described `AttackDiscoveryUpsellingPage`
5. Also for self managed, added a new `AttackDiscoveryUpsellingPageLazy` component to the existing file: `x-pack/plugins/security_solution_ess/public/upselling/lazy_upselling.tsx`
This component lazy loads the previously described `AttackDiscoveryUpsellingPageESS` component.
6. Added registration for the previously described `AttackDiscoveryUpsellingPageLazy` component to the existing `UpsellingPages` section in `x-pack/plugins/security_solution_ess/public/upselling/register_upsellings.tsx` with a `minimumLicenseRequired` of `enterprise`:
```
minimumLicenseRequired: 'enterprise',
```
7. Created a serverless-specific `AttackDiscoveryUpsellingPageServerless` component in `x-pack/plugins/security_solution_serverless/public/upselling/pages/attack_discovery/index.tsx`
This component passes serverless-specific `i18n` messages to the platform agnostic `AttackDiscoveryUpsellingPage` component.
8. Also for serverless, added a new `AttackDiscoveryUpsellingPageLazy` component to the existing file: `x-pack/plugins/security_solution_serverless/public/upselling/lazy_upselling.tsx`
9. Added registration for the previously described `AttackDiscoveryUpsellingPageLazy` component to the existing `upsellingPages` section in `x-pack/plugins/security_solution_serverless/public/upselling/upsellings.tsx` with the `assistant` PLI:
```
pli: ProductFeatureKey.assistant,
```
10. Added the `${ASSISTANT_FEATURE_ID}.ai-assistant` capability as an OR condition (via nested array, per the [framework](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/public/common/lib/capabilities/has_capabilities.ts#L11-L22)) to the Attack discovery link in `x-pack/plugins/security_solution/public/attack_discovery/links.ts`. This addition enables the security route wrapper to check for upselling pages in serverless:
```
capabilities: [[`${SERVER_APP_ID}.show`, `${ASSISTANT_FEATURE_ID}.ai-assistant`]],
```
11. Added `licenseType: 'enterprise'` to the Attack discovery link in `x-pack/plugins/security_solution/public/attack_discovery/links.ts` to require an `enterprise` license for self managed
### Upgrade CTA gallery
The screenshots in this section document the CTA (or Welcome message when the feature is licensed) displayed for various license levels after the fix:
#### Users with the `None` privilege
If users with the `None` privilege manually enter an Attack discovery URL, e.g. `http://localhost:5601/app/security/attack_discovery`, the framework will display the following error prompt:
#### Self managed BASIC
#### Self managed PLATINUM
#### Self managed TRIAL
#### Self managed ENTERPRISE
#### Serverless ESSENTIALS
#### Serverless COMPLETE
## Desk Testing
### Severless: Desk testing (just) the upgrade CTA
Reproduction steps:
1) Comment-out any preconfigured connectors in `config/kibana.dev.yml`
2) Edit the configuration of `config/serverless.security.yml` to enable the `essentials` product tier:
```yaml
xpack.securitySolutionServerless.productTypes:
[
{ product_line: 'security', product_tier: 'essentials' }
]
```
3) Start Elasticsearch
```sh
yarn es serverless --projectType security
```
4) Start a development instance of Kibana
```
yarn start --serverless=security --no-base-path
```
5) Navigate to Security > Attack discovery
**Expected result**
- An upgrade CTA is displayed, as illustrated by the _after_ screenshot below:
- The video tour is NOT displayed for serverless, as noted in the [original PR](https://github.com/elastic/kibana/pull/182605#issuecomment-2100607857)
**Actual result**
- An upgrade CTA is NOT displayed, as illustrated by the _before_ screenshot below:
### Desk testing Self Managed
To desk test self manged, we will:
1) Create (three) roles for testing
2) Create (three) users assigned to the roles
3) Test each role at `Basic`, `Trial`, `Platinum`, and `Enterprise` levels to verify:
- `Attack discovery` link visibility in the Security solution navigation
- Visibility of the upsell empty prompt for license levels where Attack discovery is unavailable
- The upsell empty prompt includes the `Subscription plans` and `Manage license` actions
- When Attack discoveries are generated, the `View in Ai Assistant` button and popover menu action are enabled / disabled, based on availability of the `AI Assistant` feature
#### Creating (three) roles for testing
In this section, we will start a new (development) self managed deployment, and create the following three roles via Kibana Dev Tools:
- `has_attack_discovery_all_assistant_all`
- `has_attack_discovery_all_assistant_none`
- `has_attack_discovery_none_assistant_all`
To start the deployment and create the roles:
1) Add a pre-configured GenAI connector to `config/kibana.dev.yml`
2) Start a new (development) instance of Elasticsearch:
```sh
yarn es snapshot -E path.data=/Users/$USERNAME/data-2024-07-31a
```
3) Start a local (development) instance of Kibana:
```
yarn start --no-base-path
````
4) Login to Kibana as the `elastic` user
5) Generate some alerts
6) Navigate to Dev Tools
7) Execute the following three API calls to create the roles:
<details><summary>PUT /_security/role/has_attack_discovery_all_assistant_all</summary>
<p>
``` ts
PUT /_security/role/has_attack_discovery_all_assistant_all
{
"cluster": [
"all"
],
"indices": [
{
"names": [
"*"
],
"privileges": [
"all"
],
"field_security": {
"grant": [
"*"
],
"except": []
},
"allow_restricted_indices": false
}
],
"applications": [
{
"application": "kibana-.kibana",
"privileges": [
"feature_securitySolutionAssistant.minimal_all",
"feature_securitySolutionAttackDiscovery.minimal_all",
"feature_siem.all",
"feature_securitySolutionCases.all",
"feature_actions.all"
],
"resources": [
"*"
]
}
],
"run_as": [],
"metadata": {},
"transient_metadata": {
"enabled": true
}
}
```
</p>
</details>
<details><summary>PUT /_security/role/has_attack_discovery_all_assistant_none</summary>
<p>
``` ts
PUT /_security/role/has_attack_discovery_all_assistant_none
{
"cluster": [
"all"
],
"indices": [
{
"names": [
"*"
],
"privileges": [
"all"
],
"field_security": {
"grant": [
"*"
],
"except": []
},
"allow_restricted_indices": false
}
],
"applications": [
{
"application": "kibana-.kibana",
"privileges": [
"feature_securitySolutionAttackDiscovery.minimal_all",
"feature_siem.all",
"feature_securitySolutionCases.all",
"feature_actions.all"
],
"resources": [
"*"
]
}
],
"run_as": [],
"metadata": {},
"transient_metadata": {
"enabled": true
}
}
```
</p>
</details>
<details><summary>PUT /_security/role/has_attack_discovery_none_assistant_all</summary>
<p>
``` ts
PUT /_security/role/has_attack_discovery_none_assistant_all
{
"cluster": [
"all"
],
"indices": [
{
"names": [
"*"
],
"privileges": [
"all"
],
"field_security": {
"grant": [
"*"
],
"except": []
},
"allow_restricted_indices": false
}
],
"applications": [
{
"application": "kibana-.kibana",
"privileges": [
"feature_securitySolutionAssistant.minimal_all",
"feature_siem.all",
"feature_securitySolutionCases.all",
"feature_actions.all"
],
"resources": [
"*"
]
}
],
"run_as": [],
"metadata": {},
"transient_metadata": {
"enabled": true
}
}
```
</p>
</details>
#### Creating (three) users assigned to the roles
In this section, we will create the following three users via Kibana Dev Tools using the API calls below (expand for details):
- `attack_discovery_all_assistant_all`
- `attack_discovery_all_assistant_none`
- `attack_discovery_none_assistant_all`
1) Navigate to Dev Tools
2) Execute the following three API calls to create the users:
<details><summary>POST /_security/user/attack_discovery_all_assistant_all</summary>
<p>
``` ts
POST /_security/user/attack_discovery_all_assistant_all
{
"username": "attack_discovery_all_assistant_all",
"password": "changeme",
"roles": [
"has_attack_discovery_all_assistant_all"
],
"full_name": "Attack Discovery All Assistant All",
"email": "user@example.com",
"metadata": {},
"enabled": true
}
```
</p>
</details>
<details><summary>POST /_security/user/attack_discovery_all_assistant_none</summary>
<p>
``` ts
POST /_security/user/attack_discovery_all_assistant_none
{
"username": "attack_discovery_all_assistant_none",
"password": "changeme",
"roles": [
"has_attack_discovery_all_assistant_none"
],
"full_name": "Attack Discovery All Assistant None",
"email": "user@example.com",
"metadata": {},
"enabled": true
}
```
</p>
</details>
<details><summary>POST /_security/user/attack_discovery_none_assistant_all</summary>
<p>
``` ts
POST /_security/user/attack_discovery_none_assistant_all
{
"username": "attack_discovery_none_assistant_all",
"password": "changeme",
"roles": [
"has_attack_discovery_none_assistant_all"
],
"full_name": "Attack Discovery None Assistant All",
"email": "user@example.com",
"metadata": {},
"enabled": true
}
```
</p>
</details>
#### Testing each role at `Basic`, `Trial`, `Platinum`, and `Enterprise` levels
In this section, we will test each of the self managed `Basic`, `Trial`, `Platinum`, and `Enterprise` license levels with the three roles we created for testing.
##### Testing the `has_attack_discovery_all_assistant_all` role
1) Login as the `attack_discovery_all_assistant_all` user
2) Navigate to the Security solution
3) For each of the `Basic`, `Trial`, `Platinum`, and `Enterprise` levels, verify your observations match the expected behavior in the table below:
| Role | License | Navigation visible | Show upsell | Upsell has actions | View in assistant enabled |
|------------------------------------------|------------|--------------------|-------------|--------------------|---------------------------|
| `has_attack_discovery_all_assistant_all` | Basic | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_all_assistant_all` | Trial | ✅ | ❌ | ❌ | ✅ |
| `has_attack_discovery_all_assistant_all` | Platinum | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_all_assistant_all` | Enterprise | ✅ | ❌ | ❌ | ✅ |
##### Testing the `has_attack_discovery_all_assistant_none` role
1) Login as the `attack_discovery_all_assistant_none` user
2) Navigate to the Security solution
3) For each of the `Basic`, `Trial`, `Platinum`, and `Enterprise` levels, verify your observations match the expected behavior in the table below:
| Role | License | Navigation visible | Show upsell | Upsell has actions | View in assistant enabled |
|-------------------------------------------|------------|--------------------|-------------|--------------------|---------------------------|
| `has_attack_discovery_all_assistant_none` | Basic | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_all_assistant_none` | Trial | ✅ | ❌ | ❌ | ❌ |
| `has_attack_discovery_all_assistant_none` | Platinum | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_all_assistant_none` | Enterprise | ✅ | ❌ | ❌ | ❌ |
##### Testing the `has_attack_discovery_none_assistant_all` role
1) Login as the `attack_discovery_none_assistant_all` user
2) Navigate to the Security solution
3) For each of the `Basic`, `Trial`, `Platinum`, and `Enterprise` levels, verify your observations match the expected behavior in the table below:
| Role | License | Navigation visible | Show upsell | Upsell has actions | View in assistant enabled |
|-------------------------------------------|------------|--------------------|-------------|--------------------|---------------------------|
| `has_attack_discovery_none_assistant_all` | Basic | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_none_assistant_all` | Trial | ❌ | ❌ | ❌ | ❌ |
| `has_attack_discovery_none_assistant_all` | Platinum | ✅ | ✅ | ✅ | ❌ |
| `has_attack_discovery_none_assistant_all` | Enterprise | ❌ | ❌ | ❌ | ❌ |
---------------------------------------------
### Serverless Testing
To desk test serverless, we will test the `essentials` and `complete` product tiers to verify:
- `Attack discovery` link visibility in the Security project navigation
- Visibility of the upsell empty prompt for license levels where Attack discovery is unavailable
- The upsell empty prompt does NOT include the `Subscription plans` and `Manage license` actions
- When Attack discoveries are generated, the `View in Ai Assistant` button and popover menu action are enabled
#### Essentials tier testing
1) Add a pre-configured GenAI connector to `config/kibana.dev.yml`
2) Edit the configuration of `config/serverless.security.yml` to enable the `essentials` product tier:
```yaml
xpack.securitySolutionServerless.productTypes:
[
{ product_line: 'security', product_tier: 'essentials' },
{ product_line: 'endpoint', product_tier: 'essentials' },
]
```
2) Start a new (development) instance of Elasticsearch:
```sh
yarn es serverless --clean --projectType security
```
3) Start a local (development) instance of Kibana:
```
yarn start --serverless=security --no-base-path
````
4) select the `admin` role
5) Generate some alerts
6) Verify your observations match the expected behavior in the table below:
| Role | Tier | Navigation visible | Show upsell | Upsell has actions | View in assistant enabled |
|-------------------------------|------------|--------------------|-------------|--------------------|---------------------------|
| `viewer` | essentials | ✅ | ✅ | ❌ | ❌ |
| `editor` | essentials | ✅ | ✅ | ❌ | ❌ |
| `t1_analyst` | essentials | ✅ | ✅ | ❌ | ❌ |
| `t2_analyst` | essentials | ✅ | ✅ | ❌ | ❌ |
| `t3_analyst` | essentials | ✅ | ✅ | ❌ | ❌ |
| `threat_intelligence_analyst` | essentials | ✅ | ✅ | ❌ | ❌ |
| `rule_author` | essentials | ✅ | ✅ | ❌ | ❌ |
| `soc_manager` | essentials | ✅ | ✅ | ❌ | ❌ |
| `detections_admin` | essentials | ✅ | ✅ | ❌ | ❌ |
| `platform_engineer` | essentials | ✅ | ✅ | ❌ | ❌ |
| `endpoint_operations_analyst` | essentials | ✅ | ✅ | ❌ | ❌ |
| `endpoint_policy_manager` | essentials | ✅ | ✅ | ❌ | ❌ |
| `admin` | essentials | ✅ | ✅ | ❌ | ❌ |
| `system_indices_superuser` | essentials | ✅ | ✅ | ❌ | ❌ |
### Complete tier testing
1) Stop the running Kibana server (from the previous Essentials tier testing)
2) Edit the configuration of `config/serverless.security.yml` to enable the `complete` product tier:
```yaml
xpack.securitySolutionServerless.productTypes:
[
{ product_line: 'security', product_tier: 'complete' },
{ product_line: 'endpoint', product_tier: 'complete' },
]
```
3) Restart a local (development) instance of Kibana:
```
yarn start --serverless=security --no-base-path
````
4) Verify your observations match the expected behavior in the table below:
| Role | Tier | Navigation visible | Show upsell | Upsell has actions | View in assistant enabled |
|-------------------------------|----------|--------------------|-------------|--------------------|---------------------------|
| `viewer` | complete | ✅ | ❌ | ❌ | ✅ |
| `editor` | complete | ✅ | ❌ | ❌ | ✅ |
| `t1_analyst` | complete | ✅ | ❌ | ❌ | ✅ |
| `t2_analyst` | complete | ✅ | ❌ | ❌ | ✅ |
| `t3_analyst` | complete | ✅ | ❌ | ❌ | ✅ |
| `threat_intelligence_analyst` | complete | ✅ | ❌ | ❌ | ✅ |
| `rule_author` | complete | ✅ | ❌ | ❌ | ✅ |
| `soc_manager` | complete | ✅ | ❌ | ❌ | ✅ |
| `detections_admin` | complete | ✅ | ❌ | ❌ | ✅ |
| `platform_engineer` | complete | ✅ | ❌ | ❌ | ✅ |
| `endpoint_operations_analyst` | complete | ✅ | ❌ | ❌ | ✅ |
| `endpoint_policy_manager` | complete | ✅ | ❌ | ❌ | ✅ |
| `admin` | complete | ✅ | ❌ | ❌ | ✅ |
| `system_indices_superuser` | complete | ✅ | ❌ | ❌ | ✅ |
## Summary
This PR has breadth, but not depth. This adds 3 new `eslint` rules. The
first two protect against the use of code generated from strings (`eval`
and friends), which will not work client-side due to our CSP, and is not
something we wish to support server-side. The last rule aims to prevent
a subtle class of bugs, and to defend against a subset of prototype
pollution exploits:
- `no-new-func` to be compliant with our CSP, and to prevent code
execution from strings server-side:
https://eslint.org/docs/latest/rules/no-new-func
- `no-implied-eval` to be compliant with our CSP, and to prevent code
execution from strings server-side:
https://eslint.org/docs/latest/rules/no-implied-eval. Note that this
function implies that it prevents no-new-func, but I don't see [test
cases](https://github.com/eslint/eslint/blob/main/tests/lib/rules/no-implied-eval.js)
covering this behavior, so I think we should play it safe and enable
both rules.
- `no-prototype-builtins` to prevent accessing shadowed properties:
https://eslint.org/docs/latest/rules/no-prototype-builtins
In order to be compliant with `no-prototype-builtins`, I've migrated all
usages and variants of `Object.hasOwnProperty` to use the newer
[`Object.hasOwn`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/hasOwn).
## Summary
Includes alerting rule APIs in our OAS snapshots.
## How to test
Using bump CLI you can preview the output:
```sh
bump preview ./oas_docs/bundle.json
# or
bump preview ./oas_docs/bundle.serverless.json
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Another attempt at fixing github action pinning. This scopes us to the
actions/checkout package only for now, and we can expand it later.
Previously: https://github.com/elastic/kibana/pull/190377, which was
successful for the auto-approve-backport acttion.
## Summary
This PR tries to stabilize the Discover sidebar tests which started to
be flaky when running against MKI.
### Details
The serverless Discover sidebar tests (`Common Group
5.x-pack/test_serverless/functional/test_suites/common/discover/group6/_sidebar·ts`)
started to become flaky when running against MKI. 5 out of the last 10
runs failed for the same reason:
```
discover/group6 discover sidebar renders field groups should render even when retrieving documents failed with an error
Error: expected '48 available fields. 6 empty fields. 4 meta fields.' to equal '49 available fields. 5 empty fields. 4 meta fields.'
at Assertion.assert (expect.js💯11)
at Assertion.apply (expect.js:227:8)
at Assertion.be (expect.js:69:22)
at Context. (_sidebar.ts:644:83)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at Object.apply (wrap_function.js:73:16)
```
So it seems the newly added field is sometimes listed as `empty` instead
of `available`.
Failure screenshot:
The screenshot shows an active global loading indicator, so that might
be the reason for the flakiness (loading not quite done sometimes, so
field listed as `empty`).
This PR tries to fix this flakiness by adding a wait for the global
loading right before fetching the sidebar details.
## Summary
ES|QL charts do not use permanent dataviews. They are just using adHoc
dataviews for technical reasons and only (as the entire Lens
architecture is based on them). By calculating (and exporting to the
state) the references is creating some bad UX behavior as the dashboards
expect the dataviews to exist.
This PR is fixing this by removing the unnecessary references from the
Lens charts.
## Summary
The default editor for the Kibana Dev Console has been changed from the
Ace editor to the Monaco editor. This PR updates the ccs n-1 test to
account for this.
PS: The screen size change is done to account the size of the response
headers in case of requests done to CCS configured indices.
Backport is only needed for 8.15 branch.
**Addresses:** https://github.com/elastic/kibana/issues/186356
**Relates to:** https://github.com/elastic/kibana/issues/184428
## Summary
This PR adds a merging JS script based on the utility implemented in https://github.com/elastic/kibana/issues/186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.
## Details
https://github.com/elastic/kibana/pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below
```js
const { merge } = require('@kbn/openapi-bundler');
(async () => {
await merge({
sourceGlobs: [/* a list of source globs goes here */],
outputFilePath: 'path/to/the/output/file.yaml',
});
})();
```
The JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.
**To run** the script use the following command from Kibana root folder
```bash
node ./oas_docs/scripts/merge_serverless_oas.js
```
## Known linting issues with Security Solution OpenAPI specs
Running Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.
We need to analyze the problem and decide if/when we should fix it.
The rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.
## Next steps
Since `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.
`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.
## Revert override alert timestamp
Previously we added override of alert timestamp for manual rule runs.
Later was decided, that timestamp for manual rule run should behave the
same as regular alert and represent time when alert generated.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
When creating a definition we automatically start the transforms when
all the components are installed. This change introduces `installOnly:
boolean` query string that can be passed to routes that create
definitions (`POST /internal/entities/definition` and `PUT
/internal/entities/managed/enablement`) in order to only install the
components
### Testing
- `PUT kbn:/internal/entities/managed/enablement?installOnly=true`
- `GET kbn:/internal/entities/definition` returns builtin definitions
with `{ state: { installed: true, running: false } }`
- check the installed transforms are not started
## Summary
This occurs when the context is very large and has difficulty flushing
to the stream. When writing a large blob to stream, the stream may not
be ready to process any further chunks. When it isn't ready, the stream
will return false. The fix is we honour this scenario, creating a back
pressure buffer to keep it in memory and checking on each new chunk
whether the write is ready.
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
This PR updates the function definitions and inline docs based on the
latest metadata from Elasticsearch.
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
These are impacting DX. Remediation will be tracked at
https://github.com/elastic/kibana/issues/190345.
## Testing
`node scripts/build_kibana_platform_plugins` and `yarn start` should not
log warnings about the `mixed-decls` sass rule
legrego helpfully pointed out that these files should be scanned - they
include client side code. The replaces the exclusion of `public` with
`target`, the compiled version of this source.
**Addresses:** https://github.com/elastic/kibana/issues/190035
## Summary
This PR fixes `operationId` conflicts in Security Solution OpenAPI specs.
## Details
API reference documentation platform (Bump.sh) expects unique `operationId`s to build proper navigation on the documentation page. It's expected each `operationId` throughout whole Kibana since Kibana API reference documentation will contain all available API endpoints.
## Summary
Increases the performance of the synthetics project monitor API by
reducing response times by approximately 85%.
When generating a Synthetics package policy, we need to know the
namespace the policy should be applied to. The priority on main is as
follows
1. Use the namespace of monitor config when available and not default
2. Use the namespace of the private location
3. Use the namespace of the agent policy
There's a few problems with this
1. There is never a namespace on the private location. We do not need
that conditional
2. To support fallback number 3, we make a call to get the agent policy
to fetch the namespace. Previously, this happened for every package
policy generation, which means this happened for every monitor and
private location combination. As it turns out, we don't need to fetch
the agent policy at all, because when we provide `undefined` as the
namespace, agent already knows to fall back to the agent policy
namespace.
5. This PR removes the unnecessary conditional and call to get the agent
policy to add the namespace. Additionally, it removes another
unnecessary call to get the agent policies when fetching all available
locations, and a third unnecessary call to fetch the number of agents
per policy.
---------
Co-authored-by: Shahzad <shahzad31comp@gmail.com>
## Summary
Allow deletion of monitor via id param !!
User can now delete monitor via passing id as url param
`DELETE <kibana host>:<port>/api/synthetics/monitors/<config_id>`
Previous bulk delete via list of ids via API body still works as well !!
Docs are updated !!
## Summary
### Security Solution impacts
PR updates the SentinelOne response actions to:
- use `sentinel_one.[data_type].agent.id` field to identify the host ID
- With this change, our uses are no longer restricted to creating SIEM
alerts only from the `logs-sentinel_one.alert*` index
- Indexes that currently include the `*.agent.id` field:
- `logs-sentinel_one.alert*`
- `logs-sentinel_one.threat*`
- `logs-sentinel_one.activity*`
- `logs-sentinel_one.agent*`
- ❗ IMPORTANT ❗ :
- Environments with a SIEM rule that looks for
`observable.serial_number` field _(the field used prior to this PR to
identify the agent id in the SentinelOne document)_ should update the
rule to use one of the new fields (see screen capture below)
- The following impacts were identified during testing for existing
deployments that may already be using the SentinelOne bi-directional
response actions (currently in Tech. Preview):
1. User will no longer be able to download the output from a previous
`get-file` command (this was just release 2 weeks ago to serverless).
2. After an upgrade, if a user opens the console and clicks on the
"Response actions history" button to display the host's response
actions, they will **not** see the response actions in the list that
were submitted prior to the upgrade. Those, however, will still be
displayed in the (global) Response Actions History Log page.
- Dev script was updated to create a SIEM rule that looks at both
`*.alert*` and `*.threat*` indexes
- Fixed the output for `processes` for SentinelOne to NOT display a Zip
file passcode for the download (not needed)
- Fixed bug that prevented the Host's OS platform icon (linux, windows,
macos) from being displayed in the console.
### Connector impacts
- SentinelOne connector sub-actions were updated to take in `agentId` as
an argument instead of `agentUUID`
