Commit graph

75984 commits

Author SHA1 Message Date
Hannah Mudge
80f3c191ce
[Embeddable Rebuild] [Controls] Clean up styling + add clear selections to timeslider (#186656)
## Summary

The primary goal of this PR is to clean up the styling of the
`ControlPanel` component for the new React control renderer.
Specifically, this fixes the following:
- I switched the inline Emotion styling to CSS classes instead
- I made it so that the timeslider control renders the drag handler in
edit mode and **doesn't** render the empty icon for the drag handler in
view mode

<p align="center"><img width="600px"
src="d5bf169b-2106-4f88-9698-f00162809d0a"/><p>

- I fixed the timeslider prepend so that it no longer wraps

<p align="center"><img width="500px"
src="7859d67b-1454-45b5-b7d8-7000086641a7"/><p>

- I moved the error component into the `EuiFormControlLayout` component,
which ensures that the drag handler is rendered for when a control has a
blocking error. I also fixed the styling for the error component:

<p align="center"><img width="600px"
src="13e0f041-8c51-494c-9079-323ed518c87b"/><p>

When I was working on these style changes, I noticed that the timeslider
control wasn't implementing `CanClearSelections` which meant that it no
longer had the clear selections action. This made me realize that this
interface should probably be part of the `DefaultControlApi` rather than
`DefaultDataControlApi` so, I moved it and added `clearSelections` to
the timeslider API.

<p align="center"><img width="600px"
src="47f7b648-bb2d-4158-b058-456bfdf5cdb5"/><p>



### Checklist

- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-07-03 08:07:21 -06:00
Julia Bardi
82d32a757f
[Fleet] added check for Enterprise license in package policy create/update APIs (#187467)
## Summary

Relates https://github.com/elastic/ingest-dev/issues/3464

Added check to reject integration policy shared by multiple agent
policies if Enterprise license is not available.

### Testing

- Enable a local enterprise licence
([steps](46802910/Internal+License+-+X-Pack+and+Endgame))
- Enable flag `enableReusableIntegrationPolicies`
- Try create/update package policy API with multiple `policy_ids`,
expect to work

Repeat the steps with any lower licence, the API should reject the
request with a 400 error.

```
POST kbn:/api/fleet/package_policies
{
  "policy_ids": [
    "policy-1", "policy-2"
  ],
  "package": {
    "name": "apache",
    "version": "1.20.0"
  },
  "name": "apache-4",
  "description": "",
  "namespace": "",
  "inputs": []
}
```

Test with Basic license:
<img width="910" alt="image"
src="d99b3765-0b0b-4abd-ae4c-dc9f396a465a">

Test with Enterprise license:
<img width="910" alt="image"
src="d42b761e-32f0-46b6-95a8-0a565f898532">


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2024-07-03 23:28:09 +10:00
Elastic Machine
83a17990fe
Update kubernetes templates for elastic-agent (#187429)
Automated by https://buildkite.com/elastic/elastic-agent/builds/10108
2024-07-03 08:16:38 -05:00
Walter Rafelsberger
c60efafe70
[ML] AIOps: Add AI Assistant contextual insight to Log Rate Analysis page in ML plugin in Observability serverless. (#186509)
## Summary

Part of #181111.

This adds O11y AI Assistant's contextual insight to the Log Rate
Analysis page in the ML plugin for O11y serverless projects..

Note the code is almost an exact copy of what's currently used in the
O11y UI itself to render the contextual insight when log rate analysis
gets embedded on alert details pages. As a follow up we will be able to
consolidate some code.

[aiops-ai-assistant-contextual-insight-0001.webm](08269a40-f45a-4eb9-bd4d-a249cdd46266)

To test this you need to set up AI Assistant in your `kibana.dev.yml`:

```yml
xpack.actions.preconfigured:
  my-gen-ai:
    name: Preconfigured Azure OpenAi
    actionTypeId: .gen-ai
    config:
      apiUrl: <YOUR-API-URL>
      apiProvider: 'Azure OpenAI'
    secrets:
      apiKey: <YOUR-API-KEY>
```

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2024-07-03 15:15:16 +02:00
Walter Rafelsberger
39d3e1a923
[APM] Reenable and stabilize APM correlations API integration tests. (#187444)
## Summary

Fixes #176544.
Fixes #187421.
Fixes #176119.
Fixes #176425.
Fixes #175855.
Fixes #175911.
Fixes #176780.

Follow up to #186182.

Reenables and stabilizes APM correlations API integration tests.

Review hint: View with the `w=1` flag to ignore whitespace changes:
https://github.com/elastic/kibana/pull/187444/files?w=1

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2024-07-03 15:07:53 +02:00
Jon
bd41c659d9
[ci] Use org wide PR status bot (#187386) 2024-07-03 08:07:19 -05:00
Julia Bardi
837cd300b3
[Fleet] update package policies to remove deleted agent policy ref (#187370)
## Summary

Closes https://github.com/elastic/kibana/issues/187331

Update package policies with multiple agent policy references to remove
deleted agent policy id.

See steps to verify in the linked issue.

- create an integration policy and add it to multiple agent policies
- delete one agent policy
- verify that the edit integration policy page loads, and the
integration policy is linked to the remaining agent policies

<img width="1305" alt="image"
src="a0c3dc22-b703-42ab-b40d-91c2723cce01">
<img width="1257" alt="image"
src="218bf1aa-7208-49f5-8bdc-e09b47c2924e">


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2024-07-03 07:30:41 -05:00
Tom Myers
19db776f4f
make EntityDefinition 'type' a plain string instead of an enum (#187451)
make EntityDefinition 'type' a plain string instead of an enum
2024-07-03 13:15:30 +01:00
Tomasz Ciecierski
93e127e44d
[EDR Workflows] Fix skipped timelines.cy.ts test (#187374) 2024-07-03 06:53:26 -05:00
Alexey Antonov
935d446835
fix: [Obs Alerts > Rules Overview][SCREEN READER]: Table rows need TH[scope="row"] for SR usability: 0001 (#186591)
Closes: https://github.com/elastic/observability-dev/issues/3533

## Summary

`rowHeader` attribute was added for the following table 


![image](215633af-bff4-4581-9d7e-a9826f1e63d4)

### Screens 

<img width="1585" alt="Screenshot 2024-06-21 at 10 52 51"
src="de9bbec9-737e-48fd-aa9d-73d4d2069f15">
2024-07-03 14:42:59 +03:00
Dzmitry Lemechko
253696ae43
[ftr] mark basic auth login method for UI tests as deprecated (#187136)
## Summary

To bring more attention about #183512 and make sure newly added tests do
not use basic auth with operator

<img width="917" alt="image"
src="22926c83-f97d-49c3-99a0-3c41f744d34c">
2024-07-03 06:39:13 -05:00
Tomasz Ciecierski
a8c6ae7e38
[EDR Workflows] Change the .skip() place for saved_queries.cy.ts (#187453) 2024-07-03 13:35:57 +02:00
Julia Bardi
260882e195
[Fleet] handle multiple policies in edit package policy extension view (#187334)
## Summary

Related to https://github.com/elastic/kibana/issues/75867

Handling multiple policies in package policy edit extension view, made
changes in CSP.

@elastic/kibana-cloud-security-posture Hey, could you help me how can I
create an agentless policy to test with?

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2024-07-03 05:54:21 -05:00
Cristina Amico
307237f659
[Fleet] Bug - Add missing instance of enableReusableIntegrationPolicies under enterprise plan (#187447)
## Summary

In https://github.com/elastic/kibana/pull/186871 I forgot to replace one
instance of `enableReusableIntegrationPolicies` with the new hook
`canUseMultipleAgentPolicies`. This could potentially cause a bug once
the flag is enabled, as that component should only be visible to
enterprise users.

It's a very minor change anyway so I don't think there's any issue in
merging after FF.
2024-07-03 12:22:58 +02:00
Marco Liberati
4359659729
[Lens] Fix telemetry for annotation layers (#187281)
## Summary

This PR fixes the telemetry code for annotation layers adding some
dedicated unit test for the event logic.


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2024-07-03 12:05:06 +02:00
Miriam
7274f44e9c
[OnWeek][ObsUX] Add fields to hosts in synthtrace to improve data generation (#187147)
## Summary

After improving the synthtrace data creation for containers we were able
to add more specific tests for container view, the aim of this spacetime
is to add some improvements to hosts so we can in the future use
synthtrace for testing

### What was done

First I thought that adding `event.dataset` was needed to get the
metadata, or make the request work, as I did for containers, but in
containers was needed not because of the metadata query itself but the
integration check to know if we need to display k8s or docker metrics.
I simplified the scenarios and data generation in the tests, adding the
metadata fields we need in the synthtrace clients for host and docker
and k8s containers, the values of the metadata fields doesn't need to
change for different scenarios, so it's ok to have them set in the
client.
2024-07-03 11:01:11 +01:00
Kevin Lacabane
988795d1ff
[eem] improve enablement error handling (#187215) 2024-07-03 11:21:02 +02:00
Tom Myers
bfe8cf9cc1
include entity type in EEM output docs (#187355)
include entity type in EEM output docs
2024-07-03 10:12:31 +01:00
Vadim Kibana
7e04249054
[ES|QL] Add weighted_avg ES|QL function support (#187439)
## Summary

- Adds `weighted_avg` function definition
- Adds basic smoke tests


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2024-07-03 04:04:12 -05:00
Alexey Antonov
ce4f4693f4
fix: [Obs Synthetics > Monitor test run][KEYBOARD]: Date (check) dropdown needs a semantic label for the current check (#187353)
Closes: https://github.com/elastic/observability-dev/issues/3689

## Description

The synthetics monitors include thumbnail screenshots that open a larger
preview window. These thumbnails must take keyboard focus, manage the
`Enter` and `Space` keypresses to open the modal, and return focus to
the originating thumbnail when the modal is closed. Screenshots attached
below.

### Steps to recreate

1. Open the
[Synthetics](https://keep-serverless-fyzdg-f07c50.kb.eu-west-1.aws.qa.elastic.cloud/app/synthetics)
view
2. Create a monitor if none exist
3. Click on that monitor and navigate to the full monitor detail
4. Click the ["View test run link"
](90a1abac-3579-11ef-b9d4-4e0e2c056918-1?locationId=us_central_qa)in
the Last test run module (about 2/3 of the way down the page)
5. Open the "Date" dropdown in the top right
6. Either listen to the dropdown nodes with a screen reader or evaluate
with Dev Tools Accessibility Tree. This will show you a standard,
non-semantic div for the current check.


### What was changed?: 

1. `aria-label` attribute was added to address that issue
2. similar changes were applied to 3 places: 
-
`x-pack/plugins/observability_solution/synthetics/public/apps/synthetics/components/step_details_page/step_page_nav.tsx`
-
`x-pack/plugins/observability_solution/uptime/public/legacy_uptime/components/monitor/synthetics/step_detail/step_page_nav.tsx`
-
`x-pack/plugins/observability_solution/uptime/public/legacy_uptime/pages/synthetics/checks_navigation.tsx`

### Screen

<img width="1269" alt="image"
src="dc1f4711-a834-4d06-825f-8fb6d49891d0">
2024-07-03 11:56:45 +03:00
Antonio
ac3ceba51f
[ResponseOps][Cases] Change login role in serverless UI tests. (#187332)
## Summary

Updated UI FTR tests to not run with operator privileges.

Fixes #184743
2024-07-03 10:41:43 +02:00
Carlos Crespo
d816fdf538
[APM][Serverless] Remove universal profiling setting from Advanced Settings (#187371)
## Summary

Remove the "Enable Universal Profiling integration in APM" setting from
Advanced Settings


Tab not displayed
<img width="1466" alt="image"
src="48c80f8a-5805-4740-9a58-602eddad4576">

Setting not available
<img width="1724" alt="image"
src="1871c6c9-0a6b-454c-b3a5-bbb067eb27d9">


# How to test
- `yarn es serverless --projectType=oblt --clean --no-ssl`
- `yarn start --serverless=oblt --no-ssl`
- Navigate to Advanced Settings
2024-07-03 10:21:26 +02:00
Alejandro Fernández Haro
81367bba9a
[Security-in-core] Cloud Chat (#187306) 2024-07-03 03:19:26 -05:00
Philippe Oberti
faada573c4
[Security Solution][Notes] - prevent user from adding note if the markdown is invalid (#187346) 2024-07-03 02:25:05 -05:00
Pablo Machado
28046696a2
[Security Solution] Fix risk score should not 404 when alerts index doesn't exist (#187158)
## Summary

* It adds a parameter to the risk score calculation that prevents it
from falling when the alert's index doesn't exist.


### How to reproduce it?
Please take a look at the original issue
https://github.com/elastic/kibana/issues/187052
2024-07-03 09:16:37 +02:00
James Gowdy
1089f57b51
[ML] Fixing upgrade warning (#187387)
The index migration checks were broken, so we were not showing our
warnings when upgrade mode is set to `true`

<img width="1139" alt="image"
src="8ede466b-c9e9-4ad9-a21a-d7e955b82247">


Also adds the warning to the ML stack management page so it is correctly
displayed rather than an access denied page.

<img width="1140" alt="image"
src="b78e4a60-109b-4ea7-8c7f-021ad26a3313">


To test, set `POST _ml/set_upgrade_mode?enabled=true`


Fixes https://github.com/elastic/kibana/issues/176773
2024-07-03 07:56:43 +01:00
Kibana Machine
f7f841efac
[api-docs] 2024-07-03 Daily api_docs build (#187437)
Generated by
https://buildkite.com/elastic/kibana-api-docs-daily/builds/757
2024-07-03 04:55:56 +00:00
Kibana Machine
df1264fd50 skip failing test suite (#187388) 2024-07-03 06:01:24 +02:00
Tiago Costa
53ebc68f72
skip flaky suite (#187421) 2024-07-03 04:59:18 +01:00
Felix Stürmer
08017ae2dc
[Logs UI] Fix log entry fly-out when response is slow (#187303)
This disables a change in polling behavior of the ESE search strategy,
which was introduced with https://github.com/elastic/kibana/pull/178921.
The response processing and progress reporting depends on it.
2024-07-03 00:37:49 +02:00
Ash
43058ca97b
[Serverless][SecuritySolution][Endpoint] Update serverless tests for scan w.r.to. PLIs (#187376)
## Summary

As `scan` response action is categorized now with Endpoint complete PLI
include `scan` action in serverless tests using
`responseActionScanEnabled` feature flag.

### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
2024-07-02 23:29:01 +02:00
Khristinin Nikita
32e7bf9828
Telemetry for manual rule run (#186364)
## Summary

report following events:
- open modal window for manual rule run
- execute manual rule run + save time range in ms
- cancel backfill job
- filter in event log by run type
- show source event date range

Epic - https://github.com/elastic/security-team/issues/2840

### How to test
enable feature flag - `manualRuleRunEnabled`

You can see feature demo here -
https://github.com/elastic/kibana/pull/184500

Check that events appears here after some time -
https://telemetry-v2-staging.elastic.dev/s/securitysolution/app/r/s/7YYlg

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-07-02 23:04:54 +02:00
Rickyanto Ang
9fae9c536a
[Cloud Security] Update CSP Version to 1.9.0 for Test (#186657)
## Summary

With 8.14 released, we want to make sure our CSP is using the latest CSP
version for our test environment
2024-07-02 13:07:33 -07:00
Steph Milovic
ad4fe84078
[Security solution] Assistant race condition bug fixing (#187186) 2024-07-02 13:42:22 -06:00
Ryland Herrick
2aa94a27f0
[Detection Engine] Adds Alert Suppression to ML Rules (#181926)
## Summary
This PR introduces Alert Suppression for ML Detection Rules. This
feature is behaviorally similar to alerting suppression for other
Detection Engine Rule types, and nearly identical to the analogous
features for EQL rules.

There are some additional UI behaviors introduced here as well, mainly
intended to cover the shortcomings discovered in
https://github.com/elastic/kibana/issues/183100. Those behaviors are:

1. Populating the suppression field list with fields from the anomaly
index(es).
1. Disabling the suppression UI if no selected ML jobs are running
(because we cannot populate the list of fields on which they'll be
suppressing).
1. Warning the user if _some_ selected ML jobs are not running (because
the list of suppression fields may be incomplete).

See screenshots below for more info.

### Intermediate Serverless Deployment
As per the "intermediate deployment" requirements for serverless, while
the schema (and declared alert SO mappings) will be extended to allow
this functionality, the user-facing features are currently hidden behind
a feature flag. Once this is merged and released, we can issue a "final"
deployment in which the feature flag is enabled, and the feature
effectively released.


## Screenshots
* Overview of new UI fields
<img width="1044" alt="Screenshot 2024-05-16 at 3 22 02 PM"
src="8c07700d-5860-4d1e-a701-eac84fc35558">
* Example of Anomaly fields in suppression combobox
<img width="881" alt="Screenshot 2024-06-06 at 5 14 17 PM"
src="9aa6ed99-1e02-44a0-ad1b-785136510d68">
* Suppression disabled due to no jobs running
<img width="668" alt="Screenshot 2024-06-17 at 11 23 39 PM"
src="a8636a52-31bd-4579-9bcd-d59d93c26984">
* Warning due to not all jobs running
<img width="776" alt="Screenshot 2024-06-17 at 11 26 16 PM"
src="f44c2400-570e-4fde-adce-e5841a2de08d">

## Steps to Review
1. Review the Test Plan for an overview of behavior
2. Review Integration tests for an overview of implementation and edge
cases
3. Review Cypress tests for an overview of UX changes
4. Testing on [Demo
Instance](https://rylnd-pr-181926-ml-rule-alert-suppression.kbndev.co/)
(elastic/changeme)
1. This instance has the relevant feature flag enabled, has some sample
auditbeat data, as well as the [anomalies archive
data](https://github.com/elastic/kibana/tree/main/x-pack/test/functional/es_archives/security_solution/anomalies)
for the purposes of exercising an ML rule against "real" anomalies
    1. There are a few example rules in the default space:
1. A simple [query
rule](f6f5960d-7e4b-40c1-ae15-501112822130)
against auditbeat data
1. An [ML
rule](9122669e-b2e1-41ce-af25-eeae15aa9ece)
with per-execution suppression on both `by_field_name` and
`by_field_value` (which ends up not actually suppressing anything)
1. An [ML
rule](0aabc280-00bd-42d4-82e6-65997c751797)
with per-execution suppression on `by_field_name` (which suppresses all
anomalies into a single alert)

## Related Issues
- This feature was temporarily blocked by
https://github.com/elastic/kibana/issues/183100, but those changes are
now in this PR.

## Checklist
- [x] Functional changes are hidden behind a feature flag. If not
hidden, the PR explains why these changes are being implemented in a
long-living feature branch.
- [x] Functional changes are covered with a test plan and automated
tests.
    * [Test Plan](https://github.com/elastic/security-team/pull/9279)
- [x] Stability of new and changed tests is verified using the [Flaky
Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner) in
both ESS and Serverless. By default, use 200 runs for ESS and 200 runs
for Serverless.
* [ESS - Cypress x
200](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6449)
* [Serverless - Cypress x
200](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6450)
* [ESS - API x
200](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6447)
* [Serverless - API x
200](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6448)
- [ ] Comprehensive manual testing is done by two engineers: the PR
author and one of the PR reviewers. Changes are tested in both ESS and
Serverless.
- [ ] Mapping changes are accompanied by a technical design document. It
can be a GitHub issue or an RFC explaining the changes. The design
document is shared with and approved by the appropriate teams and
individual stakeholders.
- [ ] (OPTIONAL) OpenAPI specs changes include detailed descriptions and
examples of usage and are ready to be released on
https://docs.elastic.co/api-reference. NOTE: This is optional because at
the moment we don't have yet any OpenAPI specs that would be fully
"documented" and "GA-ready" for publishing on
https://docs.elastic.co/api-reference.
- [ ] Functional changes are communicated to the Docs team. A ticket is
opened in https://github.com/elastic/security-docs using the [Internal
documentation request (Elastic
employees)](https://github.com/elastic/security-docs/issues/new?assignees=&labels=&projects=&template=docs-request-internal.yaml&title=%5BRequest%5D+)
template. The following information is included: feature flags used,
target ESS version, planned timing for ESS and Serverless releases.

---------

Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-07-02 14:33:11 -05:00
Kerry Gallagher
e654e46466
[Logs] Add Log Sources advanced setting and client / server access services (#186468)
## Summary

Implements part 1 of
https://github.com/elastic/observability-dev/issues/3498 (adds an
advanced setting and data access services for consumers).

## Reviewer notes

- Please see note comments inline with the code.

- The `limits.yml` change was generated by `node
scripts/build_kibana_platform_plugins.js --update-limits`.

- There are no consumers using this yet, so you'll need to make some
minor adjustments if you'd like to test the access services. In a plugin
(`infra` for example) the server side access could look like this:

```ts
const [, { logsDataAccess }] = await getStartServices();
const logSourcesService = await logsDataAccess.services.getLogSourcesService(request);
await logSourcesService.setLogSources([
  { indexPattern: 'logs-*-*' },
  { indexPattern: 'test-logs-*' },
]);
```

Public access could look like this:

```ts
const logSourcesService = plugins.logsDataAccess.services.logSourcesService;
logSourcesService.setLogSources([{ indexPattern: 'client-side-logs-*' }]);
```

- I haven't added any tests here yet as any unit tests would more or
less just be re-testing a UI settings mock. Functional tests will be
valuable once there are consumers.


## UI

<img width="1425" alt="Screenshot 2024-06-20 at 10 41 16"
src="30608dcf-4c34-4d29-9a22-441b06757c28">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-07-02 20:29:32 +01:00
Aleh Zasypkin
4007283a84
fix(dev, serverless): do not inject mock SAML IdP configuration if conflicting configuration is provided via CLI arguments (#187337)
## Summary

Our functional test server provides Kibana configuration via CLI
arguments that the code configuring the mock SAML IdP realm in dev mode
didn't account for. This means that when we run the test server locally,
both FTR and Kibana try to configure the mock SAML IdP, which crashes
the local Kibana. This issue only affects those who run functional tests
locally and doesn't impact CI, where we use the built version of Kibana
to run tests. This built version doesn't include the mock SAML IdP,
delegating the mock SAML IdP configuration solely to FTR.

This PR updates the code that attempts to automatically configure the
mock SAML IdP in dev mode to check the configuration from both config
files and CLI arguments to determine whether automatic configuration is
possible.
2024-07-02 21:26:25 +02:00
Gabriel Landau
5cb60aa23f
Defend Advanced Policy Options for Registry Event Filtering Enforcement (#186564)
## Summary

Adds a Defend Advanced Policy option to allow 8.15.0 users to opt out of
Registry Event Filtering.

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2024-07-02 14:28:32 -04:00
Sandra G
75874ca942
[APM] limit service map scripted metric agg based on shard count (#186417)
## Summary

https://github.com/elastic/kibana/issues/179229

This PR addresses the need to limit the amount of data that the scripted
metric aggregation in the service map processes in one request which can
lead to timeouts and OOMs, often resulting in the user seeing [parent
circuit
breaker](https://www.elastic.co/guide/en/elasticsearch/reference/current/circuit-breaker.html#parent-circuit-breaker)
errors and no service map visualization. This query can fire up to 20
times max depending on how many trace ids are fetched in subsequent
query, contributing more to exceeding the total allowable memory.

These changes will not remove the possibility of OOMs or circuit breaker
errors. It doesn't control for multiple users or other processes
happening in kibana, rather we are removing the current state of
querying for an unknown number of documents by providing a hard limit
and a way to easily tweak that limit.

## Changes
- Make get_service_paths_from_trace_ids "shard aware" by adding an
initial query, `get_trace_ids_shard_data` without the aggregations and
only the trace id filter and other filters in order to see how many
shards were searched
- Use a baseline of 2_576_980_377 bytes max from new config
`serverlessServiceMapMaxAvailableBytes`, for all
get_service_paths_from_trace_ids queries when hitting the
`/internal/apm/service-map`
- Calculate how many docs we should retrieve per shard and set that to
`terminateAfter` and also as part of the map phase to ensure we never
send more than this number to reduce
- Calculation is: ((serverlessServiceMapMaxAvailableBytes / average
document size) / totalRequests) / numberOfShards
Eg: 2_576_980_377 / 495 avg doc size = 5,206,020 total docs
 5,206,020 total docs / 10 requests = 520,602 docs per query
520,602 docs per query / 3 shards = **173,534 docs per shard**
Since 173,534 is greater than the default setting
`serviceMapTerminateAfter`, docs per shard is 100k
- Ensure that `map_script` phase won't process duplicate events
- Refactor the `processAndReturnEvent` function to replace recursion
with a loop to mitigate risks of stack overflow and excessive memory
consumption when processing deep trees


## Testing

### Testing that the scripted metric agg query does not exceed the
request circuit breaker
- start elasticsearch with default settings
- on `main`, without these changes, update the request circuit breaker
limit to be 2mb:
```
 PUT /_cluster/settings
{
  "persistent": {
    "indices.breaker.request.limit": "2mb"
  }
}
```
- run synthtrace `node scripts/synthtrace.js service_map_oom
--from=now-15m --to=now --clean`
- Go to the service map, and you should see this error:
<img width="305" alt="Screenshot 2024-06-20 at 2 41 18 PM"
src="517709e5-f5c0-46bf-a06f-5817458fe292">

- checkout this PR
- set the apm kibana setting to 2mb(binary):
`xpack.apm.serverlessServiceMapMaxAvailableBytes: 2097152`. this
represents the available space for the [request circuit
breaker](https://www.elastic.co/guide/en/elasticsearch/reference/current/circuit-breaker.html#request-circuit-breaker),
since we aren't grabbing that dynamically.
- navigate to the service map and you should not get this error and the
service map should appear

---------

Co-authored-by: Carlos Crespo <carloshenrique.leonelcrespo@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-07-02 14:11:56 -04:00
Drew Tate
dbd1334480
[ES|QL] capitalize things (#186340)
## Summary



8f8e618c-22e6-4a33-957e-c9d1664cc000



Close https://github.com/elastic/kibana/issues/184238

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
2024-07-02 12:55:51 -05:00
Katerina
a613a53e9b
[APM] Unskip apm api test suite (#186870)
## Summary
test suite was skipped in
8c18091792
2024-07-02 12:40:57 -05:00
Elena Stoeva
628994ea3a
[Rollups] Update rollup badges in Data view form (#187235)
Closes https://github.com/elastic/kibana/issues/186663

## Summary

Follow-up to https://github.com/elastic/kibana/pull/186321. This PR
updates the Rollup badges in the Data view create form so that they show
that Rollup is deprecated.

<details>
<summary>Before:</summary>
<img width="1327" alt="Screenshot 2024-07-01 at 11 48 21"
src="9f4e1cd1-a34c-42dd-926a-ed6758f06ed7">
</details>

<details>
<summary>Now:</summary>
<img width="1323" alt="Screenshot 2024-07-01 at 17 50 48"
src="084873f2-3707-46c4-b4ba-19b4e68c2d87">

<img width="1323" alt="Screenshot 2024-07-01 at 17 50 54"
src="d021ef34-3a84-4406-b23e-027aa2e8e868">
<img width="1323" alt="Screenshot 2024-07-01 at 17 50 59"
src="ef1246c5-00ed-4eaa-af21-e0515bce956a">
</details>

**How to test:**
1. Start Es and Kibana
2. Add the sample data "Sample web logs"
3. Elasticsearch only allows creating a rollup job if there is an
existing rollup usage in the cluster. To simulate rollup usage, create a
mock rollup index through Console:

```
PUT /mock_rollup_index
{
  "mappings": {
    "_meta": {
      "_rollup": {
        "id": "logs_job"
      }
    }
  }
}
```


4. Create a sample rollup job through Console:

```
PUT _rollup/job/logs_job
{
  "id": "logs_job",
  "index_pattern": "kibana_sample_data_logs",
  "rollup_index": "rollup_logstash",
  "cron": "* * * * * ?",
  "page_size": 1000,
  "groups": {
    "date_histogram": {
      "interval": "60m",
      "delay": "7d",
      "time_zone": "UTC",
      "field": "@timestamp"
    },
    "terms": {
      "fields": [
        "geo.src",
        "machine.os.keyword"
      ]
    },
    "histogram": {
      "interval": "1003",
      "fields": [
        "bytes",
        "memory"
      ]
    }
  }
}
```


5. Delete the mock rollup index since it causes issues for the rollup
API that we use to fetch rollup indices:
`DELETE /mock_rollup_index`
6. Navigate to Stack Management -> Data Views and start creating a new
data view.
7. Verify that the rollup badges are updated.




<!--
### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-07-02 12:36:16 -05:00
Samiul Monir
6120167bc3
Fix form submit issue with key down (#187116)
## Summary

This PR fixes an SDH issue where users are writing with a Japanese
keyboard and transform the English to Japanese, they have to press ENTER
key. Our OnKeyDown functionality automatically submits the form when the
user hits the ENTER key from the question input text field.

We removed the form submission from the text field.

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-07-02 13:35:13 -04:00
Kibana Machine
274b9fe850 skip failing test suite (#187383) 2024-07-02 19:16:10 +02:00
Gloria Hornero
2afe55eaec
[Security Solution] Updates periodic pipeline job names (#187378)
## Summary

Currently is not possible to see at first sight which execution is from
Cypress and which one from API.

<img width="2545" alt="Screenshot 2024-07-02 at 17 18 04"
src="c89c204d-e2cf-4661-87f4-1e206ad822d7">

In this PR we are updating the naming to make it easier to find out as
well as simplifying the names.
2024-07-02 12:00:15 -05:00
Joe Reuter
4066f922b5
[Observability Onboarding] OTel logs flow (#183732)
# Testing instructions

* The k8s flow should work already for logs but won't report any metrics
* The linux/mac flow requires the following stopgap measure until
everything is ready
  * Run the provided snippet
  * It will fail with `No such file or directory`
* Run either `touch otel.yml && mkdir otel_samples && curl
https://raw.githubusercontent.com/elastic/elastic-agent/871ad33afc5ac1614f0645e86f2a13c05631aa6d/internal/pkg/otel/samples/darwin/platformlogs_hostmetrics.yml
-o otel_samples/platformlogs_hostmetrics.yml` for mac or `touch otel.yml
&& mkdir otel_samples && curl
https://raw.githubusercontent.com/elastic/elastic-agent/871ad33afc5ac1614f0645e86f2a13c05631aa6d/internal/pkg/otel/samples/linux/platformlogs_hostmetrics.yml
-o otel_samples/platformlogs_hostmetrics.yml` for linux
  * Re-run the second part of the snippet (the part starting with `rm`)
  * The rest should work as before

Closes https://github.com/elastic/kibana/issues/184433

<img width="931" alt="Screenshot 2024-06-20 at 15 04 19"
src="f50392ae-d956-463f-9120-0d01b0fbfa80">


<img width="937" alt="Screenshot 2024-06-20 at 15 04 32"
src="a968f615-f173-4210-bb47-6700cc989d5a">



TODOs:
* Real kubernetes snippet
* Do not show on serverless (disabled for testability)
* Remove snapshot version and CDN url

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-07-02 11:59:45 -05:00
Kibana Machine
49c16e0a85 skip failing test suite (#187314) 2024-07-02 18:51:51 +02:00
Vadim Kibana
c46cb3a3a4
[ES|QL] Add top function definition (#187290)
## Summary

Closes https://github.com/elastic/kibana/issues/186494

- Adds ~~`top_list`~~ `top` function definition
- Adds basic smoke tests


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-07-02 11:47:00 -05:00
Alejandro Fernández Haro
24eb9e6d95
Remove joi leakage to the browser in synthetics (#187308)
Co-authored-by: jennypavlova <jennypavlova94@gmail.com>
2024-07-02 11:38:30 -05:00
Jatin Kathuria
965062dbea
[Security Solution] [Fix] Row Renderer + Notes in Flyout (#186948)
## Summary

This PR introduces below mentioned 3 changes:

### Row Renderer Switch

A quick switch to switch on/off all the row-renderers without going into
settings.

> [!Caution]
> This is only available with feature flag
`unifiedComponentsInTimelineEnabled`



88fcf4e0-a5a2-4158-bc86-b313002790ec


### Notes in a separate Flyout

- Notes do not appear inline anymore. They are now part of separate
Flyout.
- This Change also introduces a notification dot to highlight that
existing notes are available.


491e256f-36dc-42f3-89f3-4c8c90c969a5


### Color Distinction between enabled/disabled Row Renderers.

Previously it was difficult to see what row renderers are available and
what are not. This change introduces a small color distinction.



ba250c4c-cb93-4bc7-b593-235ccaf207cb


## Desk Testing 

Please desk test following functionalities with AND without below
feature flag:

1. Add Note 
2. Cancel when adding note.
3. Create a new timeline
4. Load saved timeline
5. Change from one timeline to other
6. Open Timeline page directly with saved timeline in the address bar.

In all above scenarios row renders and notes should be shown as
expected.


```
xpack.securitySolution.enableExperimental:
  - unifiedComponentsInTimelineEnabled

```
2024-07-02 11:34:34 -05:00