Commit graph

65196 commits

Author SHA1 Message Date
jennypavlova
e98abd0fb5
[Infra UI] Hosts view test: fix apm services link (#162147)
Closes https://github.com/elastic/kibana/issues/162051
Closes #159368
## Summary

This PR aims to make the APM services link test more stable. For some
reason, some of the query parameters are not present in the URL the
moment we navigate but we can see in the screenshot that the correct
dates are applied (it could be that the moment we load APM the URL is
not updated - when checking locally those parameters exist in the URL)
<img width="1612" alt="image"
src="2363108a-329e-4f28-a0d8-9f177d01d4ef">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-07-19 10:30:43 +02:00
Sander Philipse
0a6b5e92b8
[Serverless Search] Remove unwanted alterting rule and connector types (#162105)
## Summary

This moves alerting in Serverless Search to the Stack Management
alerting page, and removes

To remove these rule types, I had to disable the observability,
securitySolution, uptime, monitoring and infra plugins and make sure
that their server plugins (not just the frontend plugins) respected the
`enabled: false` flag.
2023-07-19 16:21:02 +08:00
Tomasz Ciecierski
9c7dda0b90
[Defend workflows] Stop spreading whole request to ES dsl (#162116) 2023-07-19 09:58:45 +02:00
Kibana Machine
c472fb018b
[api-docs] 2023-07-19 Daily api_docs build (#162202)
Generated by
https://buildkite.com/elastic/kibana-api-docs-daily/builds/403
2023-07-19 00:57:08 -04:00
Lukas Olson
13372c91ce
[data] Use versioned router for remaining routes (#161919)
## Summary

Uses the versioned router for the remaining routes in the data plugin:
KQL telemetry (opt-in stats), and scripting languages list.

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2023-07-18 14:40:11 -07:00
Steph Milovic
e96dd5cac4
[Security solution] Bump GenAI connector license to Enterprise (#162184) 2023-07-18 14:54:55 -06:00
Lukas Olson
2a689fa787
Create @kbn/discover-utils package (#162004)
## Summary

Creates a shared package for Discover utils/services/hooks,
`@kbn/discover-utils`. Moves a couple of utils there
(`buildDataTableRecord`/`getDocId`). This will be the future home for
many services/utils that are shared between the Discover app and its
packaged components (like unified field list, unified histogram, etc.)

### Checklist

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-07-18 13:34:09 -07:00
Ahmad Bamieh
384e2eda83
[i18n] Integrate 8.9.0 Translations (#162153) 2023-07-18 23:21:07 +03:00
jennypavlova
5a7f395003
[Infra UI] Add alerts to asset details flyout (#161677)
Closes #160371 

## Summary

This PR adds alerts section to the overview tab inside the asset details
flyout component.

Notes: A lot of changes are extracting common components from the alerts
tab to a common folder. The flyout version is not showing the chart so
it's not exactly the same component but a big part of the logic is
reused there. The tooltip content can be found in a [Figma comment
](https://www.figma.com/file/XBVpHX6pOBaTPoGHWhEQJH?node-id=843:435665&mode=design#492130894)


<img width="1616" alt="alerts_section"
src="399dd1ea-e1cb-4e7f-9ed5-917ced7cc490">

## Alerts summary widget changes:
After introducing the `hideChart` prop
[here](https://github.com/elastic/kibana/pull/161263) in this PR I
change the spinner type and size in case of no chart we want to have a
smaller section with a smaller spinner:


![image](43a3c611-0404-4c21-a503-22f1a79dc1de)



![image](a870fa9b-5367-4303-9b7d-4da9ff2eae2b)


##  Storybook
I added some changes to make the alerts widget show in the storybook
[[Workaround for
storybook](d97a2b1736)]

<img width="1905" alt="image"
src="539c9443-f977-4301-8d2b-d24f1d01b44e">
 
## Testing
- Go to Hosts view and open the single host flyout - alerts section
should be visible
- Alerts title icon should open a tooltip with links to alerts and
alerts documentation
- Alerts links:
- The Create rule link will open a flyout (on top, not closing the
existing flyout) to create an inventory rule, when closed/saved rule the
single host flyout should remain open
- The Show All link should navigate to alerts and apply time range /
host.name filter selected in the hosts view


b362042a-b9de-460c-86ae-282154b586ff

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-07-18 19:21:37 +02:00
Nathan Reese
b222f7a7d1
[maps] support null description in saved object (#162145)
Part of https://github.com/elastic/kibana/issues/162069

> This happens in maps too and possibly other SOs that have null
description (not sure if it were ever possible to have null description
in maps.) It seems from the SOs that Bhavya sent that it was possible
for visualizations. With this schema change the bug if fixed
https://github.com/elastic/kibana/pull/162106

> Not sure if it makes sense to also do it in maps. @nreese wdyt?

Add support for null description in maps saved object to be consistent
with visualize saved objects.

### test instructions
1) Install web logs sample data set
2) export map
3) manually change ndjson, changing `description:""` to
`description:null`
4) reimport map saved object
5) verify map can be opened

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-07-18 11:10:11 -06:00
James Gowdy
5548e12010
[ML] Fix flaky update_groups api test (#161326)
Related to https://github.com/elastic/kibana/issues/161324 and
https://github.com/elastic/kibana/issues/160370
Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2649

I believe the problem lies with the function `cleanMLSavedObjects` only
cleaning up saved objects in the default space and not in any other of
the spaces which jobs or trained models may have been added to.
This causes an intermittent clash where a job's saved object already
exists, but is in a different space. I don't know why this doesn't fail
on every run.
The fix is to update `cleanMLSavedObjects` so it can take a list of
additional space IDs to also clean. Any test which adds jobs or trained
models to spaces other than `default` need to call this function and
supply the list of space IDs it is using.
I've updated every test I could find in this PR.
2023-07-18 17:29:42 +01:00
Shahzad
41a8a1db01
[Synthetics] Handle a case where settings were never saved (#161834)
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-07-18 18:09:11 +02:00
Shahzad
e66ebdbd24
[Synthetics] Fixes params parsing without complete syntax (#161932)
Co-authored-by: Abdul Wahab Zahid <awahab07@yahoo.com>
2023-07-18 17:44:56 +02:00
Dario Gieselaar
b4e0ecf54a
[Observability AI Assistant] Remove default feedback URL (#162129)
Removing the default feedback URL, as we've not been able to vet the
external endpoint in time.
2023-07-18 17:35:40 +02:00
Anton Dosov
f6e6b77efc
[Serverless] Add navigation functional tests (#161856)
## Summary

close https://github.com/elastic/kibana/issues/160011

This PR adds helpers for testing serverless specific navigation. There
are helpers for sidenav, breadcrumbs, global search, recent items, logo,
checking that no page reload happened during nav.

This PR also adds some serverless specific navigation tests. The should
serve as a navigation smoke check and testing helpers example. Solution
teams can improve them as they see fit.
2023-07-18 17:31:19 +02:00
Tiago Costa
0aa5b217c7
skip flaky suite (#159368, #162051) 2023-07-18 16:29:10 +01:00
Julia Bardi
10c289d86f
[Fleet] added agent logs top errors from 100 hits (#162135)
## Summary

Closes https://github.com/elastic/kibana/issues/148976

Added back top errors from agent logs by querying 100 hits and counting
the top occurrences.


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2023-07-18 17:07:47 +02:00
Rodney Norris
b5965a303f
[Enterprise Search] De-couple Overview from ent-search (#161995)
## Summary

Updated the Overview page to render an connection error callout instead
of the full ErrorConnecting page. This allows us to render the
Elasticsearch product card even without ent-search available.

This required removing the "Insufficient permissions" view since we also
always want to show the elasticsearch product card even if the user
doesn't have access to app search and workplace search.

### Screenshots
<img width="1530" alt="image"
src="2ad5b905-3f42-435b-81e5-a7d71ce8039e">
2023-07-18 09:43:33 -05:00
Sébastien Loix
fde21b15de
[Files management] Fix flaky functional test (#162146) 2023-07-18 07:40:51 -07:00
Marco Antonio Ghiani
e1b4910f3c
[Logs+] Aggregate unmanaged datasets (#162144)
## 📓 Summary

Closes #162061 

This implementation updates and aggregates the unmanaged datasets once
they are retrieved by the state machine on initialization.

I implemented this step in the state machine against doing it on the
dataset service to keep the service pure and able to serve the whole
response in case we need to use it on other plugins/use cases.


a5883521-6dd7-4291-9b90-0cd7665bed65

---------

Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
2023-07-18 16:29:34 +02:00
Nathan Reese
9a87af0cc3
[maps] fix Map layer preview blocks adding layer until all tiles are loaded (#161994)
Closes https://github.com/elastic/kibana/issues/159872

<img width="500" alt="Screen Shot 2023-07-14 at 1 56 45 PM"
src="b4fc762e-dc5e-48dc-a37f-b0936f02b00b">


Do not disable add layer buttons while preview layer is loading

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-07-18 08:20:35 -06:00
Søren Louv-Jansen
52d34d4be7
[APM] Add support for read-only (viewer) user (#161840)
Currently it is not possible for `viewer` or `editor` roles (the
recommended stack roles) to access the diagnostics tool. This PR catches
any permission exception gracefully. User with limited access will
thereby still be able to use the diagnostics tool for the areas where
they have access.
2023-07-18 16:09:37 +02:00
Marco Liberati
e8fefc6304
[Lens] Add new set of functional tests for TSDB (#161463)
## Summary

Fixes #156473

This PR adds a new service to deal with data-stream and TSDB/TSDS to be
used within the Functional test context.

Additionally a new set of tests for the TSDB world:
* Add tests for a dataView with a mixed set of index/streams (raw,
another TSDB stream, downsampled TSDB stream)
* Add tests for "upgraded' streams (`data-stream` => `TSDB`)
* Add tests for "downgraded" TSDB (`TSDB` => `data-stream`)

Also existing tests have been refactored to work directly with
data-streams where possible.

Flaky test runner 100/100  :
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2658

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
2023-07-18 16:04:24 +02:00
Janki Salvi
2796107353
[Cases] Fix flaky tests of severity in list_view (#162137)
## Summary

Fixes #160622


### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

### Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2660
2023-07-18 15:58:38 +02:00
claracruz
2e27e81ba8
[Guided onboarding] Address dark mode issues (#162136)
Guided onboarding plugin should no longer rely on calling uiSettings to
determine which theme Kibana is displayed with

Fixes: #159200
2023-07-18 14:58:19 +01:00
Angela Chuang
26624004d6
[SecuritySolution] Styling for embeddable (#162134)
## Summary


Metrics and histograms on Hosts, network, users, rules and alerts page
are cropped.


Before:

<img width="784" alt="Screenshot 2023-07-18 at 12 40 52"
src="56ade48c-03dd-4095-a3e6-c87b2de895ad">



After:

<img width="766" alt="Screenshot 2023-07-18 at 12 38 48"
src="e0c37138-e043-4075-8842-b7ace669d163">
2023-07-18 06:41:29 -07:00
Sébastien Loix
209d35365f
[Serverless nav] Update footer + project settings cloud links (#161971) 2023-07-18 08:39:14 -05:00
Coen Warmer
af4a047006
Cleanup of field_names folder (#161838)
## Summary

This is a cleanup PR that moves Synthetics fields from the Observability
app to the Observability Shared app.
2023-07-18 15:25:58 +02:00
Davis McPhee
a1090ed1e4
[Discover] Re-enable _request_counts saved search test for SQL mode (#162086)
## Summary

This PR re-enables the disabled saved search functional test for SQL
mode in `test/functional/apps/discover/group3/_request_counts.ts`.
Recent improvements we've made to data fetching in SQL mode fixed the
underlying issue and the test now runs as expected.

Flaky test runs:
- x100:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2654
🟢

### Checklist

- [ ] ~Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)~
- [ ]
~[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials~
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] ~Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard
accessibility](https://webaim.org/techniques/keyboard/))~
- [ ] ~Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))~
- [ ] ~If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)~
- [ ] ~This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))~
- [ ] ~This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)~

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2023-07-18 10:20:50 -03:00
Ievgen Sorokopud
731f587979
[Security Solution] Default Risk score slide bar values are overlaid on Rule Creation page (#161793)
## Summary

Original ticket: #161456

These changes do workaround to display EuiRange component which is
broken due to memoization (see more details in
https://github.com/elastic/kibana/issues/160561 and
https://github.com/elastic/eui/issues/6846)

Before the fix:

<img width="435" alt="Screenshot 2023-07-12 at 19 06 01"
src="8deeb59a-712e-4990-9bda-cba31da8f23f">

After the fix:

<img width="426" alt="Screenshot 2023-07-12 at 19 05 46"
src="7556cca0-d78b-453a-ac47-8a2cd3ef73d7">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Vitalii Dmyterko <92328789+vitaliidm@users.noreply.github.com>
2023-07-18 14:36:07 +02:00
Pablo Machado
c5763bfe1a
[Security Solution] Attempt to fix flaky anomalies test on entity analytics page (#161453)
## Summary

 Attempt to fix flaky anomalies test on entity analytics page.
Test:
`x-pack/plugins/security_solution/cypress/e2e/explore/dashboards/entity_analytics.cy.ts:324`

The test was flaky because it took too long to load the table content. I
added some extra waiting points so that the rest of the application has
already loaded when the table loading starts. It reduced the flakiness,
but to prevent it from happening at all I also increase the default
timeout to 15sec.

Tested it 100 times on CI:

https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2579#_
<img width="786" alt="Screenshot 2023-07-07 at 16 01 35"
src="370b1df8-f6b6-4d27-a509-b149a831a4f7">
2023-07-18 05:17:05 -07:00
Sébastien Loix
c6deb252b2
[TableListView] Fix regression when resetting search (#162034) 2023-07-18 04:47:01 -07:00
Dzmitry Lemechko
72907cfe1e
convert more tests to use uiSettings for timePicker (#161737)
While searching in test files, I found few more places to replace
`PageObjects.timePicker.setAbsoluteRange` with
`PageObjects.common.setTime`.

Before changing I reviewed tests if it is not important to select
datePicker via UI component, but please double check scenarios because I
might miss the context.
2023-07-18 13:27:47 +02:00
Alison Goryachev
75f68da623
Disable management plugins using contextRef (#160671) 2023-07-18 13:21:37 +02:00
Abdul Wahab Zahid
37521304af
[Synthetics] Fix date format for Waterfall and TLS Certificates (#162099)
Fixes #161499 

## Summary
The PR adjusts date format in waterfall flyout and TLS Certificates page
to the common date format used in Synthetics (via `useDateFormat` hook).

|Before|After|
|:---:|:---:|
|<img width="438" alt="Screenshot 2023-07-18 at 00 25 48"
src="f671c5f0-042d-4fa1-910e-bc9deb85d1dd">|<img
width="438" alt="Screenshot 2023-07-17 at 23 40 59"
src="bf19344f-e82e-4188-97a2-65d045544245">|

|Before|After|
|:---:|:---:|
|<img width="1545" alt="Screenshot 2023-07-18 at 00 25 05"
src="ddeb4186-c1e1-4eff-b8ed-4ea6755b3e09">|<img
width="1544" alt="Screenshot 2023-07-17 at 23 53 55"
src="b35a34ea-3d16-4479-82db-0a2f808d0a6c">|
2023-07-18 12:19:47 +02:00
Julia Bardi
82eaddd86c
[Fleet] fix multi space test (#162113)
## Summary

Fixes https://github.com/elastic/kibana/issues/161624

Test failed due to an unrelated endpoint installation error, loading
empty ES to make sure endpoint doesn't cause trouble, as the test
installs system integration.
2023-07-18 11:41:10 +02:00
Julia Bardi
7485343af2
[Fleet] ignore json order in cloud preconfig test (#162110)
## Summary

Fix https://github.com/elastic/kibana/issues/133470
2023-07-18 11:40:59 +02:00
Stratoula Kalafateli
2e1d36a10a
Fix bug on null description (#162106)
## Summary

Closes https://github.com/elastic/kibana/issues/162069

Fixes the validation error on the content management service if the
description is set to null.

I am not sure how the description can be set to null. If I create a Lens
so in 7.17 and I don't set the description then it is automatically set
to '' (empty string)

I can think of 2 ways:
- It was possible in older kibana versions
- Someone changed the SO manually and set this to null

This change fixes it with allowing the schema to also set nullable
values.

Note: Maybe covers the undefined case.

To test it just import the SO given by Bhavya here
https://github.com/elastic/kibana/issues/162069

<img width="2496" alt="image"
src="481ef105-2efb-47c0-9d06-94f7fddbf703">
2023-07-18 12:08:34 +03:00
Jatin Kathuria
0516caed1d
[Security Solution][Fix] Empty Alert Table when upgrading from 8.8.x -> 8.9 (#162063)
## Summary

When users upgrade from `8.8.x` -> `8.9` version, users observe empty
table as shown below.


![image](20549edb-07b9-4124-a0ac-7515cf0e2796)


Below are steps to reproduce this issue and test it:

1. Boot Kibana@v8.8.1
2. Clear Local storage.
3. Go to Security -> Alerts
4. Add Columns `_id` or any other column 
5. Upgrade to `8.9`
6. The table will empty as shown in above screenshot.


## Fix

This fix saperates out the migraton from 8.7 -> 8.8 and add a new
migration for upgrading from 8.8 -> 8.9

`migrateAlertTableStateToTriggerActionsState` migrates table from `v8.7
-> v8.8`,

`migrateTriggerActionsVisibleColumnsAlertTable88xTo89` migrates from
`v8.8.x` -> `v8.9`

Combining both of them may lead to issues when users are migrating from
`v8.7` -> `v8.9` or `v8.8` -> `v8.9`
2023-07-18 10:57:12 +02:00
Aleh Zasypkin
303575544a
Updates header filters for request logs. (#162062)
## Summary

Updates
[header](https://www.elastic.co/guide/en/elasticsearch/reference/master/jwt-auth-realm.html#hmac-oidc-example-request-headers)
filters for request logs.

/cc @elastic/kibana-security
2023-07-18 10:56:07 +02:00
Jill Guyonnet
e1352db526
[Fleet] Exclude Synthetics from per-policy-outputs (#161949)
## Summary

Closes https://github.com/elastic/kibana/issues/155725

### Context

Basic license users cannot use Synthetics private locations and a
default Logstash output (cf. [this
comment](https://github.com/elastic/kibana/issues/155725#issuecomment-1633391319)).
This is because under Basic license per policy output is not allowed.

There was a [similar
issue](https://github.com/elastic/kibana/issues/152234) with Fleet
server that was fixed by adding logic to exclude the fleet server
integration from the checks
([PR](https://github.com/elastic/kibana/pull/153226)).

This PR follows a very similar approach for the synthetics integration.

### Changes

Compare these to [the corresponding changes that were made for Fleet
server](https://github.com/elastic/kibana/pull/153226).

- If the user tries to change the default output from ES to Logstash and
there are agent policies using synthetics, then an error is thrown.
- If an existing Logstash output is made to be the default output and
there are agent policies using synthetics, then these policies will
still be using the previous default ES output.
- If the default output is of type Logstash with an agent policy using
synthetics and another ES output, then if the user tries to delete the
ES output an error is thrown.
- Logstash type outputs are disabled in the agent policy settings if the
policy uses synthetics.

### Steps for testing

⚠️ Ideally the complete scenario reported
[here](https://github.com/elastic/kibana/issues/155725#issuecomment-1633391319)
should be tested. I was having some issues running a dev setup with a
proper Logstash output, so the steps below are not end-to-end (similar
to https://github.com/elastic/kibana/pull/153226).

1. Start with a single default ES output, an agent policy (1) with
synthetics and another (2) without (NB: you can also test combinations
with fleet server; if an error is thrown, the error for fleet server
would be thrown before the one for synthetics).
2. Edit the default ES output to be a Logstash output; this should fail
and thrown the following error:
![Screenshot 2023-07-14 at 15 07
56](bf716a7a-745b-4b89-aa49-0ced2cc825a2)
3. Create a new Logstash output and make it default on creation. Go the
agent policies and check the policies settings: the `Output for
integrations` setting should still be set to the previous ES default
output for policies with synthetics (and/or fleet server), and should
have switched to the new Logstash default for other policies.
4. Repeat the previous steps, but this time first create the Logstash
output and then edit it to make it default. This tests the update
behaviour as opposed to create. You will see the following warning:
![Screenshot 2023-07-14 at 15 06
35](283ee6c2-6d84-4847-bd3d-3c4ff734b2d6)
5. Still with the default Logstash output, try to delete the ES output
(that is still used by policies with synthetics/fleet server(. This
should fail with the following error:
![Screenshot 2023-07-14 at 15 16
03](62dfe4c1-aed6-4d85-83f0-92985eab9336)
6. (Basic license constraint) Create another ES output (not default). Go
the agent policies and check the policies settings: policies with
synthetics (or fleet server) should be allowed to switch the `Output for
integrations` setting to the new ES output:
![Screenshot 2023-07-14 at 15 18
56](98deb4fc-abb1-40e2-9556-1a4ecfa96923)
In contrast, policies without these integrations should be constrained
to the default Logstash output:
![Screenshot 2023-07-14 at 15 19
19](9856de1f-675b-47db-a471-9796239f0f85)

### Checklist

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-07-18 10:53:07 +02:00
Gloria Hornero
a38de17c6e
[Security Solution][Serverless] Adding screenshots folder to Cypress (#162107) 2023-07-18 01:13:47 -07:00
Philippe Oberti
68b8ac3fef
[Security Solution] expandable flyout - add status to flyout header (#161942) 2023-07-18 09:23:13 +02:00
Kibana Machine
f7faa8217b
[api-docs] 2023-07-18 Daily api_docs build (#162104)
Generated by
https://buildkite.com/elastic/kibana-api-docs-daily/builds/402
2023-07-18 01:08:48 -04:00
Yara Tercero
ce0e6b3497
[DE][EQL] Remove negative correlation UI tour (#162102)
Removing tour highlighting negative rule correlation.
2023-07-17 23:20:45 -04:00
Søren Louv-Jansen
c842479d63
[APM] Prevent console.error causing unit tests to fail locally (#161959)
https://github.com/elastic/kibana/pull/161636 fixed a couple of unit
tests that were failing locally but passing on CI. This PR should
prevent this from happening again.

**Why they failed locally and not on CI??**
Locally `console.error` is treated as a test failure:

7ea0dd6b11/x-pack/plugins/apm/jest_setup.js (L12-L15)

Whereas on CI `console.*` is disabled:

a78c7b02b3/packages/kbn-test/src/jest/setup/disable_console_logs.js (L9-L12)

This means that if a test logs `console.error` it would fail locally but
not on CI. This PR changes that so console.error will not cause unit
tests to fail anywhere.
2023-07-17 17:45:58 -07:00
Tiago Costa
fae89ffe62
skip flaky suite (#152204) 2023-07-18 01:10:15 +01:00
Karl Godard
bf195b8493
[D4C] process blocking now enabled for process responses (#162082)
## Summary

Removes restriction on using 'block' action for process responses. The
cloud-defend binary now supports blocking 'fork' and 'exec' events using
bpf LSMs.

For more info on "Defend for containers (D4C)" policies, see here:
https://github.com/elastic/integrations/tree/main/packages/cloud_defend#responses

### Checklist

Delete any items that are not applicable to this PR.

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials. see:
https://github.com/elastic/integrations/pull/6987
2023-07-17 16:35:31 -07:00
Tiago Costa
8a9e09a71d
skip flaky suite (#154073) 2023-07-17 22:13:30 +01:00
Karl Godard
7b31ca96dd
D4C + SessionView usage telemetry (#161385)
## Summary

Ticket: https://github.com/elastic/kibana/issues/161201

An initial pass at adding usageCollection telemetry for cloud_defend
(D4C), as well as some click tracking around the SessionView plugin.

The cloud_defend telemetry schema mirrors that of CSP (see:
https://docs.elastic.dev/security-solution/cloud-security-posture/telemetry/technical-index)
but with metrics relevant to the cloud-defend service instead of kspm
cspm etc... (e.g findings)

The cloud_defend daily telemetry schema can be seen here: 
x-pack/plugins/cloud_defend/server/lib/telemetry/collectors/schema.ts

The biggest difference is that instead of counts around findings/vuln,
it is showing file/process/alert document counts, as well as sending up
yaml and json versions of the cloud-defend policy schema. The json
policy fields are all typed and can be used to run any aggregate query
needed to dig into a a customer's policy usage. e.g which selector
conditions they use, and if they are blocking any operations like
'fork', 'exec', 'createFile', 'deleteFile', etc...

Documentation on how cloud-defend policies work can be found here:
https://github.com/elastic/integrations/tree/main/packages/cloud_defend#policy-example

TODO:
- cloud-defend binary needs to start populating kubernetes_version. I
imagine we could make use of
https://www.elastic.co/guide/en/ecs/8.5/ecs-orchestrator.html#field-orchestrator-cluster-version
for this? cc @norrietaylor

The following click tracking events have been added to session_view:
```
export type SessionViewTelemetryKey =
  | 'loaded_from_cloud_defend_log'
  | 'loaded_from_cloud_defend_alert'
  | 'loaded_from_endpoint_log'
  | 'loaded_from_endpoint_alert'
  | 'loaded_from_unknown_log'
  | 'loaded_from_unknown_alert'
  | 'refresh_clicked'
  | 'process_selected'
  | 'collapse_tree'
  | 'children_opened'
  | 'children_closed'
  | 'alerts_opened'
  | 'alerts_closed'
  | 'details_opened'
  | 'details_closed'
  | 'output_clicked'
  | 'alert_details_loaded'
  | 'disabled_tty_clicked' // tty button clicked when disabled (no data or not enabled)
  | 'tty_loaded' // tty player succesfully loaded
  | 'tty_playback_started'
  | 'tty_playback_stopped'
  | 'verbose_mode_enabled'
  | 'verbose_mode_disabled'
  | 'timestamp_enabled'
  | 'timestamp_disabled'
  | 'search_performed'
  | 'search_next'
  | 'search_previous';
```

Sample output for cloud_defend daily telemetry:
```            
           "cloud_defend": {
              "indices": {
                "alerts": {
                  "doc_count": 116,
                  "deleted": 0,
                  "size_in_bytes": 203482,
                  "last_doc_timestamp": "2023-07-15T02:11:16.478Z"
                },
                "file": {
                  "doc_count": 44,
                  "deleted": 0,
                  "size_in_bytes": 168313,
                  "last_doc_timestamp": "2023-07-15T02:11:16.478Z"
                },
                "process": {
                  "doc_count": 85353,
                  "deleted": 0,
                  "size_in_bytes": 54157433,
                  "last_doc_timestamp": "2023-07-15T02:15:47.214Z"
                },
                "latestPackageVersion": "1.0.7",
                "packageStatus": {
                  "status": "indexed",
                  "installedPackagePolicies": 1,
                  "healthyAgents": 0
                }
              },
              "accounts_stats": [
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "total_doc_count": 85513,
                  "file_doc_count": 160,
                  "process_doc_count": 85353,
                  "alert_doc_count": 116,
                  "kubernetes_version": null,
                  "cloud_provider": "gcp",
                  "agents_count": 3,
                  "nodes_count": 3,
                  "pods_count": 7
                }
              ],
              "pods_stats": [
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "pdcsi-node-shrsp",
                  "container_image_name": "gke.gcr.io/csi-node-driver-registrar",
                  "container_image_tag": "v2.8.0-gke.1",
                  "total_doc_count": 19152,
                  "file_doc_count": 0,
                  "process_doc_count": 19152,
                  "alert_doc_count": 0
                },
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "pdcsi-node-6w5nw",
                  "container_image_name": "gke.gcr.io/csi-node-driver-registrar",
                  "container_image_tag": "v2.8.0-gke.1",
                  "total_doc_count": 19149,
                  "file_doc_count": 0,
                  "process_doc_count": 19149,
                  "alert_doc_count": 0
                },
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "pdcsi-node-ltg8s",
                  "container_image_name": "gke.gcr.io/csi-node-driver-registrar",
                  "container_image_tag": "v2.8.0-gke.1",
                  "total_doc_count": 19148,
                  "file_doc_count": 0,
                  "process_doc_count": 19148,
                  "alert_doc_count": 0
                },
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "kube-proxy-gke-kg-dev-default-pool-9347b91e-rqb0",
                  "container_image_name": "gke.gcr.io/kube-proxy-amd64",
                  "container_image_tag": "v1.26.5-gke.1200",
                  "total_doc_count": 9141,
                  "file_doc_count": 0,
                  "process_doc_count": 9141,
                  "alert_doc_count": 0
                },
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "kube-proxy-gke-kg-dev-default-pool-9347b91e-lflp",
                  "container_image_name": "gke.gcr.io/kube-proxy-amd64",
                  "container_image_tag": "v1.26.5-gke.1200",
                  "total_doc_count": 9139,
                  "file_doc_count": 0,
                  "process_doc_count": 9139,
                  "alert_doc_count": 0
                },
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "kube-proxy-gke-kg-dev-default-pool-9347b91e-t9jd",
                  "container_image_name": "gke.gcr.io/kube-proxy-amd64",
                  "container_image_tag": "v1.26.5-gke.1200",
                  "total_doc_count": 9139,
                  "file_doc_count": 0,
                  "process_doc_count": 9139,
                  "alert_doc_count": 0
                },
                {
                  "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7",
                  "pod_name": "elastic-agent-667qf",
                  "container_image_name": "docker.elastic.co/elastic-agent/elastic-agent",
                  "container_image_tag": "8.8.0",
                  "total_doc_count": 645,
                  "file_doc_count": 160,
                  "process_doc_count": 485,
                  "alert_doc_count": 116
                }
              ],
              "installation_stats": [
                {
                  "package_policy_id": "7814c387-58a4-4e5c-8475-38e86f584971",
                  "package_version": "1.0.7",
                  "created_at": "2023-07-12T19:23:19.432Z",
                  "agent_policy_id": "6bece4a0-20e9-11ee-8d36-0d4244506490",
                  "agent_count": 0,
                  "policy_yaml": """process:
  selectors:
    - name: allProcesses
      operation: [fork, exec]
  responses:
    - match: [allProcesses]
      actions: [log]
file:
  selectors:
    - name: executableChanges
      operation: [createExecutable, modifyExecutable]
  responses:
    - match: [executableChanges]
      actions: [alert]
""",
                  "selectors": [
                    {
                      "name": "allProcesses",
                      "operation": [
                        "fork",
                        "exec"
                      ],
                      "type": "process"
                    },
                    {
                      "name": "executableChanges",
                      "operation": [
                        "createExecutable",
                        "modifyExecutable"
                      ],
                      "type": "file"
                    }
                  ],
                  "responses": [
                    {
                      "match": [
                        "allProcesses"
                      ],
                      "actions": [
                        "log"
                      ],
                      "type": "process"
                    },
                    {
                      "match": [
                        "executableChanges"
                      ],
                      "actions": [
                        "alert"
                      ],
                      "type": "file"
                    }
                  ]
                }
              ]
            },
```

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-07-17 13:45:40 -07:00