## Summary
Add the `use_text_extraction_service` rich configurable field to some
native connectors.
Native connectors will not have access to the feature, but the field is
required in case the user converts their native connector to a
self-managed connector.
## Summary
This PR introduces grouped common configs in serverless project FTR
tests.
### Details
* With increasing number of added tests, we're running into issues with
test run time (configurations are stopped at 40 minutes)
* This PR moves the inclusion of `common` tests from the projects' main
`config.ts` file to `common_configs/config.group1.ts`, which can easily
be extended
* As part of that, `common` tests in `api_integration/test_suites` and
`functional/test_suites` are re-organized to no longer contain a top
level index file
* Created sub-directories and index files where needed
* This makes it easier to group `common` tests when including them in
project config files
### Additional changes
* Add README files to
`x-pack/test_serverless/[api_integration|functional]/test_suites/common`
* Rename `security` directory in `common` tests to `platform_security`
to avoid confusion with the `security` project type
* Include sample data test suite in an index file (this suite wasn't
included so far and didn't run at all) and prepared it for actually
working in serverless
* it's still failing and should be fixed soon - skipped it for now
## Summary
Let's automate E2E against Serverless
Changelog:
- updated certs to include additional dns names we are using for testing
locally, `host.docker.internal`, `es01`
- updated certs generation README to include changes related to
`openssl@3`
- added new certs for Fleet server
- added fleet-server service token
- added support for `ca_trusted_fingerprint` in fleet preconfig
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tomasz Ciecierski <ciecierskitomek@gmail.com>
Co-authored-by: Tomasz Ciecierski <tomasz.ciecierski@elastic.co>
Co-authored-by: Kevin Logan <kevin.logan@elastic.co>
Closes#163282
## Summary
This PR:
* Adds a `featureFlags.metricsExplorerEnabled` property to the Infra
plugin config to enable and disable Metrics Explorer depending on the
offering type
* Prevents `MetricsExplorerViewsService` initialization for serveless
based on the feature flag
* Prevents creating Metrics Explorer frontend routes when in serverless
* Prevents registration of the MetricsExplorerViews saved object when in
serverless
* Prevents initialization of the `metrics_explorer_views` API routes
when in serverless
**Trying to access Metrics Explorer in serverless**
<img width="1829" alt="CleanShot 2023-09-22 at 12 59 35@2x"
src="2b039925-0f0b-4c07-be29-bbe910de7a34">
**Trying to access views API**
<img width="1829" alt="CleanShot 2023-09-22 at 13 00 00@2x"
src="15269ec2-becd-4ee3-9b5e-d916df28a7b8">
**`infra/metrics_explorer` API still works as per ticket requirements**
<img width="1829" alt="CleanShot 2023-09-22 at 13 00 06@2x"
src="fb23f912-c6fd-46c8-9084-c17c51e5b064">
## How to test
* Checkout locally
* Enable Infra in `serverless.oblt.yml`: `xpack.infra.enabled: true`
* Run Kibana in serverless mode
* Try accessing `/app/metrics/explorer` route and make sure it's not
available
* Make sure other Infra routes (`/app/metrics/inventory` and
`/app/metrics/hosts`) still load as expected
* In Kibana dev console make sure you get 404 for `GET
kbn:/api/infra/metrics_explorer_views`
* Also check that you don't see `metrics-explorer-view` saved object in
the response for `GET
kbn:/api/kibana/management/saved_objects/_allowed_types`
* Run Kibana in non-serverless mode and make sure Metrics Explorer is
accessible and works as usual
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/167387
Replaced using kibana version when deciding if agent upgrade is
available (only in serverless, in stateful kibana version is still
returned as an available version).
To verify locally:
- [to test stateless] add this to `kibana.dev.yml`:
`xpack.fleet.internal.onlyAllowAgentUpgradeToKnownVersions: true`
- extract the `agent_versions_list.json` to local kibana folder
`~/kibana/x-pack/plugins/fleet/target`
[agent_versions_list.json.zip](12739519/agent_versions_list.json.zip)
- verify that upgrade available warnings still work if agent is < latest
agent version (8.10.2)
- when trying to upgrade agent, verify that the default version is the
latest agent version, and 8.11 is not in the list
Agent list:
<img width="1475" alt="image"
src="f06b7bc8-97e6-4ff9-b872-736ede5e969a">
Upgrade available filter - 1 agent on latest version, 9 upgradeable:
<img width="1314" alt="image"
src="4ff5ac02-903b-493b-94df-68b1b7ad6846">
Agent details:
<img width="1512" alt="image"
src="3ff6e1d5-2ccc-4814-83e5-c4760ad63722">
Agent on latest version has disable `Upgrade agent` action:
<img width="1322" alt="image"
src="f461dbf5-04e5-4bcc-8801-48c2b1a90225">
Bulk action with one agent that is not upgradeable (already on latest
version), expected error:
<img width="1597" alt="image"
src="8bfa46ae-6684-4748-9fca-e908c142b642">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
I decided to remove the backticks suggestion as it was a copy paste from
SQL and is not going to solve ES|QL related errors. It also creates
confusion for many users.
<img width="1677" alt="image"
src="fe2a4fcb-c3e0-4d87-8568-32e7525f70d5">
Improves the display of long descriptions of transforms in the Transform
management page and when editing the description in the transform wizard
or edit flyout.
Previously If there was a long description, the text would not be
wrapped in the table on the management page, and it would not be
possible to view the full text in the text input when editing. This PR
adds line wrapping for the description column, and uses a text area for
editing the text.
Part of https://github.com/elastic/kibana/issues/163147
## Summary
Fixes a bug in ES|QL mode where you:
- have a query of index pattern 1 and select some fields
- change the query by using a different index pattern
- the selected columns do not reset
**BUG**
**NOW**
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Implement risk score engine telemetry
Here we use 2 types of telemetry:
Event base telemetry:
- Risk execution success. With parameters `scoresWritten`,
`taskCompletionTimeSeconds`, `isRunMoreThanInteval`
- Risk execution error
Usage telemetry:
- `unique_user_risk_score_total` and `unique_host_risk_score_total` -
Total amount from latest transform index for host and users
- `unique_user_risk_score_day` and `unique_host_risk_score_day` - Last
day amount from the latest transform index for host and users
- `all_host_risk_scores_total` and `all_user_risk_scores_total` - Total
amount from datastream for all risk executions for host and users
- `all_host_risk_scores_total_day` and `all_user_risk_scores_total_day`
- Last day amount from datastream for all risk executions for host and
users
- `all_risk_scores_index_size` and `unique_risk_scores_index_size` -
sizes of datastream of all risk scores and latest transform index
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Updates the exceptions flyout UI `match_any` operator to accept numerous
duplicate values that differ in case. Prior to this change, a user could
not add a field value of `foo` and `FOO` - the UI would display that the
value is a duplicate. We now will allow this as exceptions are case
sensitive and this is a necessary use case for the current exceptions
behavior.
Cypress tests and FTR tests are added.
## Summary
This is hopefully the last batch of typescript issues to be fixed,
related to https://github.com/elastic/kibana/pull/166813.
It's also re-enabling full typecheck, with this, we should be back in a
clean, typechecked main branch.
Blocked by #167428
---------
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Co-authored-by: Brad White <brad.white@elastic.co>
Co-authored-by: Thomas Watson <watson@elastic.co>
Co-authored-by: Patryk Kopyciński <contact@patrykkopycinski.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The osquery-cypress package was referring to configurations from a
parent folder. This caused difficulties with having to exclude/include
the same files in the `tsconfig.json`s, leading to hard-to-resolve
typescript issues.
This PR moves the configs in, and cleans up redundancies and type
errors.
chore(osquery): move osquery/cypress configs to cypress folder, fix type
errors
---------
Co-authored-by: Patryk Kopyciński <contact@patrykkopycinski.com>
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Co-authored-by: Brad White <brad.white@elastic.co>
## Summary
This PR is for adding the GCP Organization option as well as updating
the Single option to include Project ID field. Still rough
Changes:
- Added GCP Organization Option
- Project ID field now exist on Google Cloud Shell Single option as well
as Organization Option
- Organization ID field added to the form when user chose account_type :
GCP Organization
- Project ID are now optional (previously users aren't able to save the
integration without filling in the Project ID)
- Removed Beta tag for CIS GCP
TODO:
- Make sure previous installation using previous wont break because of
the new fields and requirement (migration)
- More tests
- Clean up
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#167075
## Summary
Adds a public asset client available in the `setup` lifecycle hook for
plugins that depend on this one. `getHosts` is the only method available
on this client for now.
TODO, before merge:
- [x] Add docs for the server client
- [x] Add docs for the public client
- [x] Remove REST docs from plugin docs, not needed
- [x] Add unit tests for public client
### Testing this PR
One way of testing this new client is to apply the attached
test-assets.patch file locally, adjust the date range in the getHosts
query that is added in the infra plugin, and then start Kibana and
navigate to the infra app. You should see print out in the browser
console.
[test-assets.patch](12718693/test-assets.patch)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR supercedes https://github.com/elastic/kibana/pull/167107
Elastic Defend for Windows now collects ETW Threat Intelligence (ETW-TI)
events. Defend calls these API events. API events currently include the
existing Credential Access and ETW-TI. We will add more events under the
API umbrella in the future.
The Windows Events Policy `Credential Access` category has been renamed
to `API` in the UI and documentation - but it remains as
`credential_access` in the yaml for backwards compatibility. This new
category definition is a superset of the previous category.
Two new advanced options are added -
* `windows.advanced.events.api_disabled` - comma separated list
* `windows.advanced.events.api_verbose` - boolean
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Resolves https://github.com/elastic/kibana/issues/165947
Resolves https://github.com/elastic/actionable-observability/issues/124
### Summary
This PR adds an Embeddable SLO Overview Widget to the Dashboard app. It
uses a [Metric
chart](https://elastic.github.io/elastic-charts/?path=/story/metric-alpha--basic)
component and displays an overview of the SLO health:
- name
- current sli value
- target
- status (background color)
### ✔️ Acceptance criteria
- The SLO widget should display the basic information listed above
- The SLO widget should be clickable and lead to the slo detail page
- The user should be able to select the SLO and filter to instanceId
- The tag "url.domain:mail.co" is the partition field and instanceId
value
<img width="1189" alt="Screenshot 2023-09-21 at 21 07 23"
src="03539b9d-23a5-45eb-aafb-df42e9421f77">
For more information regarding the key concepts and the usage of an
embeddable you can have a look at the Embeddable plugin
[README](https://github.com/elastic/kibana/tree/main/src/plugins/embeddable)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
### Error 1: Strict mode throws an error on non-filled variables. I
forgot to add this when creating the script
```
.buildkite/scripts/steps/es_serverless/promote_es_serverless_image.sh: line 40: UPLOAD_MANIFEST: unbound variable
```
### Error 2: Uploading multi-arch images
https://elastic.slack.com/archives/C5UDAFZQU/p1695725623585409
We've noticed that the downloaded `latest-verified` images are slower to
start up than normal. After inspecting the manifests, it seems we were
getting `linux/amd64` on our arm devices as well.
The solution is to grab and upload both platform variants. (using this
blog:
https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/)
This PR is next step completing discover timeline integration. All
previous/nest steps have been defined here:
https://github.com/elastic/security-team/issues/6677
## Summary
This PR implements the integration between timeline State v/s Discover
State. The purpose of this PR is to add functionality related to the
persistence of saved search which will always be linked to the timeline
user is working in.
Below diagram shows briefly how saved search is working with timeline.
```mermaid
graph TD;
DS(Discover State) -. user updates .-> SS(Saved Search);
SS(Saved Search) -. updates savedSearchId .-> TS(Timeline State) ;
TS(Timeline State) -. restores Saved Search to App state .->DS(Discover State);
```
Primarily, this PR implements below technical components:
1. `DiscoverInTimleineContext` : provides the ability across security
solution to manipulate discover state.
2. `useDiscoverInTimelineActions`: acts as a helper to provide
short-hand actions to manipulate discover state. For eg.
`resetDiscoverAppState` or `restoreAppStateFromSavedSearch`.
Here is the small demo video:
006465ba-19ce-4209-ac46-21dbb746508d
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>
## Summary
This PR follows #166460 by adding Category panels to the Form.
<img width="1807" alt="Screenshot 2023-09-27 at 3 36 16 PM"
src="2abe8cf5-5822-473f-affd-148fb7949316">
## Notes
This PR is divided into several commits, the first few being
prerequisite codemods. I recommend reviewing each commit separately, as
the codemods might obscure the actual component work.
- [e78586f - Make SettingType pre-defined to clean up
references](e78586fe44)
- This makes the `SettingType` optional, to clean up areas where the
generic need not be specific.
- [80a9988 - [codemod] Make onFieldChange and onInputChange more
distinct](80a9988516)
- The `onChange` handlers weren't very clear as you work your way up the
component tree. This makes the implementation and usage easier to
understand, (and easier to [replace with state
management](https://github.com/elastic/kibana/issues/166579)).
- [5d0beff - [fix] Fix logged errors in form
tests](5d0beff00c)
- This fixes some logged errors in the Form from `Monaco` and from some
missing `act` and `waitFor` calls.
Closes#161754Closes#166807
To make the testing and review easier I merged the old components
[cleanup PR](https://github.com/jennypavlova/kibana/pull/5) into this
one
## Summary
This PR replaces the old node details view with the asset details flyout
### Old
### New
### Testing
1. Go to inventory
2. Click on a host in the waffle map
3. Click on any **host**
- These changes are related only if a `Host` is selected- in the case of
a pod the view shouldn't be changed:
4. Check the new flyout functionality
3557821c-7964-466e-8514-84c2f81bc2fd
Note: the selected host should have a border like in the previous
version (this I fixed in the [last
commit](ff4753aa06))
so it should be added if there is a selected node:
<img width="1193" alt="image"
src="6646fe47-6333-435a-a5ec-248339402224">
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Correct the `license_uuid` field name in the Endpoint Policy. Before, it
was named `license_uid`, but the Endpoint expects `license_uuid`.
This PR in intended to be backported to `8.10.3` which brings up an
interesting problem since we already have a migration added to `main`
for the `8.11` release.
After talking with the kibana-core team, my approach is to add the
migration for this bug fix to this PR. Then, to keep the `modelVersions`
consistent, I will backport all `modelVersions` to `8.10.3` to keep the
migrations consistent. Keeping these consistent is important so that
both users upgrading from `8.10.x` in ESS and the Serverless line all
remain in sync. The end result is that the policies inside of of
`8.10.3` will have an extra field that will be unused until `8.11.0`
The following `8.10.3` backport for this will include the extra
migration and I will request reviews for it since it will be more than a
normal backport.
Policy:
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
* Adds API versioning to all routes involved in new Risk Engine, public
and private
* Adds missing PLI auth headers for some routes
* Updates API invocations to specify an appropriate version header
* Does NOT add header to legacy transform-based EA routes
### Checklist
- [x] Verify no API calls from the UI were missed
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
- Adds the missing `system_indices_superuser` role to the `role.yml`
file used when starting ES for serverless and using the security
solution override files for resources
## Summary
This PR adds Gated form when user visit Workplace Search , when`
kibana_uis_enabled == false`. User will not be able to able to access
any other Workplace search routes other than Overview page.
**Note**: Form submission and API call be included in next PR
Screen Recordings :
672a7b2e-3e5f-4fa1-8535-b5080b3a2dfc
7c8129cf-6f50-4039-9b50-b9a655361bd1
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Updated EMS Styles are waiting to be put into production. They are
already available in Elastic staging environment
([preview](maps-staging.elastic.co/?manifest=testing)). This PR is a
safe measure to ensure that this change do not break our CI tests.
The process has been as follows:
1. Momentarily replaces the EMS Tile Service `tileApiUrl` by our staging
server to force the use of the new styles and check which tests break
with the slightly different basemaps at
[12481c6](12481c6ada)
2. Look for related [broken
tests](https://buildkite.com/elastic/kibana-pull-request/builds/161870)
```
Error: expected 0.030813687704837327 to be below 0.03
```
4. Adjust the threshold for the dashboard report, since the new value
was slightly over the limit
[e655b84](e655b84569)
5. Wait for a green CI (this took a few days because of unrelated issues
with Kibana CI)
6. Revert the `tileApiUrl` change to its original value
[c0030bc](c0030bcff1)
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
