## Summary
Resolves https://github.com/elastic/synthetics-dev/issues/290.
~Creating this PR as a place to experiment with Synthetics running
against the Serverless platform. Main goals are to find areas of the
codebase that require revision and get as much of Synthetics functioning
as possible without access to public locations.~
This adds the necessary config and other features to make Synthetics
workable as part of the oblt serverless project. Notably, we aren't
including the `xpack.uptime.enabled` flag here, because we are not ready
to expose the plugin to production users yet. We're going to enable the
plugin on a per-env basis using other means to start, and when we are
ready to expose the plugin generally in prod, we will add that flag.
Also adds nav and fixes a few other things that were broken for
serverless specifically.
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
In this PR we:
* Allow using JWT credentials to grant API keys
* Extend default value of `elasticsearch.requestHeadersWhitelist` to
include both `authorization` and `es-client-authentication` to support
JWT with required client authentication _by default_. See
https://www.elastic.co/guide/en/elasticsearch/reference/8.11/jwt-auth-realm.html#jwt-realm-configuration
* Add API integration tests for both JWTs with client authentication and
without it
__NOTE:__ We're not gating this functionality with the config flag
(`xpack.security.authc.http.jwt.taggedRoutesOnly`) as we did for the
Serverless offering. It'd be a breaking change as we already implicitly
support JWT authentication without client authentication, and to be
honest, it's not really necessary anyway.
## Testing
Refer to the `Testing` section in this PR description:
https://github.com/elastic/kibana/pull/159117.
Or run already pre-configured Kibana functional test server:
1. `node scripts/functional_tests_server.js --config
x-pack/test/security_api_integration/api_keys.config.ts`
2. Create a role mapping for JWT user:
```bash
curl -X POST --location "http://localhost:9220/_security/role_mapping/jwt" \
-H "Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==" \
-H "Accept: application/json" \
-H "Content-Type: application/json" \
-d "{
\"roles\": [ \"superuser\" ],
\"enabled\": true,
\"rules\": { \"all\": [{\"field\" : { \"realm.name\" : \"jwt_with_secret\" }}] }
}"
```
3. Send any Kibana API request with the following credentials:
```bash
curl -X POST --location "xxxx"
-H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2tpYmFuYS5lbGFzdGljLmNvL2p3dC8iLCJzdWIiOiJlbGFzdGljLWFnZW50IiwiYXVkIjoiZWxhc3RpY3NlYXJjaCIsIm5hbWUiOiJFbGFzdGljIEFnZW50IiwiaWF0Ijo5NDY2ODQ4MDAsImV4cCI6NDA3MDkwODgwMH0.P7RHKZlLskS5DfVRqoVO4ivoIq9rXl2-GW6hhC9NvTSkwphYivcjpTVcyENZvxTTvJJNqcyx6rF3T-7otTTIHBOZIMhZauc5dob-sqcN_mT2htqm3BpSdlJlz60TBq6diOtlNhV212gQCEJMPZj0MNj7kZRj_GsECrTaU7FU0A3HAzkbdx15vQJMKZiFbbQCVI7-X2J0bZzQKIWfMHD-VgHFwOe6nomT-jbYIXtCBDd6fNj1zTKRl-_uzjVqNK-h8YW1h6tE4xvZmXyHQ1-9yNKZIWC7iEaPkBLaBKQulLU5MvW3AtVDUhzm6--5H1J85JH5QhRrnKYRon7ZW5q1AQ'
-H 'ES-Client-Authentication: SharedSecret my_super_secret'
....for example....
curl -X GET --location "http://localhost:5620/internal/security/me" \
-H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2tpYmFuYS5lbGFzdGljLmNvL2p3dC8iLCJzdWIiOiJlbGFzdGljLWFnZW50IiwiYXVkIjoiZWxhc3RpY3NlYXJjaCIsIm5hbWUiOiJFbGFzdGljIEFnZW50IiwiaWF0Ijo5NDY2ODQ4MDAsImV4cCI6NDA3MDkwODgwMH0.P7RHKZlLskS5DfVRqoVO4ivoIq9rXl2-GW6hhC9NvTSkwphYivcjpTVcyENZvxTTvJJNqcyx6rF3T-7otTTIHBOZIMhZauc5dob-sqcN_mT2htqm3BpSdlJlz60TBq6diOtlNhV212gQCEJMPZj0MNj7kZRj_GsECrTaU7FU0A3HAzkbdx15vQJMKZiFbbQCVI7-X2J0bZzQKIWfMHD-VgHFwOe6nomT-jbYIXtCBDd6fNj1zTKRl-_uzjVqNK-h8YW1h6tE4xvZmXyHQ1-9yNKZIWC7iEaPkBLaBKQulLU5MvW3AtVDUhzm6--5H1J85JH5QhRrnKYRon7ZW5q1AQ' \
-H 'ES-Client-Authentication: SharedSecret my_super_secret' \
-H "Accept: application/json"
----
{
"username": "elastic-agent",
"roles": [
"superuser"
],
"full_name": null,
"email": null,
"metadata": {
"jwt_claim_sub": "elastic-agent",
"jwt_token_type": "access_token",
"jwt_claim_iss": "https://kibana.elastic.co/jwt/",
"jwt_claim_name": "Elastic Agent",
"jwt_claim_aud": [
"elasticsearch"
]
},
"enabled": true,
"authentication_realm": {
"name": "jwt_with_secret",
"type": "jwt"
},
"lookup_realm": {
"name": "jwt_with_secret",
"type": "jwt"
},
"authentication_type": "realm",
"authentication_provider": {
"type": "http",
"name": "__http__"
},
"elastic_cloud_user": false
}
```
__Fixes:__ https://github.com/elastic/kibana/issues/171522
----
Release note: The default value of the
`elasticsearch.requestHeadersWhitelist` configuration option has been
expanded to include the `es-client-authentication` HTTP header, in
addition to `authorization`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Addresses https://github.com/elastic/kibana/issues/168684
## Summary
This PR adds validations of the user input for a some of the uiSettings
(see https://github.com/elastic/kibana/issues/168684) based on the
`schema` that is provided in the setting definitions. We do this by
exposing a `validateValue` API from the uiSettings browser-side client,
which calls an internal validate route to the server. Then
`validateValue` is called in the `onChange` handlers of the setting
input fields, utilising `debounce` to reduce the number of requests to
the server.
Note: One of the validation requirements is for validating that the
`defaultIndex` setting value is an existing data view. This is a more
complicated validation and will be addressed in a separate PR.
### How to test
**Testing the validations in serverless:**
1. Start Es with `yarn es serverless` and Kibana with `yarn
serverless-{es/oblt/security}`
2. Go to Management -> Advanced settings
3. Verify that the UI provides correct validation error messages for
[the settings that require
validation](https://github.com/elastic/kibana/issues/168684).
**Verify that self-managed Kibana is not affected:**
There should be no validations in self-managed Kibana and setting an
invalid value for an advanced setting should be allowed and shouldn't
cause any errors.
@elastic/kibana-core team, I know you're no longer code owners of the
uiSettings service, but any feedback on those changes will be highly
appreciated. 🙏
<!---
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Per https://github.com/elastic/security-team/issues/7803, the security
solution team would like to increase the timeout for the threat
indicator match rule to 10 minutes. They're experiencing timeouts with
the 1 minute circuit breaker and are comfortable with the trade-offs
this will have.
Monitoring of task consumption has shown the `reports:monitor` task
takes an aggressively high amount of cycles from Kibana Task Manager.
This impacts the throughput of alerts.
This first step is a serverless-only config change to give an immediate
increase in overall task throughput.
It's safe to lower this polling frequency: **the impact is limited to
the responsiveness of retries** when a report job is found to be timed
out. Long-term, the plan will be to tune other parts of the code:
https://github.com/elastic/kibana/issues/170462
## Testing
1. Adjust the Dev mode settings to match a value set in production. Add
this to `config/kibana.dev.yml`:
```
xpack.reporting.capture.maxAttempts: 3 # usually in Dev mode, this is
set to 1
```
2. Start the scripts in different terminal windows to run Elasticsearch
and Kibana dev servers
```
yarn es serverless
---------------
yarn serverless
```
3. Open a search in Discover that covers about 4,000 hits and request a
CSV export using the Share menu.
4. Monitor the Kibana server logs and wait until the background job
begins. Restart the server while the job is executing (saving a file in
code under `packages/` or `server/` will trigger a restart).
5. Around 3 minutes after restart, the report job will get a re-attempt
Compare this behavior vs in non-serverless, where a report job will get
a re-attempt around 3 seconds after restart.
## Summary
This PR removes reporting deprecation log messages in kibana server logs
on serverless environments.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>
Adds a new `tags` property to our ML Modules which contains an array of
strings to used to filter the modules returned when calling
`/internal/ml/modules/get_module` or `/internal/ml/modules/recognize`
Adds a new kibana config setting `xpack.ml.compatibleModuleType` which
will enforce a module filter.
This setting supports the values `security`, `observability` or `search`
and will be used by the serverless projects to ensure only modules
relevant to the current project are discoverable.
When matching against the `xpack.ml.compatibleModuleType` setting,
modules with no tags will be returned along with modules which match the
`compatibleModuleType` value.
The endpoints `/internal/ml/modules/get_module` and
`/internal/ml/modules/recognize` can also take a `filter` query
parameter in the form of a comma separated list of values. If any of
these filter values match a module's tags, the module will be returned.
Modules with no tags will not be matched when using the `filter`
parameter.
This PR also updates the security plugin to apply a `security` filter
when retrieving modules and running recognize. This will improve
performance, especially for recognize which runs the queries from every
module on the supplied index pattern.
Examples
**Running the Security serverless project.**
Request:
`/internal/ml/modules/get_module`
Response:
All modules containing `tags: ["security"]`
Plus all modules with no `tags` or `tags: []`
Request:
`/internal/ml/modules/get_module?filter=observability`
Response:
An empty list
**Running stateful es**
Request:
`/internal/ml/modules/get_module`
Response:
All modules
Request:
`/internal/ml/modules/get_module?filter=security`
Response:
Only modules containing `tags: ["security"]`
Request:
`/internal/ml/modules/get_module?filter=security,observability`
Response:
Modules containing `tags: ["security"]` and `tags: ["observability"]`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Branch with Infra UI enabled in Serverless.
This exists for us to have a working environment based on main, but with
infra turned on. We could merge this at a later date when we plan to
enable Infra on serverless.
Fixes https://github.com/elastic/kibana/issues/167850Fixes#168065Fixes#168853
Fixes https://github.com/elastic/kibana/issues/167654
## Summary
This PR removes the Storage size column in Data stream in serverless as
the data stream stats API on serverless doesn't currently return the
storage size.
### How to test:
Verify that the column is not displayed in serverless:
1. Start Es with `yarn es serverless` and Kibana with `yarn
serverless-{es/oblt/security}`
2. Go to Stack Management -> Index Management -> Data Streams
3. Switch on the "Include stats" toggle
4. Verify that the "Storage size" column is not shown in the table.
Verify that the column is displayed in stateful:
1. Start Es with `yarn es snapshot` and Kibana with `yarn start`
2. Go to Stack Management -> Index Management -> Data Streams
3. Switch on the "Include stats" toggle
4. Verify that the "Storage size" column is shown in the table.
<!---
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
## Summary
Fixes https://github.com/elastic/kibana/issues/165895
This PR limits which index settings are displayed on the index details
page, "Settings" tab in the edit mode. On serverless only a handful of
index settings will be editable by the user. The UI only prevents
displaying some index settings, but it's still possible for the user to
type in a setting that can't be edited. That is the case on dedicated as
well.
### How to test
1. Start Serverless ES and Kibana
2. Navigate to Index Management and create a test index
3. Click on the index name and on the details page click the tab
"Settings"
4. Toggle the "Edit mode" switch and verify that only editable settings
are displayed.
#### Screenshot
<img width="527" alt="Screenshot 2023-10-16 at 20 25 49"
src="e6678cca-3494-4c63-ae66-ace9c823d12d">
## Summary
Closes https://github.com/elastic/ingest-dev/issues/2396
Added a new kibana task that publishes Agent metrics every minute to
data streams installed by fleet_server package.
Opened the pr for review, there are a few things to finalize, but the
core logic won't change much.
To test locally:
- Install fleet_server package 1.4.0 from
[this](https://github.com/elastic/integrations/pull/8145) pr to get the
mappings
- Start kibana locally, wait for a few minutes for the metrics task to
run (every minute)
- Go to discover, `metrics-*` index pattern, filter on
`data_stream.dataset: fleet_server.*`
- Expect data to be populated in `fleet_server.agent_status` and
`fleet_server.agent_versions` datasets.
<img width="1787" alt="image"
src="615af9df-fe4b-4c17-8c8c-88646c403a18">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
fix https://github.com/elastic/kibana/issues/166233 (this is a short
term fix that removes "analytics" references from the empty state page
in serverless observability analytics apps)
## Summary
This PR implements a standard way to have different static settings for
the serverless and ess (stateful) environments. It centralizes flags,
which were set using different approaches previously, in a single
configuration.
This aims to make it easier for developers to enable/disable parts of
the application in serverless projects.
Default:
```
sideNavEnabled: true,
ILMEnabled: true,
ESQLEnabled: true,
```
Serverless:
```
xpack.securitySolution.offeringSettings: {
sideNavEnabled: false, # Internal security side navigation disabled, the serverless global chrome navigation is used instead
ILMEnabled: false, # Index Lifecycle Management (ILM) functionalities disabled, not supported by serverless Elasticsearch
ESQLEnabled: false, # ES|QL disabled, not supported by serverless Elasticsearch
}
```
### Consume the settings
#### Server
- Plugin parsed `ConfigType`:
`this.config.settings.ESQLEnabled`
#### UI
- Plugin attribute:
`this.configSettings.ESQLEnabled`.
- Components can access it from Kibana services:
`useKibana().services.configSettings.ESQLEnabled;`
---------
Co-authored-by: Vitalii Dmyterko <92328789+vitaliidm@users.noreply.github.com>
## Summary
This PR leverages the work done
[here](https://github.com/elastic/kibana/pull/165596) to introduce ES|QL
into timeline. The goal of this PR is to provide security users easy
access to ESQL from within the security solution. It will be released in
`technical preview` for the 8.11 release.
<img width="1725" alt="image"
src="0e275cf7-bbce-476b-b1dc-8936427ad14f">
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Jatin Kathuria <jatin.kathuria@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jatin Kathuria <jtn.kathuria@gmail.com>
Switches to using the serverless config file to enabled/disable ML
features rather than a function shared from the setup contract.
Storing these flags in a config file means they are already available
when setup runs and so can be used when registering integrations into
other plugins.
Removes the dependency on ML from `security_solution_serverless`,
`serverless_observability` and `serverless_search`
## Summary
Updated the grok debugger and painless lab plugins to handle the
`enabled` config flag so they can easily be disabled for serverless
search.
### Screenshots
Before:
After:
<img width="1914" alt="image"
src="ad48ba1b-0f1d-4da3-aeac-4868781f6832">
Resolves https://github.com/elastic/kibana/issues/162264
## Summary
Adds a limit on the maximum number of actions that can be queued with a
circuit breaker. The limit in serverless is set to 10,000, and 1,000,000
in the other environments.
- If a rule execution exceeds the limit, the circuit breaker kicks in
and stops triggering actions.
- Alerting rule's status updated to warning when circuit breaker is hit
Did not update the `enqueueExecution` bc it's going to be removed in
https://github.com/elastic/kibana/pull/165120.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To Verify
- Create a 2 rules that have actions
- Set `xpack.actions.queued.max` in kibana.yml to a low number like 2 or
3
- Use the run soon button to queue up actions and hit the circuit
breaker.
- The actions will not be scheduled and the rule status will be set to
warning
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves: https://github.com/elastic/kibana/issues/162262
This PR is the backend changes to add a circuit breaker
`xpack.alerting.rules.maxScheduledPerMinute` to both serverless and
other environments that limits the number of rules to 400 runs / minute
and 10000 runs / minute, respectively. There will be another PR to
follow this one that gives the user UI hints when creating/editing rules
that go over this limit.
This circuit breaker check is applied to the following routes:
- Create Rule
- Update Rule
- Enable Rule
- Bulk Enable Rule
- Bulk Edit Rule
Also adds a new route: `/internal/alerting/rules/_schedule_frequency` to
get the current total schedules per minute (of enabled rules) and the
remaining interval allotment.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>
## Summary
This moves connectors functions to a shared package so Serverless Search
and Enterprise Search can both use it.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/137810
Part of https://github.com/elastic/kibana/issues/163248
This PR is the **first iteration** of the new ES|QL language in kibana.
The majority of the functionality is based on the existing functionality
for SQL (which has already been merged). This builds on top of it to
enable the functionality for ES|QL. We decided to remove SQL for now so
this PR enables ES|QL and hides SQL. We are not removing the code for
now.
### Important notes:
- This PR contains basic autocomplete functionality for the ES|QL
commands. We want to improve it in follow up PRs
- The majority of the tests for SQL were moved to work with ES|QL
instead
- The search strategy is a very simple endpoint for now as we don't have
async search nor pagination
- Now that we remove SQL, the ui for ES|QL selection has changed
- The documentation for ESQL has been handled by the docs team so it is
already reviewed and in sync with the official documentation
- ES|QL is disabled in serverless projects for now
### Changes from SQL:
- The Discover histogram now is being created with ES|QL (using the
date_trunc function). This gives it the ability to be saved on a
dashboard and also to be edited inline.
- ES|QL sometimes returns some warnings (on the search headers). For
example when we are trying to date parse a string that doesnt contain a
valid date). These warnings are also reported on the UI. There is a bug
in ES and the warning doesnt come always
https://github.com/elastic/elasticsearch-internal/issues/1465)
- We have 2 types of commands:
- Transformational commands (keep, stats) -> when they exist in the
query Discover should render the Table view (selected columns)
- Non transformational commands (all the others) -> in that case
Discover renders the Document view
- ESQL switch on advanced settings is now on by default
**Discover view with non transformational commands**
<img width="1678" alt="image"
src="abe100e1-01e9-4fe0-9b89-6d8bdf6443fc">
**Discover view with transformational commands**
<img width="1679" alt="image"
src="e46af422-daeb-4be5-88cf-522211674ff5">
### Missing
- ESQL autocomplete is not perfect, we are going to work on it on a
future PR. Specifically:
- There is a bug in autocomplete, sometimes writing a query overwrites
the existing one, will deal with it in a future PR
- Further improvements
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Alexey Antonov <alexwizp@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Peter Pisljar <peter.pisljar@elastic.co>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
## Summary
Enable ZDT migrations to run on `migrator` node roles (the default).
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Issues when rolling out Kibana's | Med. | Low | It's possible this
change could lead to interruption in internal Elastic environments |
This adds an empty connectors page to the Elasticsearch project. The
Enterprise Search plugin exports relevant connectors data and functions,
the Search plugin simply consumes these.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#161337
## Summary
Uses build flavor(see #161930) to disable specific Kibana security,
spaces, and encrypted saved objects HTTP API routes in serverless (see
details in #161337). HTTP APIs that will be public in serverless have
been handled in #162523.
**IMPORTANT: This PR leaves login, user, and role routes enabled. The
primary reason for this is due to several testing mechanisms that rely
on basic authentication and custom roles (UI, Cypress). These tests will
be modified to use SAML authentication and serverless roles in the
immediate future. Once this occurs, we will disable these routes.**
### Testing
This PR also implements testing API access in serverless.
- The testing strategy for disabled routes in serverless is to verify a
`404 not found `response.
- The testing strategy for internal access routes in serverless is to
verify that without the internal request header
(`x-elastic-internal-origin`), a `400 bad request response` is received,
then verify that with the internal request header, a `200 ok response`
is received.
- The strategy for public routes in serverless is to verify a `200 ok`
or `203 redirect` is received.
~~blocked by #161930~~
~~blocked by #162149 for test implementation~~
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@elastic.co>
