## Summary
Updated the grok debugger and painless lab plugins to handle the
`enabled` config flag so they can easily be disabled for serverless
search.
### Screenshots
Before:
After:
<img width="1914" alt="image"
src="ad48ba1b-0f1d-4da3-aeac-4868781f6832">
Resolves https://github.com/elastic/kibana/issues/162264
## Summary
Adds a limit on the maximum number of actions that can be queued with a
circuit breaker. The limit in serverless is set to 10,000, and 1,000,000
in the other environments.
- If a rule execution exceeds the limit, the circuit breaker kicks in
and stops triggering actions.
- Alerting rule's status updated to warning when circuit breaker is hit
Did not update the `enqueueExecution` bc it's going to be removed in
https://github.com/elastic/kibana/pull/165120.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To Verify
- Create a 2 rules that have actions
- Set `xpack.actions.queued.max` in kibana.yml to a low number like 2 or
3
- Use the run soon button to queue up actions and hit the circuit
breaker.
- The actions will not be scheduled and the rule status will be set to
warning
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves: https://github.com/elastic/kibana/issues/162262
This PR is the backend changes to add a circuit breaker
`xpack.alerting.rules.maxScheduledPerMinute` to both serverless and
other environments that limits the number of rules to 400 runs / minute
and 10000 runs / minute, respectively. There will be another PR to
follow this one that gives the user UI hints when creating/editing rules
that go over this limit.
This circuit breaker check is applied to the following routes:
- Create Rule
- Update Rule
- Enable Rule
- Bulk Enable Rule
- Bulk Edit Rule
Also adds a new route: `/internal/alerting/rules/_schedule_frequency` to
get the current total schedules per minute (of enabled rules) and the
remaining interval allotment.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>
## Summary
This moves connectors functions to a shared package so Serverless Search
and Enterprise Search can both use it.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/137810
Part of https://github.com/elastic/kibana/issues/163248
This PR is the **first iteration** of the new ES|QL language in kibana.
The majority of the functionality is based on the existing functionality
for SQL (which has already been merged). This builds on top of it to
enable the functionality for ES|QL. We decided to remove SQL for now so
this PR enables ES|QL and hides SQL. We are not removing the code for
now.
### Important notes:
- This PR contains basic autocomplete functionality for the ES|QL
commands. We want to improve it in follow up PRs
- The majority of the tests for SQL were moved to work with ES|QL
instead
- The search strategy is a very simple endpoint for now as we don't have
async search nor pagination
- Now that we remove SQL, the ui for ES|QL selection has changed
- The documentation for ESQL has been handled by the docs team so it is
already reviewed and in sync with the official documentation
- ES|QL is disabled in serverless projects for now
### Changes from SQL:
- The Discover histogram now is being created with ES|QL (using the
date_trunc function). This gives it the ability to be saved on a
dashboard and also to be edited inline.
- ES|QL sometimes returns some warnings (on the search headers). For
example when we are trying to date parse a string that doesnt contain a
valid date). These warnings are also reported on the UI. There is a bug
in ES and the warning doesnt come always
https://github.com/elastic/elasticsearch-internal/issues/1465)
- We have 2 types of commands:
- Transformational commands (keep, stats) -> when they exist in the
query Discover should render the Table view (selected columns)
- Non transformational commands (all the others) -> in that case
Discover renders the Document view
- ESQL switch on advanced settings is now on by default
**Discover view with non transformational commands**
<img width="1678" alt="image"
src="abe100e1-01e9-4fe0-9b89-6d8bdf6443fc">
**Discover view with transformational commands**
<img width="1679" alt="image"
src="e46af422-daeb-4be5-88cf-522211674ff5">
### Missing
- ESQL autocomplete is not perfect, we are going to work on it on a
future PR. Specifically:
- There is a bug in autocomplete, sometimes writing a query overwrites
the existing one, will deal with it in a future PR
- Further improvements
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Alexey Antonov <alexwizp@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Peter Pisljar <peter.pisljar@elastic.co>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
## Summary
Enable ZDT migrations to run on `migrator` node roles (the default).
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Issues when rolling out Kibana's | Med. | Low | It's possible this
change could lead to interruption in internal Elastic environments |
This adds an empty connectors page to the Elasticsearch project. The
Enterprise Search plugin exports relevant connectors data and functions,
the Search plugin simply consumes these.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#161337
## Summary
Uses build flavor(see #161930) to disable specific Kibana security,
spaces, and encrypted saved objects HTTP API routes in serverless (see
details in #161337). HTTP APIs that will be public in serverless have
been handled in #162523.
**IMPORTANT: This PR leaves login, user, and role routes enabled. The
primary reason for this is due to several testing mechanisms that rely
on basic authentication and custom roles (UI, Cypress). These tests will
be modified to use SAML authentication and serverless roles in the
immediate future. Once this occurs, we will disable these routes.**
### Testing
This PR also implements testing API access in serverless.
- The testing strategy for disabled routes in serverless is to verify a
`404 not found `response.
- The testing strategy for internal access routes in serverless is to
verify that without the internal request header
(`x-elastic-internal-origin`), a `400 bad request response` is received,
then verify that with the internal request header, a `200 ok response`
is received.
- The strategy for public routes in serverless is to verify a `200 ok`
or `203 redirect` is received.
~~blocked by #161930~~
~~blocked by #162149 for test implementation~~
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@elastic.co>
Part of https://github.com/elastic/kibana/issues/155764.
In this PR, I'm setting the
`xpack.task_manager.allow_reading_invalid_state` serverless setting to
`false` so Kibana doesn't allow reading invalid state when loading
tasks.
I'm also doing the same for the functional tests to ensure valid task
state is always read.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/163340
This PR disables graph in serverless. Specifically:
- creates a serverless yml setting for disabling graph
- adds the setting in serverless.yml
## Summary
fixes https://github.com/elastic/kibana/issues/160337
This PR
- hides cases in the serverless Elasticsearch project, cases APIs throw
error
- throws 403 from API when `owner=cases` for security or observability
serverless mode
- verifies the behaviour in serverless functional as well as
api_integration tests
**How to test**
- Boot up `es` serverless solution and make sure that `cases` from the
navbar is hidden and cannot not be accessible through url as well
- Boot up `observability` or `security` serverless solutions and make
sure that `cases` is available in the navbar and works fine
- Boot up classic kibana and make sure that the left navbar has the same
menu entries it always had.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
20c1974e-44f0-45b0-80aa-e644fec148ff
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Reverts elastic/kibana#159674
The Elasticsearch team has changed their guidance about `fast_refresh`
and want this setting to be applied from within an Elasticsearch plugin
## Summary
Adds the necessary yml settings to the serverless config. These settings
are making ui changes in dashboards / visualizations editors and they
apply for all projects.
For testing run `yarn start --serverless=es` and check that the ui is
working as expected with the legacy editors (TSVB, agg based) in a read
only mode.
Starting in Node.js 17 the IP address from a DNS lookup resolves in the
order returned from the resolver. Prior versions resolve with IPv4
addresses sorted before IPv6 addresses.
This restores the previous behavior. In distributions, this can be
configured via the `--dns-result-order` flag in `node.options`.
Closes#163006
[Node.js 17
changelog](https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V17.md)
[Documentation](https://nodejs.org/docs/latest-v18.x/api/cli.html#--dns-result-orderorder)
Testing
1) Resolve localhost to 127.0.0.1 and then ::1 in `/etc/hosts`
2) `yarn start`. Logs should indicate connecting to elasticsearch over
127.0.0.1:9200
## Summary
Closes https://github.com/elastic/kibana/issues/160141
The avatar menu needs to be displayed for serverless. It was previously
required to be hidden in serverless, so a config 'showNavLinks' was
added. This config is no longer needed, so it has been removed.
## Testing
Start KB with the `--serverless` flag and login as `elastic`.
The Avatar should appear in the top right coner.
## Summary
This PR sets up the reporting plugin for the serverless implementation
by adding properties to the existing reporting config.
Image reporting is enabled for dev mode but disabled for serverless.
Canvas is disabled for serverless.
## To Test
Run `yarn es snapshot --license trial` in one terminal and then `yarn
start`. Load sample data and you should be able to see the option to
have PDF and PNG reports in Dashboard's Share Menu.
Now run `yarn es snapshot --license trial` and `yarn serverless-es`. You
should see that Dashboard's share menu does not include PDF or PNG
Reports. However there is still the option to see run CSV reports and
see the Reporting in Management.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
## Summary
First step for https://github.com/elastic/security-team/issues/6677
Aim of this PR is embed Discover in Security Solution. Discover must be
embedded as a complete app with certain set of capabilities working. The
set capabilities that need to working are listed here :
https://github.com/elastic/security-team/issues/6673
Release notes should be based on
https://github.com/elastic/security-team/issues/6673
### ⚠️ Note
- These changes are only available in serverless mode of security
solution behind a feature-flag called `discoverInTimeline`. Adds below
options to `serverless.security.yml`:
```yaml
# Serverless security specific options
xpack.securitySolution.enableExperimental:
- discoverInTimeline
```
You can use below command to run serverless instance of security
solution :
```bash
yarn serverless-security
```
This Implements following changes for each plugin.
### Discover
1. Exports Discover App as Lazy component.
2. Ability to override Discover Services.
3. Adds a parameter `mode` which switches off/on certain options based
on the `mode`. `Mode` has possible values of `embedded` and
`standalone`. For example, `embedded` switches off Discover breadcrumb
syncing, because consuming app may not need it.
### Unified Search
1. Ability to export a Search bar with custom depedency instances.
2. For example, today Unified Search uses a singleton global
`dataService` which store global KQL filters and queries. This
customization, let consumers of unified search to pass a new instance of
`dataService`.
4. Please see below diagram for more clarity.
### Navigation
1. Ability to export a custom stateful TopNav Menu which includes:
- DataView picker
- KQL Search Bar
- TimeRange Selector
2. Currently navigation consumes an instance of unified service which
uses a global singleton `data` service.
3. This PR creates a new instance of unified search which is then passed
to navigation to get a custom instance of `TopNav` Menu.
### Security Solution
1. Imports Discover Container Component
2. Uses customization point to pass a custom query bar.
3. Implements Custom KQL Query Bar with below customizations
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
This is a follow-up to https://github.com/elastic/kibana/pull/160671,
where the Management plugins were disabled using `contextRef`.
The configs for disabling the UI of the security management plugins were
added in https://github.com/elastic/kibana/pull/158186. In this PR, they
are changed so that they follow the same convention for disabling the
Management plugins - setting the default values of the configs to `true`
and explicitly setting them to `false` in the `serverless.yml` file.
This way, we have a clear view in `serverless.yml` of all
plugins/functionalities that have been disabled.
## Summary
This moves alerting in Serverless Search to the Stack Management
alerting page, and removes
To remove these rule types, I had to disable the observability,
securitySolution, uptime, monitoring and infra plugins and make sure
that their server plugins (not just the frontend plugins) respected the
`enabled: false` flag.
Closes https://github.com/elastic/kibana/issues/160160
## Summary
This PR adds functionality to the new autocomplete generation script for
creating an `availability` property in the spec files that is used for
filtering out endpoints that are not available in the current
environment (e.g. `serverless` or `stack`). It also adds a config
setting in the console plugin that specifies the current environment.
This setting is also configured accordingly for serverless.
**How to test**
1. Checkout the [ES specification
repo](https://github.com/elastic/elasticsearch-specification)
2. Run the command with `node scripts/generate_console_definitions.js
--source <ES_SPECIFICATION_REPO> --emptyDest` where
`<ES_SPECIFICATION_REPO>` is the absolute path to the root of the ES
specification repo
3. Start the classic Kibana and verify that Console suggests only
endpoints that are available in the `stack` environment.
4. Start Kibana in any of the serverless modes and verify that Console
suggests only endpoints that are available in the `serverless`
environment.
Here are some example endpoints that can be used for testing:
| Endpoint | Available in Stack | Available in Serverless |
| ------------- | ------------- | ------------- |
| [POST
_bulk](https://github.com/elastic/elasticsearch-specification/blob/main/specification/_global/bulk/BulkRequest.ts)
| Yes | Yes |
| [DELETE
_security/oauth2/token](https://github.com/elastic/elasticsearch-specification/blob/main/specification/security/invalidate_token/SecurityInvalidateTokenRequest.ts)
| Yes | No |
Closes: https://github.com/elastic/kibana/issues/155330
Adds `index.fast_refresh` to `.apm-custom-link` in order to ensure fast
index refreshes on serverless (1 second periodic refreshes instead of 5
second which is the new default on serverless).
This is pending on Elasticsearch adding support for `index.fast_refresh`
(https://github.com/elastic/elasticsearch/pull/96660)
---------
Co-authored-by: miriam.aparicio <miriam.aparicio@gmail.com>
Co-authored-by: Miriam <31922082+MiriamAparicio@users.noreply.github.com>
## Summary
closes: https://github.com/elastic/kibana/issues/159685
- Renaming _x-pack/plugins_:
`serverless_security` -> `security_solution_serverless`
`ess_security` -> `security_solution_ess`
- All the related configurations and types have also been renamed.
- i18n translation prefixes updated
- relocation of internal `security_solution_serverless` directories to
be consistent with `security_solution_ess`
### Eslint
I also added the plugins in the `.eslintrc` configuration, defining the
same rules as the `security_solution` plugin.
All eslint errors have been addressed (mainly _type_ imports errors)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/159392
## Summary
This PR hides the feature visibility section on the space management
screen and disables adding `disabledFeatures` when creating or updating
spaces using the REST API or spaces client on serverless.
## Screenshot
## Testing
1. Start Kibana in serverless mode: `yarn start --serverless`
2. Edit default space and observe that the feature visibility section is
not rendered
3. Quit Kibana and restart using classic mode: `yarn start`
4. Edit default space and observe that the feature visibility section is
rendered correctly
5. Other considerations:
- Disabling feature visibility in the classic offering should throw an
error (`xpack.spaces.allowFeatureVisibility: false`)
- Enabling feature visibility on serverless should throw an error
(`xpack.spaces.allowFeatureVisibility: true`)
Partially addresses https://github.com/elastic/kibana/issues/159590
## Summary
This PR disables the Advanced settings plugin for all projects in
serverless.
**How to test:**
1. Start Elasticsearch with `yarn es snapshot` and Kibana with `yarn
serverless-{mode}` where {mode} can be `es`, `security`, or `oblt`.
2. Verify that the Advanced settings app is not accessible and its
endpoint (`app/management/kibana/settings`) leads to the Stack
Management landing page.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
