Closes https://github.com/elastic/kibana/issues/45931
PR updates bsearch service to return request params from
elasticsearch-js client requests. This provides inspector with the exact
details used to fetch data from elasticsearch, ensuring inspector
displays request exactly as used by elasticsearch-js client.
**ESQL** This PR makes it possible to open ESQL searches in console.
<img width="500" alt="Screen Shot 2023-09-16 at 4 19 58 PM"
src="56019fb5-ca88-46cf-a42f-86f5f51edfcc">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR removes the `systemIndicesSuperuser` auth in the `esSupertest`
service for serverless test runs, adds `certificateAuthorities` to the
Elasticsearch server config in serverless and adds a tiny test to verify
functionality of this fix.
### Details
Issues before this PR when using the `esSupertest` service in serverless
tests (can be reproduced by running the added `elasticsearch_api` test
without the supertest and config changes):
1. Running against a local `functional_tests_server`, `esSupertest`
produces an error: `Error: self-signed certificate in certificate chain`
2. Running against an MKI project produces an error: `unable to
authenticate user [system_indices_superuser] for REST request [/]`,
because the `system_indices_superuser` doesn't exist in MKI.
How this PR addresses the issues:
1. Add `certificateAuthorities` to `servers.elasticsearch` in
`x-pack/test_serverless/shared/config.base.ts` in the same way we
already have it for `servers.kibana`
2. Modify the `esSUpertest` service to not override the default ES auth
when running in serverless, instead go with the default auth (regular
superuser), which is the best we can get.
### Additional information
It has been considered to add a serverless specific version of
`esSupertest` in order to avoid the `config.get('serverless') ?` code.
The fact that a number of stateful services are planned to be re-used in
serverless and rely on `esSupertest` in combination with the very small
change in a central service made it seem worth to make an exception
here.
Note that service methods or tests that use `esSupertest` can still run
into issues on serverless if they actually try to modify system indices.
This should be avoided anyways and particularly for serverless tests.
When in data view management an exact match index pattern is entered (without wildcard), there's now a status code of 404 instead of 500 returned, and no more error message logged.
## Summary
I'm waiting for this to fail CI - this means that a new ES snapshot is
being used. Then we update the test snapshot and should be good to
merge.
The lastest ES snapshot is returning field types in a different order
which really doesn't matter to this endpoint so I'll just update the
snapshot.
Closes https://github.com/elastic/kibana/issues/164753
## Summary
Resolves https://github.com/elastic/kibana/issues/153497.
Updates the saved query service to properly handle & return errors from
the saved object client. Instead of displaying "internal server error"
and returning 500, specific error messages occur for corresponding saved
object client errors.
After:
### To do
- [x] API integration tests
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Some simple dev UX improvements to the swap_references data views api -
```
POST /api/data_views/swap_references/_preview
{
"fromId" : "abcd-efg",
"toId" : "xyz-123"
}
returns
{
result: [{ id: "123", type: "visualization" }],
}
```
```
POST /api/data_views/swap_references
{
"fromId" : "abcd-efg",
"toId" : "xyz-123",
"delete" : true // optional, removes data view which is no longer referenced
}
returns
{
result: [{ id: "123", type: "visualization" }],
deleteStatus: {
remainingRefs: 0,
deletePerformed: true
}
```
Additional params -
```
fromType: string - specify the saved object type. Default is `index-pattern` for data view
forId: string | string[] - limit the affected saved objects to one or more by id
forType: string - limit the affected saved objects by type
```
Improves upon https://github.com/elastic/kibana/pull/157665
Docs will be created in follow up PR
Currently, if you try loading archive with index mappings not having
replica set into stateless ES, it won't work properly: you will get 503
error on calling `GET <index_name>/_stats`:
```
{
"error": {
"root_cause": [
{
"type": "no_shard_available_action_exception",
"reason": null
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "indices-stats",
"node": null,
"reason": {
"type": "no_shard_available_action_exception",
"reason": null
}
}
]
},
"status": 503
}
```
In stateless replica is
[required](https://elastic.slack.com/archives/C037J0RKRAN/p1690218904855299)
in order to perform search requests (the "search shard").
This PR updates index mappings in es_archives with
`"auto_expand_replicas": "0-1"`, in order to be compatible with
stateless ES and so that we can re-use existing data sets rather than
creating new ones.
I checked with Core Team that we should fine to just adapt all mapping
files, but let me know if that doesn't work for you.
The same value is used to create the "real" SO
[indices](c79c09c3d0/packages/core/saved-objects/core-saved-objects-migration-server-internal/src/actions/constants.ts (L21))
in Kibana.
## Summary
Part of https://github.com/elastic/kibana/issues/161882
Updates all archives of the `encrypted_saved_objects_api_integration`
test suite:
* Removing SO index definitions.
* Updating documents defined in `data.json`.
## Summary
Uses the versioned router for the remaining routes in the data plugin:
KQL telemetry (opt-in stats), and scripting languages list.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Part of https://github.com/elastic/kibana/issues/161882
Updates the
`test/api_integration/fixtures/es_archiver/saved_objects/delete_unknown_types`
archive:
* deleting `mappings.json`
* updating documents in `data.json`
This way, _esArchiver_ will not delete SO indices and recreate them,
eliminating the odds of the related tests being flaky.
## Summary
Managing large number of saved objects can be cumbersome. This api
endpoint allows the management of references without clicking through a
lot of different UIs.
For example -
This swaps all data view id `abcd-efg` references to `xyz-123`
```
POST /api/data_views/swap_references
{
"from_id" : "abcd-efg",
"to_id" : "xyz-123",
"preview" : false, // optional, necessary to save changes
"delete" : true // optional, removes data view which is no longer referenced
}
returns
{
preview: false,
result: [{ id: "123", type: "visualization" }],
deleteSuccess: true
}
```
Additional params -
```
from_type: string - specify the saved object type. Default is `index-pattern` for data view
for_id: string | string[] - limit the affected saved objects to one or more by id
for_type: string - limit the affected saved objects by type
```
Closes https://github.com/elastic/kibana/issues/153806
## Summary
Under some circumstances passing `override` to `POST
/api/data_views/data_view` would fail. Its now fixed.
To test - Try using the override param from the Kibana dev console. I
found it reproduced the problem before the fix and shows its resolved
after the fix. The problem did not appear in the integration tests.
I suspect the problem had to do with how quickly the delete was
performed - if it completed before the create command then everything
was fine. If it didn't then the error would appear. Passing the
overwrite param to the saved object client eliminates the possibility of
the delete failing to complete.
Closes https://github.com/elastic/kibana/issues/161016
## Summary
- Move field preview to internal route
- Add versioning to route
- Endpoint is called with version
- Response schema validation
Closes https://github.com/elastic/kibana/issues/159158
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Attempt to fix https://github.com/elastic/kibana/issues/156581
The best explanation I have is that the two retrieved visualisations are
not necessarily retrieved in the expected order.
Sorting the results should ensure the expected order.
Will use the flaky test runner to check the fix.
## Summary
This updates connector tiles for Enterprise Search to bring them in line
with 8.9.
<img width="1032" alt="Screenshot 2023-06-26 at 16 26 40"
src="a222a2a3-17ad-4e72-8ce8-57e09d89d05e">
<img width="1035" alt="Screenshot 2023-06-26 at 16 25 04"
src="3ce1e78c-b4fe-404b-b314-e02f3d6f439e">
<img width="1086" alt="Screenshot 2023-06-26 at 16 24 40"
src="14dd6ca5-875f-4c2d-9408-a2fa15abbbec">
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Related to #159582
To better understand the PR, we need to have a look at `esArchiver.load`
function and how it handles `mappings.json`:
- It processes the file and if it finds saved object index (e.g.
`.kibana`), it will delete all SO indexes (.kibana,
.kibana_task_manager, .kibana_alerting_cases, etc.) created on Kibana
start. Then esArchiver will re-create all indexes according to defined
mappings.
In order to keep our tests relevant to the latest code changes, it is
good to avoid re-creating SO indexes and use "official" ones whenever it
is possible. This PR removes/cleans `mappings.json` files where it seems
reasonable. I also deleted few not used archives:
- es_archiver/saved_objects/ui_counters
- es_archiver/saved_objects/usage_counters
- security_solution_cypress/es_archives/empty_kibana
## Summary
Part of [54306](https://github.com/elastic/kibana/issues/154306)
Required #158468 to be merged first.
This PR integrates only the migration part of the new Metric new
formatter task.
This migration implementation defines takes the simplest route for the
migration route of metric visualization formatter:
* if any custom formatter is set in Lens, then apply a `compact` flag on
it to keep it consistent
* if `Default` formatter is set in Lens, then assume nothing and let the
regular formatter hierarchy do its own work
### Example
Original dashboard
After migration
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
Version alllll the data view routes.
Best viewed with whitespace hidden -
https://github.com/elastic/kibana/pull/158608/files?diff=unified&w=1
In this PR:
- All REST (public and internal) routes are versioned
- Internal routes are called with version specified
- Internal and public routes are now stored in directories labeled as
such
- All routes have a response schema
- All responses are typed with `response` types, separate from internal
api types. This is to help prevent unacknowledged changes to the api.
- Moves some functional tests from js => ts
For follow up PRs:
- Move to `internal` path for internal routes
- Proper typing and schema for `fields_for_wildcard` filter
Closes https://github.com/elastic/kibana/issues/157099
Closes https://github.com/elastic/kibana/issues/157100
---------
Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
## Summary
Fix https://github.com/elastic/kibana/issues/150079
Add support for the `*` wildcard for by-type export, allowing to more
easily export all the exportable SO types
```
POST /api/saved_objects/_export
{
types: '*',
}
```
## Release Note
The savedObjects export API now supports exporting all types using the
`*` wildcard. Please refer to the documentation
for more details and examples.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
- Closes https://github.com/elastic/kibana/issues/147885
- Closes https://github.com/elastic/kibana/issues/157109
## Summary
**Before:**
Unified Field List plugin has internal routes (wrappers for client code)
which exist only to run api functional tests against them:
- `/api/unified_field_list/existing_fields/{dataViewId}`
- `/api/unified_field_list/field_stats`
Client code does not call these routes directly. So there is no reason
in keeping and versioning them.
**After:**
- Internal routes are removed
- A new "Unified Field List Examples" page was created
http://localhost:5601/app/unifiedFieldListExamples
- API functional tests (which used the routes) were converted to
functional tests against this new example page
- Created a new `unifiedFieldList` page object which is used now in
functional tests (methods are extracted from existing `discover` page
object).
**For testing:**
Steps:
1. Run Kibana with examples: `yarn start --run-examples`
2. Install sample data
3. And navigate to Developer Examples > Unified Field List Examples
page.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* Add deferred migrations parameter.
* Update outdated documents query to take into account deferred migrations.
* Update outdated documents query to take into account the core migration version.
* Update read operations in the saved objects repository to perform deferred migrations.
## Summary
The interface for fields allow array of string see below but the
validation with `@kbn/config-schema` only allow string. This will allow
to pass fields as array.
<img width="355" alt="image"
src="04c099c6-6a84-49ef-af45-587efa8e508b">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Fixes https://github.com/elastic/kibana/issues/155611
This PR changes the API prefix to indicate that the endpoints are
intended for internal use only. The plugin has been setup incorrectly
initially with the prefix `/api` that is intended for a public API.
There should not be any changes to the UI or functionality of the
plugin.
### Release Note
Fixed the guided onboarding API prefix to indicate that it's intended
for internal use
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
