## Summary
Running FTR tests locally (not on CI) that make API requests to public
versioned Kibana endpoints is currently broken. This is because:
* we require version headers to be set for both internal and **public**
endpoints in **dev** by way of a runtime check (ensures our code is
locked to a version)
* the vast majority of FTR tests do not set these headers for talking to
public endpoints
* on CI, this is different as we run these tests against a distributable
build (i.e., non-dev)
This manifests locally as a 400 response. E.g. the current
api_integration tests for data views by running:
```
node scripts/functional_test_runner.js --config ./test/api_integration/config.js --grep 'index_pattern_crud'
```
---
There are a few ways to resolve this, this PR proposes that we:
* Keep FTR tests as they are (i.e., don't update all of them to set
headers when making requests to public versioned endpoints), this is a
useful way to exercise our intended end-user behaviour
* Make the version resolution behaviour not depend on `--dev`: rather
set the default resolution to `none` (new setting) when we run in
`--dev` so that it can be configured to behave as we want for tests.
In this way we keep the runtime check for its intended purpose and can
run our FTR tests "as end users" of our endpoints.
---
Close https://github.com/elastic/kibana/issues/161435
## Summary
Fix a bug that was causing the cluster to fall into an unrecoverable
state if an error occurs during the document migration phase of a `v2`
to `zdt` migration (first zdt migration to take over the `v2`
algorithm).
When an error occurs during the document migration phase, the index gets
in a state where `mappingVersions` is present but not `docVersions`, and
the algorithm wasn't able to understand what this state was, and
considered it to just be a plain `zdt` predecessors, which caused an
error when entering the document migration phase, as `docVersions`
couldn't be found.
This PR addresses this, by properly identifying this specific state, and
acting accordingly (by initiating a document migration without checking
for the versions in `docVersions`, as we do then coming from a plain v2
algo)
## Summary
Related to https://github.com/elastic/kibana/issues/150296
- Add the following SO model version change types:
- `data_removal`
- `unsafe_transform`
- Change the existing `data_backfill` type to something less permissive
- Update the tsdoc and documentation accordingly
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/160052
This PR adds support for the header banner in serverless projects.
<img width="1437" alt="image"
src="c950bbe7-46bb-4b82-b9ee-406aef7748dc">
**NOTE:** This PR adds support for the header banner by applying the
same techniques as in the "classic" layout: special class names of
`header__bar`/`header_firstBar`/`header__secondBar`. The effect is the
"header action menu" bar, which is an optional 2nd bar in the chrome
header, works better after this change when it is always visible. This
is a temporary issue that unblocks the header banner, and fixes other
overlapping issues with the current project layout stylings.
_As followup work_, the @elastic/appex-sharedux team will investigate
converting the Kibana page template to the new
[EuiPageTemplate](https://elastic.github.io/eui/#/templates/page-template/examples)
which supports horizontal sections of a page that have dynamic height --
we need this for the dynamic presence of the header banner, and the
intended dynamic presence of the header action menu.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
## Summary
Close https://github.com/elastic/kibana/issues/160207.
Adds a new, additional mechanism for negotiating version (via query
parameter). This is specifically intended to enable Kibana versioned
endpoints to be usable in browser environments where setting a header is
not always possible (like in an `img` tag's `href`).
Note: the header-based version still takes precedence if present.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Fix https://github.com/elastic/kibana/issues/156803
Use `apm.captureError` to properly capture errors thrown by the route
handler (as those errors are then converted to generic 500 errors before
being returned by the server to avoid leaking internal info)
## Summary
Tackles the first improvement described in
https://github.com/elastic/kibana/issues/160038.
When "picking up" the updated mappings, we add a "query" in order to
select and update only the SO types that have been updated, compared to
the previous version.
We achieve this by comparing `migrationMappingPropertyHashes`; we
compare the hashes stored in the
`<soIndex>.mapping._meta.migrationMappingPropertyHashes` against the
ones calculated from the definitions from the `typeRegistry`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- close https://github.com/elastic/kibana/issues/156517
- built on top of https://github.com/elastic/kibana/pull/156855
- [Serverless chrome breadcrumbs
requirements](https://docs.google.com/document/d/1e5SbDPpySiPeBrjgLJD6Qw6fJyiy34uO2dmGLHlu38E/edit)
- [Serverless chrome breadcrumbs API tech
do](https://docs.google.com/document/d/1_qL14NMGYdI0913eclJd3DXG0lQ2jkE0V3578iaDASs/edit#heading=h.ndqge1i76y6p)
This PR implements serverless (project) breadcrumbs. Now Chrome
automatically renders the main part of the breadcrumbs based on the
project navigation tree and current active path. Plugins can append
their deeper context breadcrumbs or override the navigation breadcrumbs.
```
plugins.serverless.setBreadcrumbs(myDeeperContextBreadcrumbs);
plugins.serverless.setBreadcrumbs(myCustomBreadcrumbs, {absolute: true});
```
Oblt:
Search:
Security:
Security hasn't changed because they don't set the project navigation
tree yet. They still have regular breadcrumbs.
-----
Notes: We thought of a possible edge case where an app would set deeper
breadcrumbs `plugins.serverless.set breadcrumbs({text: 'foo', href:
'/foo'});` but the project navigation already have this link as part of
the path. If we hit this edge in the real world, we can workaround this
by merging the same consequent breadcrumbs by a deep link id.
---------
## Summary
Why?
To simplify the process of migration to react-router@6.
https://github.com/remix-run/react-router/discussions/8753
What problems exactly it solves?
- In my previous PR I added `CompatRouter`
https://github.com/elastic/kibana/pull/159173, which caused changes in
~50 files and pinged 15 Teams. And this is just meant to be a temporary
change, so when we're done with the migration I would have to revert
these changes and engage everyone to review the PR again. And it is just
a single step in the migration strategy. So to make our lives easier I
think it would be better to have a common place where we do import our
router components because it will allow us to surface some extra logic
in single place instead of going through the whole source code again.
- `react-router@6` doesn't support a custom `Route` component, so that
means our custom `Route` component that we're using almost everywhere
today, will need to be replaced by a different solution. I have decided
to add `Routes` component, which will be responsible for rendering the
proper component (`react-router@6` renamed `Switch` to `Routes`, so I
have named this component to align with the dictionary of the new
router) and also is going to add the logic that today is done in `Route`
(moving logic to `Routes` will be done in the follow-up PR, here I just
wanted to focus on using the common router components to make the review
process easier)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/149249
Add a new EBT context providing the page_title field to events.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/149249
Add a new EBT context providing the `page_url` field to events.
`page_url` is based on the current url's `pathname` and `hash`
exclusively (no domain, port, query param...)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/158910
Changes the behavior of the `/api/status` endpoint to always returns a
consistent http status code, and in particular:
- during the preboot stage
- when accessed by unauthenticated users and `status.allowAnonymous` is
`false`.
That way, `/api/status` can properly be used for readiness checks.
Please refer to https://github.com/elastic/kibana/issues/158910 for more
details.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/159646
Fix the config creation-from-previous-one logic by always using the
latest config for the new version's creation
## Release Note
Fix a bug that could cause old Kibana deployments to loose their
uiSettings after an upgrade
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds logic (and tests) to ensure that all registered public routes are
set to `2023-10-31` for now. This check is only performed in dev mode
which allows us to test our existing route default logic.
### Notes
This works best as a runtime check given the versioned router API, but
perhaps I missed a way to do this with just type checking?
## Summary
Adds a helper/util to the Versioned router mock so that tests can more
easily be expressed against registered versioned routes.
### Usage
See included test. Thanks @paul-tavares for providing some prior art, I
adapted slightly to rather return all the versions. Let me know what you
think!
CC @pgayvallet @paul-tavares
## Summary
Across the kibana code base there are a number of http client wrappers,
some of which have special handling of the version header and some
don't. This can be quite confusing when you get an error message that
says "Please specify a version." and you _are_ specifying a version
header but you're not specifying _the_ version header. Now the error
message will remind you where it expects to find the version.
Closes https://github.com/elastic/kibana/issues/157810
Closes https://github.com/elastic/kibana/issues/158879
## Summary
1. Moves the Logo icon out of the side nav and to the left of the
breadcrumbs
1. Moves the project switcher from the right of the header to the just
to the right of the logo
1. Removes the link to cloud from the side nav. Design is still TBD.
1. Adds new `serverless.setProjectHome` API since the home link is no
longer in the side nav
1. Removes the `linkToCloud` prop from the Navigation component since
design is still TBD.
1. Exposes the Global Search bar
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Screenshots
**Default**
<img width="1732" alt="Screenshot 2023-06-06 at 11 46 52 AM"
src="45cb55f6-aea6-4b40-86c2-3b386691fbf5">
**Global search (not yet collapsed by default)**
<img width="1732" alt="Screenshot 2023-06-06 at 11 46 37 AM"
src="9ca6c28d-755f-45be-b680-89b5b4b1fb36">
**Collapsed side nav**
<img width="1732" alt="image"
src="f334ab99-0fda-455c-a9e5-6b5f64aa954d">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR introduces two changes:
(1) Refactors the handler resolution logic to _not_ depend on the
`--serverless` cli arg by adding a new piece of config
`server.versioned.routeResolution` that accepts `newest | oldest`. This
piece of config is passed down instead of the `serverless` cli arg as
well as updating test cases
(2) Adds a new piece of config to turn off the client version checking.
This will be needed for rolling upgrades to allow old browser traffic to
reach new Kibana servers when there is stack version change.
Close https://github.com/elastic/kibana/issues/158723
## Open questions
* Do we want to make the version check still take _major_ version bumps
into account?
## Summary
Version alllll the data view routes.
Best viewed with whitespace hidden -
https://github.com/elastic/kibana/pull/158608/files?diff=unified&w=1
In this PR:
- All REST (public and internal) routes are versioned
- Internal routes are called with version specified
- Internal and public routes are now stored in directories labeled as
such
- All routes have a response schema
- All responses are typed with `response` types, separate from internal
api types. This is to help prevent unacknowledged changes to the api.
- Moves some functional tests from js => ts
For follow up PRs:
- Move to `internal` path for internal routes
- Proper typing and schema for `fields_for_wildcard` filter
Closes https://github.com/elastic/kibana/issues/157099
Closes https://github.com/elastic/kibana/issues/157100
---------
Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
## Summary
Fix https://github.com/elastic/kibana/issues/150079
Add support for the `*` wildcard for by-type export, allowing to more
easily export all the exportable SO types
```
POST /api/saved_objects/_export
{
types: '*',
}
```
## Release Note
The savedObjects export API now supports exporting all types using the
`*` wildcard. Please refer to the documentation
for more details and examples.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Closes#155137, with some extra reorganisation, modularisation and unit
tests.
### Refactors to `maybeAddConfig`
### Refactoring serve.js <-> bootstrap.ts
### Unit tests for `compileConfigStack`
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Closes#154252
## Summary
This PR implements a default [permissions policy
header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy)
for Kibana. It takes into consideration only the directives that are
_not_ still in development/pre-release. These directives are:
- camera
- display-capture
- fullscreen
- geolocation
- microphone
- web-share
All directives above are set to an empty or "none" allow list, with the
exception of `fullscreen`. The default permissions policy for Kibana
will be: `camera=(), display-capture=(), fullscreen=(self),
geolocation=(), microphone=(), web-share=()`.
The `display-capture` directive only affects access to the [Screen
capture
API](https://developer.mozilla.org/en-US/docs/Web/API/Screen_Capture_API),
whereas our reporting utilities rely on a headless Chrome feature.
FullyStory also does not appear to rely on capturing the screen, but
rather on [capturing the DOM and
CSS](3600206230-Does-FullStory-work-with-my-complex-site-)
to reproduce the screen and user stories. We have confirmed with
FullStory support that they do not use the Screen capture API
(`getDisplayMedia()`).
The `fullscreen` directive affects access to the
[element.requestFullscreen
API](https://developer.mozilla.org/en-US/docs/Web/API/Element/requestFullscreen).
Our fullscreen view for Dashboards and visualizations appear to be
scoped to the Kibana application and not the browser or system, but
there is one use of `requestFullscreen` in the synthetics plugin's
`grid_group_item`, hence the default setting of 'self' here.
## Testing
Due to screen capture reports (PNG & PDF) and fullscreen modes working
with the configuration defined above, I wanted to double-check the
header was working, so I tested both camera and microphone access. I did
this by adding some code into our team's user management screen that
would attempt to access both the camera and microphone. I could confirm
that with the permissions policy header (as defined), no access prompts
would appear for either resource. When I removed the directives for the
camera and microphone from the permissions policy header, prompts
appeared in the browser (Chrome) to approve access to the camera and
microphone.
### Manual Test:
1. Start Elasticsearch and Kibana from this PR
2. Sign into Kibana and open your browser's dev tools
3. In the network tab, inspect one of the requests for localhost
4. In the Headers tab, verify the permissions-policy header is present
with the new default value defined above, `camera=(),
display-capture=(), fullscreen=(self), geolocation=(), microphone=(),
web-share=()`
5. Edit the kibana.yml file, add
`server.securityResponseHeaders.permissionsPolicy: camera=(self)`
6. Refresh Kibana in the browser, verify the permissions-policy header
is present with the value set in kibana.yml in step 5
7. Edit the kibana.yml file, add
`server.securityResponseHeaders.permissionsPolicy: null`
8. Refresh Kibana in the browser, verify the permissions-policy header
is no longer present
## Release Notes:
Security
Adds default Permissions-Policy header including all non-experimental
directives [#158514](https://github.com/elastic/kibana/pull/158514)
---------
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
## Summary
Makes version headers required for internal endpoints. We also require
version headers for public endpoints when in dev mode.
### Note to reviewers
This PR is a re-revert of the original
https://github.com/elastic/kibana/pull/158667 with some minor additions
(see comments).
The original was reverted due to failing Cypress tests blocking Kibana
promotion for 8.8.1 (CC @stephmilovic,
https://github.com/elastic/kibana/pull/158961)
Not sending headers to versioned, internal endpoints will return 400!
Due to the somewhat sensitive nature of this change, I went through all
of the existing `.versioned` endpoints and tried to ensure that for
_internal_ endpoints we send through a version as this is now
**required**.
I would greatly appreciate it if code owners could check their code,
think of any existing consumers of your versioned endpoints and ensure
they are sending a version.
Closes https://github.com/elastic/kibana/issues/158722
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
## Summary
Fixes: #155154 (introduced in #149878), builds on #155436 .
- Adds tests to ensure the configuration merging order, check those for
reference.
- Updates the README to explain the intention
For the tests, I needed to output something to the logs. I hope it's not
a big issue to log it. If needed, I might hide that behind a verbose- or
feature flag.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
