## Summary
This PR is part of the Kibana Sustainable Architecture effort.
The goal is to start categorising Kibana packages into _generic
platform_ (`group: "platform"`) vs _solution-specific_.
```
group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'
```
Uncategorised modules are considered to be `group: 'common', visibility:
'shared'` by default.
We want to prevent code from solution A to depend on code from solution
B.
Thus, the rules are pretty simple:
* Modules can only depend on:
* Modules in the same group
* OR modules with 'shared' visibility
* Modules in `'observability', 'security', 'search'` groups are
mandatorily `visibility: "private"`.
Long term, the goal is to re-organise packages into dedicated folders,
e.g.:
```
x-pack/platform/plugins/private
x-pack/observability/packages
```
For this first wave, we have categorised packages that seem
"straightforward":
* Any packages that have:
* at least one dependant module
* all dependants belong to the same group
* Categorise all Core packages:
* `@kbn/core-...-internal` => _platform/private_
* everything else => _platform/shared_
* Categorise as _platform/shared_ those packages that:
* Have at least one dependant in the _platform_ group.
* Don't have any `devOnly: true` dependants.
### What we ask from you, as CODEOWNERS of the _package manifests_, is
that you confirm that the categorisation is correct:
* `group: "platform", visibility: "private"` if it's a package that
should only be used from platform code, not from any solution code. It
will be loaded systematically in all serverless flavors, but solution
plugins and packages won't be able to `import` from it.
* `group: "platform", visibility: "shared"` if it's a package that can
be consumed by both platform and solutions code. It will be loaded
systematically in all serverless flavors, and anybody can import / use
code from it.
* `group: "observability" | "security" | "search", visibility:
"private"` if it's a package that is intented to be used exclusively
from a given solution. It won't be accessible nor loaded from other
solutions nor platform code.
Please refer to
[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)
for any related questions.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Summary
Adds a new API deprecations feature inside core.
This feature enabled plugin developers to mark their versioned and
unversioned public routes as deprecated.
These deprecations will be surfaced to the users through UA to help them
understand the deprecation and address it before upgrading. This PR also
surfaces these deprecations to UA.
Closes https://github.com/elastic/kibana/issues/117241
1. Core service to flag deprecated routes
2. UA code to surface and resolve deprecated routes
## Flagging a deprecated Route
### The route deprecation option
We have three types of route deprecations:
- `type: bump`: A version bump deprecation means the API has a new
version and the current version will be removed in the future in favor
of the newer version.
- `type: remove`: This API will be completely removed. You will no
longer be able to use it in the future.
- `type: migrate`: This API will be migrated to a different API and will
be removed in the future in favor of the other API.
All route deprecations expect a documentation link to help users
navigate. We might add a generic documentation link and drop this
requirement in the future but for now this is required.
### Deprecated Route Example
Full examples can be found in the `routing_example` example plugin
located in this directory:
`examples/routing_example/server/routes/deprecated_routes`
```ts
router[versioned?].get(
{
path: '/',
options: {
deprecated: {
documentationUrl: 'https://google.com',
severity: 'warning',
reason: {
type: 'bump',
newApiVersion: '2024-10-13',
},
},
},
},
async (context, req, res) => {
...
```
## Surfaced API deprecations in UA
The list of deprecated APIs will be listed inside Kibana deprecations
along with the already supported config deprecations.
<img width="1728" alt="image"
src="https://github.com/user-attachments/assets/5bece704-b80b-4397-8ba2-6235f8995e4a">
Users can click on the list item to learn more about each deprecation
and mark it as resolved
<img width="1476" alt="image"
src="https://github.com/user-attachments/assets/91c9207b-b246-482d-a5e4-21d0c61582a8">
### Marking as resolved
Users can click on mark as resolved button in the UA to hide the
deprecation from the Kiban deprecations list.
We keep track on when this button was clicked and how many times the API
has been called. If the API is called again the deprecation will
re-appear inside the list. We might add a feature in the future to
permenantly supress the API deprecation from showing in the list through
a configuration (https://github.com/elastic/kibana/issues/196089)
If the API has been marked as resolved before we show this in the flyout
message:
> The API GET /api/deprecations/ has been called 25 times. The last time
the API was called was on Monday, October 14, 2024 1:08 PM +03:00.
> The api has been called 2 times since the last time it was marked as
resolved on Monday, October 14, 2024 1:08 PM +03:00
Once marked as resolved the flyout exists and we show this to the user
until they refresh the page
<img width="1453" alt="image"
src="https://github.com/user-attachments/assets/8bb5bc8b-d1a3-478f-9489-23cfa7db6350">
## Telemetry:
We keep track of 2 new things for telemetry purposes:
1. The number of times the deprecated API has been called
2. The number of times the deprecated API has been resolved (how many
times the mark as resolved button in UA was clicked)
## Code review
- [x] Core team is expected to review the whole PR
- [ ] Docs team to review the copy and update the UA displayed texts
(title, description, and manual steps)
- [x] kibana-management team is expected to review the UA code changes
and UI
- [ ] A few teams are only required to approve this PR and update their
`deprecated: true` route param to the new deprecationInfo object we now
expect. There is an issue tracker to address those in separate PRs later
on: https://github.com/elastic/kibana/issues/196095
## Testing
Run kibana locally with the test example plugin that has deprecated
routes
```
yarn start --plugin-path=examples/routing_example --plugin-path=examples/developer_examples
```
The following comprehensive deprecated routes examples are registered
inside the folder:
`examples/routing_example/server/routes/deprecated_routes`
Run them in the console to trigger the deprecation condition so they
show up in the UA:
```
# Versioned routes: Version 1 is deprecated
GET kbn:/api/routing_example/d/versioned?apiVersion=1
GET kbn:/api/routing_example/d/versioned?apiVersion=2
# Non-versioned routes
GET kbn:/api/routing_example/d/removed_route
POST kbn:/api/routing_example/d/migrated_route
{}
```
1. You can also mark as deprecated in the UA to remove the deprecation
from the list.
2. Check the telemetry response to see the reported data about the
deprecated route.
3. Calling version 2 of the API does not do anything since it is not
deprecated unlike version `1` (`GET
kbn:/api/routing_example/d/versioned?apiVersion=2`)
4. Internally you can see the deprecations counters from the dev console
by running the following:
```
GET .kibana_usage_counters/_search
{
"query": {
"bool": {
"should": [
{"match": { "usage-counter.counterType": "deprecated_api_call:total"}},
{"match": { "usage-counter.counterType": "deprecated_api_call:resolved"}},
{"match": { "usage-counter.counterType": "deprecated_api_call:marked_as_resolved"}}
]
}
}
}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
fix https://github.com/elastic/kibana/issues/163654
This PR enforces internal API restrictions in our standard offering.
Internal APIs are subject to rapid change and are intentionally not
public. By restricting their access, we protect consumers from these
rapid changes.
This PR does not change any public APIs and they remain available for
external consumption.
## Note to reviewers:
I chose the most practical way of resolving the failures (add the header
or disable the restriction).
## Details
Requests to internal Kibana APIs will be restricted globally. This
allows more flexibility in making breaking changes to internal APIs,
without a risk to external consumers.
## Why are we doing this?
The restriction is there to help mitigate the risk of breaking external
integrations consuming APIs. Internal APIs are subject to frequent
changes, necessary for feature development.
## What this means to plugin authors :
Kibana core applies the restriction when enabled through HTTP config.
## What this means to Kibana consumers:
Explicitly restricting access to internal APIs has advantages for
external consumers:
- Consumers can safely integrate with Kibana's stable, public APIs
- Consumers are protected from Internal route development, which may
involve breaking changes
- Relevant information in Kibana's external documentation that is
user-friendly and complete.
KB article explaining the change (tracked as part of
https://github.com/elastic/kibana-team/issues/1044)
## Release note
Starting with this release, requests to internal Kibana APIs are
globally restricted by default. This change is designed to provide more
flexibility in making breaking changes to internal APIs while protecting
external consumers from unexpected disruptions.
**Key Changes**:
• _Internal API Access_: External consumers no longer have access to
Kibana’s internal APIs, which are now strictly reserved for internal
development and subject to frequent changes. This helps ensure that
external integrations only interact with stable, public APIs.
• _Error Handling_: When a request is made to an internal API without
the proper internal identifier (header or query parameter), Kibana will
respond with a 400 Bad Request error, indicating that the route exists
but is not allowed under the current Kibana configuration.
## How to test this
### Running kibana
1. Set `server.restrictInternalApis: true` in `kibana.yml`
2. Start es with any license
3. Start kibana
4. Make an external request to an internal API:
<details>
<summary>curl request to get the config for 9.0:</summary>
```unset
curl --location
'localhost:5601/abc/api/kibana/management/saved_objects/_bulk_get' \
--header 'Content-Type: application/json' \
--header 'kbn-xsrf: kibana' \
--header 'x-elastic-internal-origin: kibana' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--data '[
{
"type": "config",
"id": "9.0.0"
}
]'
```
</details>
The request should be successful.
5. Make the same curl request without the internal origin header
<details>
<summary>curl:</summary>
```unset
curl --location
'localhost:5601/abc/api/kibana/management/saved_objects/_bulk_get' \
--header 'Content-Type: application/json' \
--header 'kbn-xsrf: kibana' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--data '[
{
"type": "config",
"id": "9.0.0"
}
]'
```
</details>
The response should be an error similar to:
`{"statusCode":400,"error":"Bad Request","message":"uri
[/api/kibana/management/saved_objects/_bulk_get] with method [post]
exists but is not available with the current configuration"}`
6. Remove `server.restrictInternalApis` from `kibana.yml` or set it to
`false`.
7. Repeat both curl requests above. Both should respond without an
error.
### Checklist
Delete any items that are not applicable to this PR.
- [X] [Documentation] was added for features that require explanation or
tutorials (In PR https://github.com/elastic/kibana/pull/191943)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios (and PR
https://github.com/elastic/kibana/pull/192407)
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
(docker list updated in https://github.com/elastic/kibana/pull/156935,
cloud stack packs for 9.0 kibana to follow)
### Risk Matrix
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| The restriction is knowingly bypassed by end-users adding the required
header to use `internal` APIs | Low | High | Kibana's internal APIs are
not documented in the OAS specifications. External consumption will be
prevented unless explicitly bypassed. |
| Upstream services don't include the header and requests to `internal`
APIs fail | Medium | Medium | The Core team needs to ensure intra-stack
components are updated too |
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Extended `KibanaRouteOptions` to include security configuration at the
route definition level.
## Security Config
To facilitate iterative development security config is marked as
optional for now.
- `authz` supports both simple configuration (e.g., single privilege
requirements) and more complex configurations that involve anyRequired
and allRequired privilege sets.
- `authc` property has been added and is expected to replace the
existing `authRequired` option. This transition will be part of an
upcoming deprecation process in scope of
https://github.com/elastic/kibana/issues/191711
- For versioned routes, the `authc` and `authz` configurations can be
applied independently for each version, enabling version-specific
security configuration. If none provided for the specific version it
will fall back to the route root security option.
- Validation logic has been added that ensures only supported
configurations are specified.
- Existing `registerOnPostAuth` hook has been modified to incorporate
checks based on the new `authz` property in the security configuration.
- Comprehensive documentation will be added in the separate PR before
sunsetting new security configuration and deprecating old one.
## How to Test
You can modify any existing route or use the example routes below
### Routes
<details>
<summary><b>Route 1:
/api/security/authz_examples/authz_disabled</b></summary>
```javascript
router.get(
{
path: '/api/security/authz_examples/authz_disabled',
security: {
authz: {
enabled: false,
reason: 'This route is opted out from authorization',
},
},
validate: false,
},
createLicensedRouteHandler(async (context, request, response) => {
try {
return response.ok({
body: {
message: 'This route is opted out from authorization',
},
});
} catch (error) {
return response.customError(wrapIntoCustomErrorResponse(error));
}
})
);
```
</details>
<details>
<summary><b>Route 2:
/api/security/authz_examples/simple_privileges_1</b></summary>
```javascript
router.get(
{
path: '/api/security/authz_examples/simple_privileges_1',
security: {
authz: {
requiredPrivileges: ['manageSpaces', 'taskManager'],
},
},
validate: false,
},
createLicensedRouteHandler(async (context, request, response) => {
try {
return response.ok({
body: {
authzResult: request.authzResult,
},
});
} catch (error) {
return response.customError(wrapIntoCustomErrorResponse(error));
}
})
);
```
</details>
<details>
<summary><b>Route 3:
/api/security/authz_examples/simple_privileges_2</b></summary>
```javascript
router.get(
{
path: '/api/security/authz_examples/simple_privileges_2',
security: {
authz: {
requiredPrivileges: [
'manageSpaces',
{
anyRequired: ['taskManager', 'features'],
},
],
},
},
validate: false,
},
createLicensedRouteHandler(async (context, request, response) => {
try {
return response.ok({
body: {
authzResult: request.authzResult,
},
});
} catch (error) {
return response.customError(wrapIntoCustomErrorResponse(error));
}
})
);
```
</details>
<details>
<summary><b>Versioned Route:
/internal/security/authz_versioned_examples/simple_privileges_1</b></summary>
```javascript
router.versioned
.get({
path: '/internal/security/authz_versioned_examples/simple_privileges_1',
access: 'internal',
enableQueryVersion: true,
})
.addVersion(
{
version: '1',
validate: false,
security: {
authz: {
requiredPrivileges: ['manageSpaces', 'taskManager'],
},
authc: {
enabled: 'optional',
},
},
},
(context, request, response) => {
return response.ok({
body: {
authzResult: request.authzResult,
version: '1',
},
});
}
)
.addVersion(
{
version: '2',
validate: false,
security: {
authz: {
requiredPrivileges: ['manageSpaces'],
},
authc: {
enabled: true,
},
},
},
(context, request, response) => {
return response.ok({
body: {
authzResult: request.authzResult,
version: '2',
},
});
}
);
```
</details>
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Closes: https://github.com/elastic/kibana/issues/191710__
__Related: https://github.com/elastic/kibana/issues/191712,
https://github.com/elastic/kibana/issues/191713__
### Release Note
Extended `KibanaRouteOptions` to include security configuration at the
route definition level.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/192004
Calling `client.asSecondaryAuthUser` from a client scoped to a fake
request instantiated with `getKibanaFakeRequest` returns the following
error:
`Error: asSecondaryAuthUser called from a client scoped to a request
without 'authorization' header.`.
This is because we use the same branch when dealing with a real or fake
request and expect the headers to be cached. There are existing tests to
verify a fake request works but these requests are raw objects not
created through `getKibanaFakeRequest`
### Testing
This snippet does not throw
```
const fakeRequest = getFakeKibanaRequest({ id: apiKey.id, api_key: apiKey.apiKey });
const esClient = server.core.elasticsearch.client.asScoped(fakeRequest).asSecondaryAuthUser;
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
At the moment, our package generator creates all packages with the type
`shared-common`. This means that we cannot enforce boundaries between
server-side-only code and the browser, and vice-versa.
- [x] I started fixing `packages/core/*`
- [x] It took me to fixing `src/core/` type to be identified by the
`plugin` pattern (`public` and `server` directories) vs. a package
(either common, or single-scoped)
- [x] Unsurprisingly, this extended to packages importing core packages
hitting the boundaries eslint rules. And other packages importing the
latter.
- [x] Also a bunch of `common` logic that shouldn't be so _common_ 🙃
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/72880
- Generate translation files for all locales (including all internal
plugins) during the CDN asset generation task
- Adapt the `rendering` service to use the translation files from the
CDN if configured/enabled
### How to test
Connect to the serverless project that was created for the PR, and
confirm the translation file is being loaded from the CDN
<img width="907" alt="Screenshot 2024-04-25 at 15 55 23"
src="5a6d9110-2e92-41e5-b066-e792e0015134">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds a new core-owned experimental HTTP API: `GET /api/oas` with the
following query params for filtering:
- `pluginId` - request a specific plugin's OAS
- `pathStartsWith` - request OAS for paths that start with the given
string
The current rationale is that this will provide sufficient scoping to
allow developers and docs maintainers to request only the OAS they need
(since Kibana's surface area is HUGE).
Also added a new experimental config: `server.oas.enabled: <boolean>`.
Set this to `true` to play around with this feature locally.
## How to test
1. Add `server.oas.enabled: true` to `kibana.dev.yml`
2. Start Kibana with `yarn start --no-base-path`
3. `curl -uelastic:changeme
http://localhost:5601/api/oas\?pathStartsWith\=/api/status | jq`
4. You should have a JSON representation of an OAS doc containing the
`/api/status` endpoint!
<details>
<summary>example</summary>
```json
{
"openapi": "3.0.0",
"info": {
"title": "Kibana HTTP APIs",
"version": "0.0.0"
},
"servers": [
{
"url": "http://localhost:5601"
}
],
"paths": {
"/api/status": {
"get": {
"responses": {},
"parameters": [
{
"in": "header",
"name": "elastic-api-version",
"schema": {
"type": "string",
"enum": [
"2023-10-31"
],
"default": "2023-10-31"
}
},
{
"name": "v7format",
"in": "query",
"required": false,
"schema": {
"type": "boolean"
}
},
{
"name": "v8format",
"in": "query",
"required": false,
"schema": {
"type": "boolean"
}
}
],
"operationId": "/api/status#0"
}
}
},
"components": {
"schemas": {},
"securitySchemes": {
"basicAuth": {
"type": "http",
"scheme": "basic"
},
"apiKeyAuth": {
"type": "apiKey",
"in": "header",
"name": "Authorization"
}
}
},
"security": [
{
"basicAuth": []
}
]
}
```
</details>
## Risks
* The work to generate OAS can be expensive (for both CPU and memory).
To mitigate, the endpoint will only be registered provided a specific
config and we are using concurrency control (`Semaphore`) to allow only
one of these operations to be handled at a time.
## Future work
* We must start the long tail work of furnishing endpoints with missing
response schemas. I intend to do so for the `/api/status` route next
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Follow up of [first CDN
PR](https://github.com/elastic/kibana/pull/169408). Primary focus is
replacing our build nr with SHA that allows cache busting and maintains
anti-collision properties.
## How to test
Start Kibana as usual navigating around the app with the network tab
open in your browser of choice. Keep an eye out for any asset loading
errors. It's tricky to test every possible asset since there are many
permutations, but generally navigating around Kibana should work exactly
as it did before regarding loading bundles and assets.
## Notes
* did a high-level audit of usages of `buildNum` in `packages`, `src`
and `x-pack` adding comments where appropriate.
* In non-distributable builds (like dev) static asset paths will be
prefixed with `XXXXXXXXXXXX` instead of Node's `Number.MAX_SAFE_INTEGER`
* Added some validation to ensure the CDN url is of the expected form:
nothing trailing the pathname
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Risk Matrix
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| We break some first or third party dependencies on existing asset
routes | Med | High | Attempting to mitgate by serving static assets
from both old and new paths where paths have updated to include the
build SHA. Additioanlly: it is very bad practice to rely on the values
of the static paths, but someone might be |
| Cache-busting is more aggressive | High | Low | Unlikely to be a big
problem, but we are not scoping more static assets to a SHA and so every
new Kibana release will require clients to, for example, download fonts
again. |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/170421
### 1. Introduce the `http.staticAssets` service
Which can be used to generate hrefs to Kibana's static assets in a
CDN-friendly way (based on the CDN url if defined in the config, and the
Kibana's basePath otherwise)
The service is exposed both on the browser and server-side.
For now a single API is exposed: `getPluginAssetHref`
```ts
// returns "/plugins/{pluginId}/assets/some_folder/asset.png" when CDN isn't configured
core.http.statisAssets.getPluginAssetHref('some_folder/asset.png');
```
### 2. Plug it on some of the `home` plugin assets
Adapt the sample data sets and tutorial schemas to use the service for
links to the associated assets
## How to test
#### 1. Edit`/etc/hosts`
add a line `127.0.0.1 local.cdn`
#### 2. Edit `kibana.yaml`
Add `server.cdn.url: "http://local.cdn:5601"`
#### 3. Boot kibana and navigate to sample data set installation
(if started in dev mode, use `--no-base-path`)
Confirm that the sample data set presentation images are pointing to the
CDN url and properly displayed:
<img width="1565" alt="Screenshot 2023-11-13 at 09 28 51"
src="23a887af-00cb-400c-9ab1-511ba463495f">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR adds new behaviour to log a message when we are not enforcing
strict client-server build number checks and see a different build
number.
## How to test
* Start Kibana
* Set `server.versioned.strictClientVersionCheck: false` in
`kibana.dev.yml`
* Send a request that will return a bad response, in my case I sent
`POST /api/files/files/defaultImage` with an empty body `{}`
## Assumptions
* We only want to log when we are returning a bad response (>=400). It's
possible that we miss events this way, but the assumption is that status
codes >=400 are a strong indication that something is wrong, i.e. we
don't care about mismatches when good responses happen
* We want to treat this as a `WARN` level log
* We will be able to track this in o11y using the `logger` context (like
`kbn-build-number-mismatch`)
## Example
```
[2023-11-02T13:52:48.134+01:00][WARN ][http.kbn-build-number-mismatch] Client build (999999) is older than this Kibana server build (9007199254740991). The [400] error status in req id [29ad1f7c-056a-419b-9051-b45598aebd2c] may be due to client-server incompatibility!
```
## Summary
Fix https://github.com/elastic/kibana/issues/158910
Changes the behavior of the `/api/status` endpoint to always returns a
consistent http status code, and in particular:
- during the preboot stage
- when accessed by unauthenticated users and `status.allowAnonymous` is
`false`.
That way, `/api/status` can properly be used for readiness checks.
Please refer to https://github.com/elastic/kibana/issues/158910 for more
details.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR introduces two changes:
(1) Refactors the handler resolution logic to _not_ depend on the
`--serverless` cli arg by adding a new piece of config
`server.versioned.routeResolution` that accepts `newest | oldest`. This
piece of config is passed down instead of the `serverless` cli arg as
well as updating test cases
(2) Adds a new piece of config to turn off the client version checking.
This will be needed for rolling upgrades to allow old browser traffic to
reach new Kibana servers when there is stack version change.
Close https://github.com/elastic/kibana/issues/158723
## Open questions
* Do we want to make the version check still take _major_ version bumps
into account?
## Dearest Reviewers 👋
I've been working on this branch with @mistic and @tylersmalley and
we're really confident in these changes. Additionally, this changes code
in nearly every package in the repo so we don't plan to wait for reviews
to get in before merging this. If you'd like to have a concern
addressed, please feel free to leave a review, but assuming that nobody
raises a blocker in the next 24 hours we plan to merge this EOD pacific
tomorrow, 12/22.
We'll be paying close attention to any issues this causes after merging
and work on getting those fixed ASAP. 🚀
---
The operations team is not confident that we'll have the time to achieve
what we originally set out to accomplish by moving to Bazel with the
time and resources we have available. We have also bought ourselves some
headroom with improvements to babel-register, optimizer caching, and
typescript project structure.
In order to make sure we deliver packages as quickly as possible (many
teams really want them), with a usable and familiar developer
experience, this PR removes Bazel for building packages in favor of
using the same JIT transpilation we use for plugins.
Additionally, packages now use `kbn_references` (again, just copying the
dx from plugins to packages).
Because of the complex relationships between packages/plugins and in
order to prepare ourselves for automatic dependency detection tools we
plan to use in the future, this PR also introduces a "TS Project Linter"
which will validate that every tsconfig.json file meets a few
requirements:
1. the chain of base config files extended by each config includes
`tsconfig.base.json` and not `tsconfig.json`
1. the `include` config is used, and not `files`
2. the `exclude` config includes `target/**/*`
3. the `outDir` compiler option is specified as `target/types`
1. none of these compiler options are specified: `declaration`,
`declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`,
`paths`
4. all references to other packages/plugins use their pkg id, ie:
```js
// valid
{
"kbn_references": ["@kbn/core"]
}
// not valid
{
"kbn_references": [{ "path": "../../../src/core/tsconfig.json" }]
}
```
5. only packages/plugins which are imported somewhere in the ts code are
listed in `kbn_references`
This linter is not only validating all of the tsconfig.json files, but
it also will fix these config files to deal with just about any
violation that can be produced. Just run `node scripts/ts_project_linter
--fix` locally to apply these fixes, or let CI take care of
automatically fixing things and pushing the changes to your PR.
> **Example:** [`64e93e5`
(#146212)](64e93e5806)
When I merged main into my PR it included a change which removed the
`@kbn/core-injected-metadata-browser` package. After resolving the
conflicts I missed a few tsconfig files which included references to the
now removed package. The TS Project Linter identified that these
references were removed from the code and pushed a change to the PR to
remove them from the tsconfig.json files.
## No bazel? Does that mean no packages??
Nope! We're still doing packages but we're pretty sure now that we won't
be using Bazel to accomplish the 'distributed caching' and 'change-based
tasks' portions of the packages project.
This PR actually makes packages much easier to work with and will be
followed up with the bundling benefits described by the original
packages RFC. Then we'll work on documentation and advocacy for using
packages for any and all new code.
We're pretty confident that implementing distributed caching and
change-based tasks will be necessary in the future, but because of
recent improvements in the repo we think we can live without them for
**at least** a year.
## Wait, there are still BUILD.bazel files in the repo
Yes, there are still three webpack bundles which are built by Bazel: the
`@kbn/ui-shared-deps-npm` DLL, `@kbn/ui-shared-deps-src` externals, and
the `@kbn/monaco` workers. These three webpack bundles are still created
during bootstrap and remotely cached using bazel. The next phase of this
project is to figure out how to get the package bundling features
described in the RFC with the current optimizer, and we expect these
bundles to go away then. Until then any package that is used in those
three bundles still needs to have a BUILD.bazel file so that they can be
referenced by the remaining webpack builds.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* Use brotli compression
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
* Add integration test for brotli support
* Use import instead of require()
* Suppress build error on importing brok
* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'
* add brok as explicit package dep
* add `server.compression.brotli` config settings
* update documentation
* fix test utils
* fix more test configs
* add tests for endpoints too
* remove against endpoint for now
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co>
* [packages] add kibana.jsonc files
* auto-migrate to kibana.jsonc
* support interactive pkg id selection too
* remove old codeowners entry
* skip codeowners generation when .github/CODEOWNERS doesn't exist
* fall back to format validation if user is offline
* update question style
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* refact(NA): apply root_input_dir=src to each already created pkg
* refact(NA): update package generator
* fix(NA): correctly use rootDir
* fix(NA): use root input dir on latest introduced pkgs for jsts_transpiler macro
* chore(NA): merge with main
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
