This PR stops importing:
- `functionWrapper` and `fontStyle` test helpers in
`@kbn/presentation-util-plugin/common`
- `createFleetAuthzMock` test helper in `@kbn/fleet-plugin/common`
These are instead imported from a `*/mocks` or `*/test_helpers` import
target and the related `eslint-disable` comments were removed.
While working through some changes to the package system I fixed a bug
in the source classifier which made these issues clear and made a fix
required. The changes have been broken out of my massive PR to make
review simpler.
The location of plugins was previously somewhat irrelevant, but as we
move into packages it's more important that we can find all plugins in
the repository, and we would like to be able to do that without needing
to maintain a manifest somewhere to accomplish this. In order to make
this possible we plan to find any plugin/package by spotting all
kibana.json files which are not "fixtures". This allows plugin-like code
(but not actual plugin code) to exist for testing purposes, but it must
be within some form of "fixtures" directory, and any plugin that isn't
in a fixtures directory will be automatically pulled into the system
(though test plugins, examples, etc. will still only be loaded when the
plugin's path is passed via `--plugin-path`, the system will know about
them and use that knowledge for other things).
Since this is just a rename Operations will review and merge by EOD Jan
12th unless someone has a blocking concern.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
It seems like these files were unintentionally committed, or left
behind, so I have added a package linter rule which will verify that we
don't have any JS/TS files with exactly the same path, but different
extensions.
Updates the TsProject loading logic to automatically rebuild the
tsconfig-paths.json cache of tsconfig file locations which was
previously only updated by bootstrap. This means that people will no
longer be stopped when they are running tasks which didn't used to
strictly require a bootstrap, and told to bootstrap the repo. This is
specifically usable in this context because we are only populating the
cache in bootstrap because we are already scanning all the files in the
repo at that point. It's a quick operation (though one we ideally
wouldn't be executing constantly), so adding it here too just provides a
better DX.
Additionally, this PR adds a check to the start of the precommit hook
which will bootstrap the repo if the user just finished resolving a
merge. This puts the repo in a more correct state before running code to
validate code via eslint, etc.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
In order to avoid searching the `api_docs` directory at the root of the
repo in VSCode we have to exclude all api_docs dirs from search, so
instead of putting the source of the code that generates the api_docs in
a similarly named dir this just removes that wrapper.
This PR implements a linter like the TS Project linter, except for
packages in the repo. It does this by extracting the reusable bits from
the TS Project linter and reusing them for the project linter. The only
rule that exists for packages right now is that the "name" in the
package.json file matches the "id" in Kibana.jsonc. The goal is to use a
rule to migrate kibana.json files on the future.
Additionally, a new rule for validating the indentation of tsconfig.json
files was added.
Validating and fixing violations is what has triggered review by so many
teams, but we plan to treat those review requests as notifications of
the changes and not as blockers for merging.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds capability to run capacity testing for single apis #143066
Currently in main we have to 2 types of performance tests:
- single user performance journey that simulates single end-user
experience in browser
- scalability journey that uses APM traces from single user performance
journey to simulate multiple end-users experience
This new type of performance tests allow to better understand how each
single server api scale under the similar load.
How to run locally:
make sure to clone the latest main branch of
[elastic/kibana-load-testing](https://github.com/elastic/kibana-load-testing)
in Kibana repo run:
`node scripts/run_scalability.js --journey-path
x-pack/test/scalability/apis/api.core.capabilities.json`
How it works:
FTR is used to start Kibana/ES and run Gatling simulation with json file
as input. After run the latest report matching journey name is parsed to
get perf metrics and report using EBT to the Telemetry cluster.
How will it run after merge:
I plan to run pipeline every 3 hours on bare metal machine and report
metrics to Telemetry staging cluster.
<img width="2023" alt="image"
src="https://user-images.githubusercontent.com/10977896/208771628-f4f5dbcb-cb73-40c6-9aa1-4ec3fbf5285b.png">
APM traces are collected and reported to Kibana stats cluster:
<img width="1520" alt="image"
src="https://user-images.githubusercontent.com/10977896/208771323-4cca531a-eeea-4941-8b01-50b890f932b1.png">
What metrics are collected:
1. warmupAvgResponseTime - average response time during warmup phase
2. rpsAtWarmup - average requests per second during warmup phase
3. warmupDuration
4. responseTimeMetric (default: 85%) Gatling has response time
25/50/75/80/85/90/95/99 percentiles, as well as min/max values
5. threshold1ResponseTime (default 3000 ms)
6. rpsAtThreshold1 requests per second when `responseTimeMetric` first
reach threshold1ResponseTime
7. threshold2ResponseTime
8. rpsAtThreshold2 (default 9000 ms)
9. threshold3ResponseTime
10. rpsAtThreshold3 (default 15000 ms)
As long as we agree on metrics I will update indexer for telemetry.
Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
There was recent discussion on tracking down a possible ES regression,
and the best way to find the snapshot build schedule. This updates our
dev docs to include links to the snapshot update pipelines.
Addresses: https://github.com/elastic/kibana/issues/140263
This PR was inspired by https://github.com/elastic/kibana/pull/146649 and while working on persistent rules table state https://github.com/elastic/kibana/pull/145111.
## Summary
It includes improvements for url search parameters manipulation functionality. In particular `useGetInitialUrlParamValue` and `useReplaceUrlParams` hooks.
The main idea is to isolate the encoding layer ([rison](https://github.com/Nanonid/rison)) as an implementation detail so `useGetInitialUrlParamValue` and `useReplaceUrlParams` consumers can just provide types they wish and the hooks serialize/desirealize the data into appropriate format under the hood.
On top of that after `@kbn/rison` was added in https://github.com/elastic/kibana/pull/146649 it's possible to use this package directly to encode and decode parameters whenever necessary.
The improvements include
- store unserialized url parameters state in the redux store
- encode and decode data by using functions from `@kbn/rison` directly
- let `useReplaceUrlParams` accept an updater object
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
In https://github.com/elastic/kibana/pull/146212 the default
`sourceMaps` config for babel register changed from `both` to `true`,
which creates the source maps but does not embed them in the compiled
output so that dev-tools are able to inspect them even though they are
not on the filesystem.
cc @vitaliidm
Resolves: https://github.com/elastic/kibana/issues/109937
In this PR, I'm fixing the `Task Manager detected a degradation in
performance` log message to provide the proper versioned link to the
task manager health monitoring docs. Prior to this PR, it would always
point to main / master docs.
## To verify
1. Turn on debug logging in your kibana.yml file
```
logging:
loggers:
- name: plugins.taskManager
level: debug
```
2. Move this code line outside of the `if (logLevel..` statement =>
4c7ce9d249/x-pack/plugins/task_manager/server/lib/log_health_metrics.ts (L99)
3. Startup Kibana
4. Notice the `Task Manager detected a degradation in performance...`
logged
5. Test the URL provided by the log message
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Autocomplete support for ESQL lang. Initially target branch for that PR
was
[elastic:feature-esql](https://github.com/elastic/kibana/tree/feature-esql)
but then we decided to merge it separately. This PR is as copy of
#148006
## Notes:
Important: please do `yarn kbn clean & yarn kbn bootstrap` before
testing.
## How to update syntax.
`antlr` syntax was copied from `ES` and was slightly modified. In case
if you need to update it please follow next steps:
- modify `esql_parser.g4 `and/or `esql_lexer.g4` files
- go to `kbn-monaco` package and execute `bazel clean & npm run
build:antlr4ts:painless`
- go to /painless_parser.ts file and revert the following change:
<img width="478" alt="image"
src="https://user-images.githubusercontent.com/20072247/209540586-bb77cad1-a6f0-42fa-9875-025bd4afbace.png">
- do `yarn kbn bootstrap`
Follow up to #147526 which had to be reverted.
Resolves#127481
## Release notes
Include IP address in audit log
## Testing
1. Start Elasticsearch with trial license: `yarn es snapshot --license
trial`
2. Update `kibana.dev.yaml`:
```yaml
xpack.security.audit.enabled: true
xpack.security.audit.appender:
type: console
layout:
type: json
```
3. Observe audit logs in console when interacting with Kibana:
```json
{
"@timestamp": "2022-12-13T15:50:42.236+00:00",
"message": "User is requesting [/dev/internal/security/me] endpoint",
"client": {
"ip": "127.0.0.1"
},
"http": {
"request": {
"headers": {
"x-forwarded-for": "1.1.1.1, 127.0.0.1"
}
}
}
}
```
Note: You will see the `x-forwarded-for` field populated when running
Kibana in development mode (`yarn start`) since Kibana runs behind a
development proxy.
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
We're seeing many extra eslint-disabled counts on PRs right now because
new .gitignore'd files are being written to plugins/packages. This fixes
the logic to stop checking every file in a plugin/package directory, and
instead only check files that are committed to the repository.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/124946
## Summary
Takes a step toward optimising our migration paths by only reindexing
(an expensive operation) when needed by checking whether the current SO
mappings have "changed".
By "changed" we mean that we have detected a new md5 checksum of any
registered saved object type relative to the hashes we have stored.
## How to test
These changes are constrained to the `model.ts`, a test was added for
correctly detecting that mappings are the same during the `INIT` phase
to send us down the correct migration path.
Additionally, we have a new Jest integration test `skip_reindex.test.ts`
that runs Kibana and inspects the logs for the expected model
transitions.
Everything else should remain the same.
## Happy path for skipping reindex
```
RUN INIT
IF !versionMigrationIsComplete AND
!kibana indexBelongsToLaterVersion AND
!waitForMigrationCompletion AND
mappingsAreUnchanged
THEN
the migration operations must target the existing .kibana_x.y.z_001 index
RUN PREPARE_COMPATIBLE_MIGRATION (new state)
we remove old version aliases (prevent old reads/writes), and set the current version alias (prevent old migrations)
SKIP LEGACY_SET_WRITE_BLOCK
SKIP ...
SKIP SET_SOURCE_WRITE_BLOCK
SKIP ...
SKIP REFRESH_TARGET
RUN OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT
...
RUN MARK_VERSION_INDEX_READY
DONE
```
## Notes
* This optimisation will only be triggered when there are no mappings
changes AND we are upgrading to a new version. This does not cover all
cases. In future we will make this more sophisticated by checking for
incompatible changes to mappings and only reindexing when those occur.
## Related
* https://github.com/elastic/kibana/pull/147503
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Gerard Soldevila <gerard.soldevila@elastic.co>
## Dearest Reviewers 👋
I've been working on this branch with @mistic and @tylersmalley and
we're really confident in these changes. Additionally, this changes code
in nearly every package in the repo so we don't plan to wait for reviews
to get in before merging this. If you'd like to have a concern
addressed, please feel free to leave a review, but assuming that nobody
raises a blocker in the next 24 hours we plan to merge this EOD pacific
tomorrow, 12/22.
We'll be paying close attention to any issues this causes after merging
and work on getting those fixed ASAP. 🚀
---
The operations team is not confident that we'll have the time to achieve
what we originally set out to accomplish by moving to Bazel with the
time and resources we have available. We have also bought ourselves some
headroom with improvements to babel-register, optimizer caching, and
typescript project structure.
In order to make sure we deliver packages as quickly as possible (many
teams really want them), with a usable and familiar developer
experience, this PR removes Bazel for building packages in favor of
using the same JIT transpilation we use for plugins.
Additionally, packages now use `kbn_references` (again, just copying the
dx from plugins to packages).
Because of the complex relationships between packages/plugins and in
order to prepare ourselves for automatic dependency detection tools we
plan to use in the future, this PR also introduces a "TS Project Linter"
which will validate that every tsconfig.json file meets a few
requirements:
1. the chain of base config files extended by each config includes
`tsconfig.base.json` and not `tsconfig.json`
1. the `include` config is used, and not `files`
2. the `exclude` config includes `target/**/*`
3. the `outDir` compiler option is specified as `target/types`
1. none of these compiler options are specified: `declaration`,
`declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`,
`paths`
4. all references to other packages/plugins use their pkg id, ie:
```js
// valid
{
"kbn_references": ["@kbn/core"]
}
// not valid
{
"kbn_references": [{ "path": "../../../src/core/tsconfig.json" }]
}
```
5. only packages/plugins which are imported somewhere in the ts code are
listed in `kbn_references`
This linter is not only validating all of the tsconfig.json files, but
it also will fix these config files to deal with just about any
violation that can be produced. Just run `node scripts/ts_project_linter
--fix` locally to apply these fixes, or let CI take care of
automatically fixing things and pushing the changes to your PR.
> **Example:** [`64e93e5`
(#146212)](64e93e5806)
When I merged main into my PR it included a change which removed the
`@kbn/core-injected-metadata-browser` package. After resolving the
conflicts I missed a few tsconfig files which included references to the
now removed package. The TS Project Linter identified that these
references were removed from the code and pushed a change to the PR to
remove them from the tsconfig.json files.
## No bazel? Does that mean no packages??
Nope! We're still doing packages but we're pretty sure now that we won't
be using Bazel to accomplish the 'distributed caching' and 'change-based
tasks' portions of the packages project.
This PR actually makes packages much easier to work with and will be
followed up with the bundling benefits described by the original
packages RFC. Then we'll work on documentation and advocacy for using
packages for any and all new code.
We're pretty confident that implementing distributed caching and
change-based tasks will be necessary in the future, but because of
recent improvements in the repo we think we can live without them for
**at least** a year.
## Wait, there are still BUILD.bazel files in the repo
Yes, there are still three webpack bundles which are built by Bazel: the
`@kbn/ui-shared-deps-npm` DLL, `@kbn/ui-shared-deps-src` externals, and
the `@kbn/monaco` workers. These three webpack bundles are still created
during bootstrap and remotely cached using bazel. The next phase of this
project is to figure out how to get the package bundling features
described in the RFC with the current optimizer, and we expect these
bundles to go away then. Until then any package that is used in those
three bundles still needs to have a BUILD.bazel file so that they can be
referenced by the remaining webpack builds.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/147853
(See issue for more context)
Given we don't want to spend much effort in user impersonation right
now, the compromise would be to make sure that our system is resilient
to creating 'stubbed' requests. The best way to do that is to offer an
official, proper way to instantiate `CoreKibanaRequest` without having a
proper `HAPI.Request` instance.
This PR is introducing the `FakeRawRequest` type, and changing the
signature of `CoreKibanaRequest.create` to accept this new type.
d2eb3e1801/packages/core/http/core-http-server/src/router/raw_request.ts (L14-L36)
The PR also adapts the existing, non-test usages of
`CoreKibanaRequest.from(someUglyStub as unknown as Request)` to use the
new `FakeRawRequest` type instead.
Part of #141780
## Release notes
Include Cross-Origin-Opener-Policy in default response headers
## Testing
Load Kibana; you should see Kibana respond with
`Cross-Origin-Opener-Policy: same-origin` header for page loads and API
requests.
eui@70.4.0 ⏩ eui@71.0.0
---
## [`71.0.0`](https://github.com/elastic/eui/tree/v71.0.0)
- Implemented new `EuiRange` and `EuiDualRange` designs where the
`levels` are now on top of the tracks
([#6092](https://github.com/elastic/eui/pull/6092))
- Added `discuss` and `dotInCircle` glyphs to `EuiIcon`
([#6434](https://github.com/elastic/eui/pull/6434))
- Added `article` glyph to `EuiIcon`
([#6437](https://github.com/elastic/eui/pull/6437))
- Changed the `EuiProvider` usage warnings to not rely on development
mode. ([#6451](https://github.com/elastic/eui/pull/6451))
**Breaking changes**
- `EuiDualRange` now explicitly requires both `min` and `max` via props
types, to match `EuiRange`
([#6092](https://github.com/elastic/eui/pull/6092))
- `EuiRange` and `EuiDualRange`'s `compressed` size no longer impacts
track or level sizes, but continues to compress tick and input sizes.
([#6092](https://github.com/elastic/eui/pull/6092))
- Removed all variables for the following components from EUI's theme
JSON files: ([#6443](https://github.com/elastic/eui/pull/6443))
- `euiCollapsibleNav*`
- `euiColorPicker*`
- `euiContextMenu*`
- `euiControlBar*`
- `euiDataGrid* `(except for z-indices and cell padding sizes)
- `euiDatePicker*`
- `euiSuperDatePicker*`
- `euiDragAndDrop*`
- `euiEuiEmptyPrompt*`
- `euiFilePicker*`
- `euiRange*`
- `euiHeaderLinks*`
- `euiKeyPad*`
- `euiMarkdownEditor*`
- `euiResizable*`
- `euiSelectable*`
- `euiSideNav*`
- `euiStep*`
- `euiSuggest*`
- `euiTable*` (except for color variables)
- `euiTooltip*`
- `euiButtonFontWeight`, `euiButtonDefaultTransparency`, and
`euiButtonMinWidth`
- If you were importing any of the above removed JSON variables, we
strongly recommend using generic color or sizing variables from
`useEuiTheme()` instead.
([#6443](https://github.com/elastic/eui/pull/6443))
**CSS-in-JS conversions**
- Converted `EuiRange` and `EuiDualRange` to Emotion; Removed
`$euiRangeThumbRadius`
([#6092](https://github.com/elastic/eui/pull/6092))
- Added a new `logicalStyles` utility that automatically converts all
non-logical properties in a `style` object to their corresponding
logical properties ([#6426](https://github.com/elastic/eui/pull/6426))
- Added a new `logicalShorthandCSS` utility that automatically converts
`margin`, `padding`, and other 4-sided shorthands to their corresponding
logical properties ([#6429](https://github.com/elastic/eui/pull/6429))
- Added a new `logicalBorderRadiusCSS` utility that automatically
converts `border-radius` to corresponding logical properties
([#6429](https://github.com/elastic/eui/pull/6429))
Co-authored-by: Constance Chen <constance.chen@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
