## Dearest Reviewers 👋
I've been working on this branch with @mistic and @tylersmalley and
we're really confident in these changes. Additionally, this changes code
in nearly every package in the repo so we don't plan to wait for reviews
to get in before merging this. If you'd like to have a concern
addressed, please feel free to leave a review, but assuming that nobody
raises a blocker in the next 24 hours we plan to merge this EOD pacific
tomorrow, 12/22.
We'll be paying close attention to any issues this causes after merging
and work on getting those fixed ASAP. 🚀
---
The operations team is not confident that we'll have the time to achieve
what we originally set out to accomplish by moving to Bazel with the
time and resources we have available. We have also bought ourselves some
headroom with improvements to babel-register, optimizer caching, and
typescript project structure.
In order to make sure we deliver packages as quickly as possible (many
teams really want them), with a usable and familiar developer
experience, this PR removes Bazel for building packages in favor of
using the same JIT transpilation we use for plugins.
Additionally, packages now use `kbn_references` (again, just copying the
dx from plugins to packages).
Because of the complex relationships between packages/plugins and in
order to prepare ourselves for automatic dependency detection tools we
plan to use in the future, this PR also introduces a "TS Project Linter"
which will validate that every tsconfig.json file meets a few
requirements:
1. the chain of base config files extended by each config includes
`tsconfig.base.json` and not `tsconfig.json`
1. the `include` config is used, and not `files`
2. the `exclude` config includes `target/**/*`
3. the `outDir` compiler option is specified as `target/types`
1. none of these compiler options are specified: `declaration`,
`declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`,
`paths`
4. all references to other packages/plugins use their pkg id, ie:
```js
// valid
{
"kbn_references": ["@kbn/core"]
}
// not valid
{
"kbn_references": [{ "path": "../../../src/core/tsconfig.json" }]
}
```
5. only packages/plugins which are imported somewhere in the ts code are
listed in `kbn_references`
This linter is not only validating all of the tsconfig.json files, but
it also will fix these config files to deal with just about any
violation that can be produced. Just run `node scripts/ts_project_linter
--fix` locally to apply these fixes, or let CI take care of
automatically fixing things and pushing the changes to your PR.
> **Example:** [`64e93e5`
(#146212)](64e93e5806)
When I merged main into my PR it included a change which removed the
`@kbn/core-injected-metadata-browser` package. After resolving the
conflicts I missed a few tsconfig files which included references to the
now removed package. The TS Project Linter identified that these
references were removed from the code and pushed a change to the PR to
remove them from the tsconfig.json files.
## No bazel? Does that mean no packages??
Nope! We're still doing packages but we're pretty sure now that we won't
be using Bazel to accomplish the 'distributed caching' and 'change-based
tasks' portions of the packages project.
This PR actually makes packages much easier to work with and will be
followed up with the bundling benefits described by the original
packages RFC. Then we'll work on documentation and advocacy for using
packages for any and all new code.
We're pretty confident that implementing distributed caching and
change-based tasks will be necessary in the future, but because of
recent improvements in the repo we think we can live without them for
**at least** a year.
## Wait, there are still BUILD.bazel files in the repo
Yes, there are still three webpack bundles which are built by Bazel: the
`@kbn/ui-shared-deps-npm` DLL, `@kbn/ui-shared-deps-src` externals, and
the `@kbn/monaco` workers. These three webpack bundles are still created
during bootstrap and remotely cached using bazel. The next phase of this
project is to figure out how to get the package bundling features
described in the RFC with the current optimizer, and we expect these
bundles to go away then. Until then any package that is used in those
three bundles still needs to have a BUILD.bazel file so that they can be
referenced by the remaining webpack builds.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/147853
(See issue for more context)
Given we don't want to spend much effort in user impersonation right
now, the compromise would be to make sure that our system is resilient
to creating 'stubbed' requests. The best way to do that is to offer an
official, proper way to instantiate `CoreKibanaRequest` without having a
proper `HAPI.Request` instance.
This PR is introducing the `FakeRawRequest` type, and changing the
signature of `CoreKibanaRequest.create` to accept this new type.
d2eb3e1801/packages/core/http/core-http-server/src/router/raw_request.ts (L14-L36)
The PR also adapts the existing, non-test usages of
`CoreKibanaRequest.from(someUglyStub as unknown as Request)` to use the
new `FakeRawRequest` type instead.
## Summary
The defaultIndex setting title and description were poor. I've updated
them to correctly reference data views and be more specific about where
they're used.
Closes https://github.com/elastic/kibana/issues/143609
## Summary
<details><summary>Display link on rule details page -> add exception
flyout</summary>
<img width="1108" alt="image"
src="https://user-images.githubusercontent.com/616158/209036807-973a83b9-2903-4d45-a15d-7fd6751f30a1.png">
</details>
<details><summary>Adds manage rules functionality to list view overflow
menu</summary>
<img width="802" alt="Screen Shot 2022-12-21 at 4 59 11 PM"
src="https://user-images.githubusercontent.com/915763/209010900-4b609f9c-6fb1-46ca-a1f9-6aa3e67f5366.png">
</details>
<details><summary>Renames page to Shared exception lists and updates
page description with hyperlink to manage rules page</summary>
<img width="635" alt="Screen Shot 2022-12-21 at 4 58 58 PM"
src="https://user-images.githubusercontent.com/915763/209010964-2c7ba8bd-7b1c-4bf9-a0a6-12a456386aed.png">
</details>
<details><summary>Updates sidebar title + navigation title</summary>
<img width="590" alt="Screen Shot 2022-12-21 at 4 58 36 PM"
src="https://user-images.githubusercontent.com/915763/209011050-45df0005-37eb-42b7-840f-eb916472d987.png">
</details>
~~@jethr0null FYI displaying all shared exception lists on the rule
details -> add exception flyout is not possible for 8.6. The ability to
add any exception to any list was not part of the original design of the
component and would require more time to update the component and the
data model to allow for this sort of flexibility.~~
Looks like the above is possible if we pass in `null` for the rules
property of the `AddExceptionFlyout` but we might want to finalize a
design to better help users trying to add the exception to a shared list
while **inside** of the rule details page.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Pedro Jaramillo <pedro.jaramillo@elastic.co>
Co-authored-by: Khristinin Nikita <nikita.khristinin@elastic.co>
## Summary
Looking at why Discover was loading data view field lists were loading
more than needed, I found more reasons than actual field list loads
occurring but I think they're all deserving of improvement.
- Discover was loading the default data view to see if there was an
existing data view. We can rely on the `hasData` api instead.
- We were loading a data view and then refreshing the field list. If the
data view was being loaded fresh, its meant the field list was loaded
twice. Field list refresh has been integrated into the dataViews api.
- We were loading a data view, checking it if was adhoc, and tossing it
if it wasn't. Now we keep the reference.
Previously on page load, discover would make 3 calls to
`fields_for_wildcard`, now it makes two. It loads all the field with one
request and it makes an additional request where it applies the current
filter to find relevant fields.
Closes https://github.com/elastic/kibana/issues/147744
## Summary
Removes an old, deprecated feature that calls the Agents API filtered by
Endpoints. This type of call hasn't been supported for some time (over a
year or so). It was realized that it was failing more when we made the
RBAC changes for this release and we were getting `403` errors from the
API.
NOTE: This feature hasn't worked since the creation of Fleet Server, so
it is safe to remove. The required query functionality of getting a set
of Agents filtered by only those running Endpoint was removed when we
switched over to Fleet Server about 1.5 years ago.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of #143455
Previously agents would be unenrolled after a given time by the fleet
server. Instead, they'll be considered `Inactive`. Agents in an
`Inactive` state are hidden from the UI by default, but their API keys
remain active. This allows these agents to check in again at any time
without requesting new API keys.`inactivity_timeout` defaults to 10
minutes or can be configured on a per policy basis.
Agents that are manually unenrolled will go into the new `Unenrolled`
status.
These changes mean that we now need to get agent policies before knowing
the agents status, we have used a runtime field to calculate the status
at search time, this allows us to easily filter and aggregate on the
status.
### Performance
For 120 agents (20 of each main status):
- filter call with filters: 90ms
- agent status summary call: 83ms
For 12k agents (2k of each main status):
- filter call with filters: 455ms
- agent status summary call: 500ms
For 120k agents (20k of each main status):
- filter call with filters: 2.2s
- agent status summary call: 2.1s
### Manual Testing
the create agent script can be used to test this at scale e.g create 10k
agents of each of the given statuses:
```bash
cd x-pack/plugins/fleet
node scripts/create_agents --count 10000 --kibana http://localhost:5601/myprefix--status offline,online,inactive,error,updating,unenrolled --inactivityTimeout 360 --delete
```
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
This breaks out inline code in
`x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts` to
functions in separate files and adds jest unit tests for each function.
The mocks used for jest unit tests are used as expected data in the API
integration tests. This allows to make sure that the static mocks are
still up to date should we have to update API integration tests based on
upstream tests.
### Description
Recently it was discovered that our initial path to analytics brings
some inconsistency into Enterprise Search routes, so it was decided to
replace the `analytics/api` path with `api/analytics`.
This PR is dedicated to changing Analytics DNS path from `analytics/api`
to `api/analytics`
## Summary
Resolves: https://github.com/elastic/kibana/issues/139788
Changes the createTimeline function to use the
kibana.alert.rule.created_by string for investigation guide author name
in timeline, and uses the actively logged in user for regular comments,
so that what is displayed does not oscillate from hard coded 'elastic'
to the current user name.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>
## Summary
This PR:
- Adds index aliases to the destination index for Transforms:
- It will automatically appends `{{package-version}}` name to the
`destination_index_name` specified in transform.yml
- Create a `{destination_index_name}.all` that points to all the
destination indices from all the previous versions and new version
- Create a `{destination_index_name}.latest` that points to just the
destination index of the new version
- Upgrading package to a new version no longer deletes the destination
index
- Downgrading package to an older version (e.g. from v3 to v1) will:
- Delete the transform from the newer version (v3), create transform for
the older version (v1)
- If the older version was previously installed and the destination
index of the older version v1 already exists: update the alias
`{destination_index_name}.latest` to point to destination index v1.
- If the older version was never installed and destination index of the
older version v1 does not exist: create the destination index with
`{destination_index_name}.all` and `{destination_index_name}.latest`
alias.
- Support installing transforms concurrently and sequentially.
- If the `order` is specified in the `transform.yml`'s `_meta` section,
and all the numerical order are unique, transforms will be created and
started sequentially. If not, they will be created and started
concurrently.
- Support versioning of transforms. If `fleet_transform_version` is
specified in `transform.yml`'s `_meta` section:
- If `fleet_transform_version` changed (either incremented or
decremented): delete old transform, keep the old destination index,
install new index templates, component templates, and transform
- If `fleet_transform_version` remains the same: keep old transform,
keep the old destination index, do nothing new
- Fixes an issue with the mappings and template not being applied to the
destination index correctly when the destination index has an ingest
pipeline. Previously, when the transform is associated with an ingest
pipeline, we add the ingest pipeline to the settings when calling `PUT
index/{transform-destination-index}`. This in turns makes the settings
and mappings from the component templates not apply correctly to the
destination. This PR changes so that it will add the pipeline to the
component template.
Technical changes:
- [Adds a new `ElasticsearchAssetType` for `index`
](https://github.com/elastic/kibana/pull/142920/files#diff-395b753abcf65cdc07993651d6211a49194a76c0497e5f234ea13736cf24a2c0)
- [Adds a new `version` for `PACKAGES_SAVED_OBJECT_TYPE`
](https://github.com/elastic/kibana/pull/142920/files#diff-4e164e3802d5171bf96a2cf9c91c20e97c5e0b74b2f93187a072d9a3139f1c18)
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Fixes#147648
## Summary
Replace React.lazy and withSuspense with async imports in expressions
plugins
The `withSuspense` method shows a loading state (`EuiLoadingSpinner`)
while React.lazy imports the component module. However, expression
renderers have a second loading state after the module is imported. This
causes a flash of two separate loading states . Unfortunately, this
caused a much larger problem with reporting where the report was
triggered before the second loading state was complete.
Using async imports rather than `React.lazy` ensures that components are
fully available before the expression renderers call their `done`
method.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
