Resolves https://github.com/elastic/kibana/issues/161393
## Summary
This PR is the first of a series to implement the summary and search
improvement feature on SLO.
Every PR will be merged against the feature branch:
`slo/feature-branch`. And we'll update this feature branch with main as
often as possible to keep conflict as a minimum.
This PR changes the SLO rollup index mapping, adding more fields to it,
that we are going to use to summarize the SLO rollup data later.
It also includes the SLO summary index templates (mappings, settings,
template) and install them with the other templates.
Since this is a **breaking change**, any SLO running will be shown as
`NO_DATA`. You can remove the SLO through the API or the UI, or update
them (make sure you change something significant (time window, indicator
params, ...) in order to induce a revision bump) so the underlying
transform is recreated using the new index structure.
> Pre-req for https://github.com/elastic/kibana/issues/56406
## Summary
We've had a long-standing problem in Kibana around our use of React
context, particularly with EUI and i18n. There hasn't existed an
idempotent context structure, and that has lead to a lot of unexpected
results, (e.g. missing translations, inconsistent dark mode, excess
context providers, etc).
The biggest change coming from this PR is knowing exactly which provider
to use in a particular use case. This means, for example,
`ReactDOM.render` calls won't be missing `i18n` or `theme` due to a
missing context. It also allows consumers to use `darkMode` without
having to read the `uiSetting` themselves, instead allowing the context
to do it for them.
We also haven't been honoring the intended [`EuiProvider`
API](https://eui.elastic.co/#/utilities/provider#theming-and-global-styles)...
in some cases we've been creating and re-creating the Emotion caches,
often by copy/paste of the cache code. We've also been nesting
`EuiThemeProvider` contexts unnecessarily-- thinking we need to render a
theme provider in an isolated component-- which renders an additional
`span` element into the DOM.
This PR attempts to address this inconsistency by creating a set of
context providers divided by use case:
### `KibanaRootContextProvider`
A root context provider for Kibana. This is the top level context
provider that wraps the entire application. It is responsible for
initializing all of the other contexts and providing them to the
application. It's provided as a package for specific use cases, (e.g.
the `RenderingService`, cases where we replace the entire page content,
Storybook, testing, etc), but not intended for plugins.
### `KibanaRenderContextProvider`
A render context provider for Kibana. This context is designed to be
used with ad-hoc renders of React components, (usually with
`ReactDOM.render`).
### `KibanaThemeContextProvider`
A theme context provider for Kibana. A corollary to EUI's
`EuiThemeProvider`, it uses Kibana services to ensure the EUI Theme is
customized correctly.
### (deprecated) `KibanaStyledComponentsThemeProvider`
A styled components theme provider for Kibana. This package is supplied
for compatibility with legacy code, but should not be used in new code.
## Deprecation strategy
This PR does *not* change any use of context by consumers. It maps the
existing contexts in `kibanaReact` to the new contexts, (along with the
loose API). This means that we won't have completely fixed all of our
dark mode issues yet. But this is necessary to keep this PR focused on
the change, rather than drawing in a lot of teams to review individual
uses.
We should, however, see an immediate performance improvement in the UI
from the reduction in `EuiProvider` calls.
## Open questions
- [ ] Does it make sense to expose a `useTheme` hook from
`@kbn/react-kibana-context-theme` to replace `useEuiTheme`?
## Next steps
- [ ] Update deprecated uses to new contexts.
- [ ] Audit and update calls to `ReactDOM.render`.
- [ ] Add ESLint rule to warn for use of EUI contexts.
- [ ] Delete code from `kibanaReact`.
This PR makes the following changes:
- Update look & feel of contextual insights (previously called prompts)
according to the new design that is being developed. Some things might
still change, but hopefully not too much.
- Move all the Observability AI Assistant (previously called CoPilot)
code into a separate plugin for better isolation, more specific code
ownership and to solve some circular dependency issues
- Use connectors instead of a kibana.yml setting
Note: for OpenAI, the model is currently hardcoded to `gpt-4` until
https://github.com/elastic/kibana/issues/162204 has been addressed.
557676b6-065a-4b6f-86b2-1f0c2fd5e07e
---------
Co-authored-by: Coen Warmer <coen.warmer@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Uses the recently created [category validation
package](https://github.com/elastic/kibana/pull/161261) to perform
validation on the field selected for pattern analysis.
If the field is considered unsuitable for categorization, a warning
callout is displayed which lists the reasons it is unsuitable.
If the field is suitable, no callout is displayed.
Other changes:
- Adds the selected field to the URL state, so it is remembered on page
refresh.
- If no field is in the URL, it will look for a field called `message`
in the data view and auto select it.
- renames the ML route `/jobs/categorization_field_examples` to
`/jobs/categorization_field_validation` as it is a more accurate name
and it's consistent with the newly added route in AIOPs.
**Log Pattern Analysis page in ML**
**Log Pattern Analysis flyout in Discover**
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#159777
## 📝 Summary
In this PR, I changed the timestamp used in rule execution and the
preview chart of the new threshold rule.
I created a separate ticket
(https://github.com/elastic/kibana/issues/162560) to implement
`infra/metrics_explorer` in the observability plugin, and in the
meantime, I adjusted the API also to accept timeFieldName.
Also, I have a separate ticket for improving data view validation
(https://github.com/elastic/kibana/issues/162554)
## 🧪 How to test
### New threshold rule
- Create a data view of a timestamp other than `@timestamp`
- Create a rule with this data view and make sure the preview and
generated alerts are as expected
- Create a rule with grouping and filtering and check the preview and
generated alerts
### Metric threshold rule
- Create a metric threshold rule to make sure preview and rule execution
works as before
Fixes https://github.com/elastic/kibana/issues/156980
Fixes https://github.com/elastic/kibana/issues/123986
Addresses https://github.com/elastic/kibana/issues/162218
## Summary
This PR fixes the empty error toasts that are displayed when an index
action (delete, open, close, flush, etc.) is unsuccessful. The errors
from each of these actions are handled by the
[handleEsError](a3c0914dae/src/plugins/es_ui_shared/__packages_do_not_import__/errors/handle_es_error.ts (L24))
function which is why we need to use the `error.body.message` field to
get a correct error message (instead of `error.message` which is
undefined).
**How to test:**
Start Kibana and perform an invalid action on some index.
For example, to test the error toast from the Delete action, delete a
hidden index that is a write index of an existing data stream:
<img width="1553" alt="Screenshot 2023-07-27 at 18 19 52"
src="91b5e5db-3c1c-4c15-addd-83bedfac6785">
---------
Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
## Summary
* Introduces a new API, POST `/api/risk_scores/calculate`, that triggers
the code introduced here
* As with the [preview
route](https://github.com/elastic/kibana/pull/155966), this endpoint is
behind the `riskScoringRoutesEnabled` feature flag
* We intend to __REMOVE__ this endpoint before 8.10 release; it's mainly
a convenience/checkpoint for testing the existing code. The next PR will
introduce a scheduled Task Manager task that invokes this code
periodically.
* Updates to the /preview route:
* `data_view_id` is now a required parameter on both endpoints. If a
dataview is not found by that ID, the id is used as the general index
pattern to the query.
* Response has been updated to be more similar to the [ECS risk
fields](https://github.com/elastic/ecs/pull/2236) powering this data.
* Mappings created by the [Data
Client](https://github.com/elastic/kibana/pull/158422) have been updated
to be aligned to the ECS risk fields (linked above)
* Adds/updates the [OpenAPI
spec](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/server/lib/risk_engine/schema/risk_score_apis.yml)
for these endpoints; useful starting point if you're trying to get
oriented here.
## Things to review
* [PR Demo
environment](https://rylnd-pr-161503-risk-score-task-api.kbndev.co/app/home)
* Preview API and related UI still works as expected
* Calculation/Persistence API correctly bootstraps/persists data
* correct mappings/ILM are created
* things work in non-default spaces
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Follow up to #161676.
Since we added support for handling SSE-streams returned by OpenAI APIs
to `@kbn/ml-response-stream` in #162335, this updates the "Gen AI
Streaming Response" developer example to make use of the custom hook
`useFetchStream()` from `@kbn/ml-response-stream` instead of its inline
code to process the stream on the client.
It also improves the refresh behaviour: Previously, once a prompt was
entered and the accordion opened, every keystroke in the prompt textarea
would cause a new request to the AI endpoint. This update adds a
`Refresh prompt` to only do a new request on this user action. Support
for cancelling requests when unmounting the `StreamingResponse`
component was also added.
### Checklist
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
I removed the participants' tests from
`x-pack/plugins/cases/public/components/case_view/components/case_view_activity.test.tsx`
There were a few things being tested:
| Old Test | Where is it covered |
| ------------- | ------------- |
| should render the participants correctly | Moved to an e2e test |
| should render Unknown users correctly | Moved to `UserList` |
| should render assignees in the participants section | Moved to an e2e
test |
I also added a small test for reporters.
Fixes#152204
Flaky test runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2734
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds new meta fields for `license_uid`, `cluster_uuid`, and
`cluster_name` to Endpoint Policy for use by the Endpoint in the
upcoming Cloud Lookup feature. These values will be used by Endpoint to
call an external API for use in analyzing malware.
New policies will have the fields populated.
Existing policies will go through an SO migration to add the empty
fields. Users will turn on the Cloud Lookup feature via Policy (coming
in another PR) which will then trigger the update callback to populate
the fields for those existing policies.
Policy fields look like this (taken from my local dev instance):
```
...
policy:
meta:
license: trial
license_uid: 1a427caf-9417-442c-b674-84e4b7100c29
cluster_uuid: DttCDxWnSF6UTSWvnUTZvg
cluster_name: elasticsearch
cloud: false
...
```
Policy in the app:
<img width="1728" alt="image"
src="480458fc-6096-422e-af3a-207b1b71b069">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/security-team/issues/7102
Establishes the `/common/api` folder structure for the lists plugin.
This PR simply imports and re-exports the schemas from the schema
package since the schemas are all already separated from the `server`
code. Future PRs will replace these re-exports with schemas generated
from OpenAPI specs.
## Summary
This PR adds a rule preview panel to the expandable flyout:
- Preview panel skeleton is added, now we can open a preview on top of
right section of flyout
- Go to rule details button is replaced by a button that will open a
rule preview panel
- The rule preview contains placeholder sections (About, Definition,
Schedule) and footer with a link to rule details page
**How to test**
- add `xpack.securitySolution.enableExperimental:
['securityFlyoutEnabled']` to the `kibana.dev.json` file
- go to the Alerts page, and click on the expand detail button on any
row of the table
- click on `Overview`, `About`, then `Rule summary`
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Unblocks #162149
## Summary
Fixes serverless security response headers tests by using the internal
request header. This PR also opts to use the `/logout` redirect endpoint
in testing, as it is more relevant to serverless.
## Summary
Minor updates to ML inference multi-field configuration UI (no
functional changes):
- Add spacer between field selector and selected field list
- Rename labels
- ~Remove "Actions" label (since there's only a single action)~
## Summary
This PR enhances the multi-field configuration screen for creating an ML
inference pipeline. It adds different behavior to the target field based
on the selected model:
- For ELSER pipelines the target field is not editable, and the output
field names are automatically generated as `ml.inference.<source
field>_expanded`
- For non-ELSER pipelines the target field is editable if there's a
single source field selected. For multiple source fields the names are
automatically generated as `ml.inference.<source field>`
The mapping auto-updater process now receives `model_id` and only
changes the mapping for ELSER pipelines.
In order to keep the scope of changes smaller, we're NOT switching over
to the multi-field selector for non-ELSER pipelines just yet. We're
making the logic ready for this here, but the actual switchover will
happen in a following PR.
Non-ELSER pipelines
ELSER pipelines
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/159382.
### Changes
- `kibanaVersion` is passed as a resources to the routes.
- `elastic-agent` version is based on `kibanaVersion`.
#### Before
<img width="1073" alt="image"
src="00e291a2-3f8d-4d1d-9520-c0a29cc9454e">
#### After
<img width="1115" alt="image"
src="dffcf95f-f3a2-46f0-bb78-daebe8597b89">
Before:
If agent.type !== endpoint, we were disabling the Respond button in the
Take Action dropdown on the Alert Detail page. However, an alert coming
from, for example, filebeats, could be originating from an agent with
Defend integration, which supports respond actions.
After:
I removed the check for agent.type, resulting in the API
/endpoint/metadata/:id being called in all cases. If the API returns a
200 status code, we display the respond button since the endpoint
supports the response console. If the API returns a 404 status code, we
display a tooltip saying 'Please add Defend integration' and keep the
button disabled.`
Take Action button
Timelines
## Summary
Bumps node.js to 18.17.0 (replacement for PR #144012 which was later
reverted)
As a result, these categorical additions were needed:
- `node` evocations will need the `--openssl-legacy-provider` flag,
wherever it would use certain crypto functionalities
- tests required updating of the expected HTTPS Agent call arguments,
`noDelay` seems to be a default
- `window.[NAME]` fields cannot be written directly
- some stricter typechecks
This is using our in-house built node.js 18 versions through the URLs
the proxy-cache. (built with
https://github.com/elastic/kibana-custom-nodejs-builds/pull/4)
These urls are served from a bucket, where the RHEL7/Centos7 compatible
node distributables are. (see:
https://github.com/elastic/kibana-ci-proxy-cache/pull/7)
Further todos:
- [x] check docs wording and consistency
- [ ] update the dependency report
- [x] explain custom builds in documentation
- [x] node_sass prebuilts
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Thomas Watson <w@tson.dk>
Resolves https://github.com/elastic/kibana/issues/162393
Adds a new feature flag
`xpack.observability.unsafe.alertDetails.observability.enabled` to
show/hide threshold alert details page until it is ready for GA.
Fix https://github.com/elastic/kibana/issues/160974
Fix https://github.com/elastic/kibana/issues/160929
## Summary
From Es 8.10, the `version` field from the Get Snapshot response has a
different format - it will no longer be a string representing the
release version, but rather it will be a string representing the version
id (see
https://github.com/elastic/kibana/issues/160974#issuecomment-1618087194
for more details).
This PR replaces the use of the `version` field with the `versionId`
field when checking in the Snapshot Restore form if the current version
is greater than 7.12 (which corresponds to `versionID` 7120099).
The PR also unskips the functional test that failed with the promotion
of Es to 8.10.
**How to test:**
1. Start Es with `yarn es snapshot -E path.repo=./tmp/snap` and Kibana
with `yarn start`
2. Go to Stack Management -> Snapshot and Restore
3. Create a Shared File System repository with location `./tmp/snap`
4. Create a policy that uses the repository created in the previous
step.
5. Run the policy to create a snapshot.
6. Go to the Snapshots tab and click the Restore button for the created
snapshot.
7. Verify that the Snapshot restore form works as expected.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Fixes#152206
## Summary
I removed the reporter tests from
`x-pack/plugins/cases/public/components/case_view/components/case_view_activity.test.tsx`
There were a few things being tested:
| Old Test | Where is it covered |
| ------------- | ------------- |
| does the case view render `'case-view-user-list-reporter'` | [a
functional test already covers
this](https://github.com/elastic/kibana/blob/main/x-pack/test/functional_with_es_ssl/apps/cases/group1/view_case.ts#L47)
|
| Is a reporter name displayed correctly | covered in the UserList
component tests |
| a reporter without uid is rendered correctly | moved this logic to
`parseCaseUsers` and tested it there in this PR |
| fallbacks correctly to the caseData.createdBy user correctly | moved
this logic to `parseCaseUsers` and tested it there in this PR |
- Originally Kibana's `http` service did not support receiving streams,
that's why we used plain `fetch` for this. This has been fixed in
#158678, so this PR updates the streaming helpers to use Kibana's `http`
service from now on.
- The PR also breaks out the response stream code into its own package
and restructures it to separate client and server side code. This brings
down the `aiops` bundle size by `~300KB`! 🥳
- The approach to client side throttling/buffering was also revamped:
There was an issue doing the throttling inside the generator function,
it always waited for the timeout. The buffering is now removed from
`fetchStream`, instead `useThrottle` from `react-use` is used on the
reduced `data` in `useFetchStream`. Loading log rate analysis results
got a lot snappier with this update!
## Summary
Part of the CSP teams quick wins effort, this PR aims to improve the
experience for users first landing on the Findings page, or returning to
it.
Currently we always set the "Misconfigurations" tab as default
regardless of whether there are misconfigs or not. Even if there are
vulnerabilities (the primary tab), it will still default to misconfigs.
A small component was created to handle the root route for 'findings'
and decide which Tab to be redirected to. Vulnerabilities or
Misconfigurations. If a user has never made a tab selection, there are
no vulnerabilities, but there are findings, the user will be redirected
to the "Misconfigurations" tab. If the user had previously clicked to
select a tab, it will always remember which tab they came from last.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
