closes#113998
In order to make FTR functional tests faster and less flaky, we started
migrating time picker selection from UI action to Kibana API call:
```
await this.kibanaServer.uiSettings.update({
'timepicker:timeDefaults': `{ "from": <startTime>, "to": <endTime>}`,
});
```
In this PR I updated most of the tests listed in the meta issue, so we
can close it.
Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2606
## Summary
This PR add a new indicator to support histogram fields. This will allow
you to either use a `range` aggregation or `value_count` aggregation for
the good and total events; including support for filtering with KQL on
both event types. When using a `range` aggregation, both the `from` and
`to` thresholds are required for the range and events will be to total
number of events within that range.[ Keep in mind, with the `range`
aggregation, the range includes the `from` value and excludes the `to`
value.](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-range-aggregation.html)
This PR also includes support for using the histogram field for a
"Custom Metric" indicator, `sum` is calculated on the values and not the
counts. If you need it calculated on the counts then you have to use the
histogram indicator.
<img width="776" alt="image"
src="1d46b722-df13-417e-bf3b-b3c450933da2">
---------
Co-authored-by: Kevin Delemme <kdelemme@gmail.com>
Resolves: #155766Resolves: #159302
With this PR we aim to skip a task that has invalid direct and indirect
params.
In order to do that,
1- We validate the task params before calling the subtask's run method
and skip if if the task params are invalid.
2- We skip execution of a subtask (rule, action etc) when the run method
of it returns a `SkipError`
Therefore, validations in the run methods needs to be moved to top of
the run method and executed before anything else to return skip if the
data is invalid.
We also added a config to enable/disable the skip feature, and define
the delay duration of task reschedule.
As this may become an infinitive loop, we are supposed to limit the
attempts.
Follow on issue to implement that: #159302
---------
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Updated the Kibana collapsible nav Enterprise Search category to Search.
Updated the App Search & Workplace Search app navLinkStatuses to
`hidden` to remove them from the Kibana nav.
### Screenshots
<img width="270" alt="image"
src="1397e701-ca87-46f2-8c15-565bc3a9202c">
## Summary
Handles elastic/security-team#6971
This PR mainly resolved below 3 issues:
### Rename to `Add To Timeline` control in conversation code blocks to
`Investigate in Timeline`
- `Add to Timeline` according to existing Security Solution actions
means, adding a condition to the timeline with an `OR` clause without
affecting the existing Timeline.
- But the `Add to Timeline` control in the Security Assistant, creates a
new timeline on each action by the user, which contradicts the above
workflow. Hence, it might confuse user.
- `Investigate in Timeline` already means that a new timeline will be
created.
### `Add To Timeline` control was visible on types of codeblock. For
example, it does not make sense for a `Query DSL` to have an `Add to
Timeline` control.
- This PR adds the list of eligible types of queries/code blocks on
which `Add To Timeline` action can be added.
- Currently, that list only contains `kql`, `dsl` and `eql`. Below is
the complete list of types of query that can occur in code blocks.
- Please feel free to suggest a change.
```
'eql' | 'kql' | 'dsl' | 'json' | 'no-type';
```
### Lazy calculation of CodeBlockPortals and CodeBlock Action container
- To add controls to the conversation code blocks, we need to follow
below 2 steps.
1. get the codeBlock containers on which the controls can be added.
2. create portals in the HTML container with our `Add to Timeline`
control.
- Below are issues these steps sometime created.
1. We get codeBlock container in the `useLayoutEffect` but at the time,
all conversations might not have loaded because of which containers are
returns as the undefined.
2. Then, we try to create portal in the `undefined` container, which
fails and hence, `Add to Timeline` controls are not visible.
- Solution:
1. Instead of getting the codeblock container in useLayoutEffect, we get
the function which will eventually return that container, whenever we
are creating the portal.
2. Converted codeBlock Portal to a callback such that callback can be
called during the rendering which makes sure that all needed
conversations are available and using above step we can easily get the
portal containers.
Feel free to let me know if there are any issues with above strategy.
### Better Pattern matching.
- Currently, when we are trying to identify the type of codeblock it
might result in unexpected output because of below reason.
1. Let say, we are trying to identify KQL Query and for that we use
below phrases to match in the `OpenAI` response.
`'Kibana Query Language', 'KQL Query'`
2. Because of this, if the `OpenAI` response contains the phrase `KQL
query` or `kql query`, that fails because of case senstivity when
searching the above phrases.
3. This PR makes that part of pattern matching case insensitive
### Before
b472178a-0145-42d8-8fb9-ab107915086a
### After
b499f099-a7a1-435f-99b2-ab27ee1f5680
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
## Summary
Closes https://github.com/elastic/kibana/issues/157203
Closes https://github.com/elastic/kibana/issues/158051
Closes https://github.com/elastic/kibana/issues/158049
With the addition of Per User Dark Mode, components can no longer rely
on `uiSettings` as the source of truth for theme.
CodeEditor fields used to call `uiSettings` to determine if Dark Mode
was enabled, which had been provided in each callers Kibana React
Context.
The new source of truth for theme is the `CoreStart >
ThemeServiceStart`.
Currently, not all callers of CodeEditor provide the `theme` service in
their Kibana Context in a similar way and some callers don't provide it
at all.
This PR updates CodeEditor to get theme values from the Kibana Context
using a new `useKibanaTheme` react hook.
It also attempts audit the callers of CodeEditor to see if their Kibana
Context contains the theme service at the top level (Where I could add
theme to a caller's Context without major changes, I did. Some cases
will require CodeOwner guidance).
The new `useKibanaTheme` react hook will throw a TypeError if theme
isn't found in the top level of the Kibana Context, this will help with
testing as the component will not render. I will remove this after
testing so as not to introduce breaking changes.
## Testing
Please review files for which you are CODEOWNER.
I've attempted to tag all usages of `CodeEditor`/`CodeEditorFIeld` with
a TODO comment with one of the following scenarios:
1) a note where theme was provided already/where I made changes to
provide it in the appropriate context
2) I've asked for CODEOWNER guidance
For scenario 1, please pull and test that CodeEditor locally:
1. Enable Dark Mode from Edit User Profiles by clicking on the Profile
Icon on the top right, and updating your profile.
2. Navigate to the CodeEditors in the plugins you own. If they render
and display in Dark Mode - add a green check to the table below - and
you're done!
3. If it is not rendering, please help me figure out where the theme
service should be provided in the context.
For scenario 2, we will need to figure out where to make changes so your
context is providing theme. Some of the more complex usages may need to
addressed in separate issues.
## Tracking
| Team | Plugin | Theme in Context ? | Verified Working |
| - | - | - | - |
| apm-ui | apm | APM Storybook broken | ? |
| kibana-presentation | presentation_util | Yes. | Yes |
| response-ops | trigger_actions_ui | Yes | Yes |
| response-ops | stack_alerts | Yes | Yes |
| kibana-security | security | Yes | Yes |
| security-defend-workflows | osquery | Yes | Yes |
| kibana-app-services | examples/expression_explorer | Yes | Yes |
| ml-ui | transform | Yes | Yes |
| ml-ui | ml | Yes | Yes |
| uptime | synthetics | Yes | Yes |
| kibana-gis | maps | Yes | Yes |
| kibana-gis | file_upload | Yes | Yes |
| platform-deployment-management | watcher | Yes | [AG] Yes |
| platform-deployment-management | snapshot_restore | Yes | [AG] Yes |
| platform-deployment-management | runtime_fields | Yes | [AG] Yes |
| platform-deployment-management | painless_lab | Yes | [AG] Yes |
| platform-deployment-management | ingest_pipelines | Yes | [AG] Yes |
| platform-deployment-management | index_management | Yes | [AG] Yes |
| platform-deployment-management | grokdebugger | Yes | [AG] Yes |
| platform-deployment-management | es_ui_shared | Yes | [AG] Yes |
| fleet | fleet | Yes | Yes |
| enterprise-search-frontend | enterprise_search | Yes | [AG] Yes |
| kibana-cloud-security-posture | cloud-security-posture | Yes | yes |
| sec-cloudnative-integrations | cloud_defend | Yes | Yes |
| kibana-visualizations/kibana-data-discovery | data | Yes | Yes |
| kibana-visualizations | examples/testing_embedded_lens | Yes | Yes |
| kibana-visualizations | vis_types | Yes | Yes |
| kibana-visualizations | vis_default_editor | Yes | Yes |
| kibana-visualizations | unified_search | Yes | Yes |
| kibana-visualizations | packages/kbn-text-based-editor | Yes | Yes |
| kibana-visualizatons | lens | Yes | Yes|
| kibana-core | saved_objects_management | Yes | Yes |
| kibana-presentation | inspector | Yes | Yes |
| kibana-presentation | canvas | Yes | Yes |
| kibana-data-discovery | discover | Yes | Yes |
| kibana-data-discovery | data_view_management | Yes | Yes |
| kibana-data-discovery | data_view_field_editor | Yes | Yes |
| appex-sharedux | advanced_settings | Yes | Yes |
| enterprise-search-frontend | serverless_search | Yes | [AG] Yes |
| - | - | - | - |
## Unit tests
Currently, many tests are failing since they are probably not providing
`theme` in the context. Once CODEOWNERs have weighed in on CodeEditors
usages that require discussion, I will update the accompanying tests.
## Release note
- Fixes theming of CodeEditors
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
Co-authored-by: Dima Arnautov <dmitrii.arnautov@elastic.co>
Co-authored-by: Dima Arnautov <arnautov.dima@gmail.com>
## Summary
Remove allow slack channels feature for 8.9 until we have a better way
to deal with channels
### Checklist
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Resolves https://github.com/elastic/kibana/issues/157258
## Summary
Gets the runtime_mappings from the es query, and includes them in the
query to retrieve the fields.
Also gets runtime mappings from data views.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To verify
- Create a new Es Query rule using DSL
- Include runtime mappings in your query and verify that the runtime
fields are listed in the group by or aggregate over options below the
query
- Create a dataview and include runtime fields
- Create a new Es Query rule using KQL
- Verify that the runtime fields are listed in the group by or aggregate
over options below the query
## Summary
Currently user can add a note in timeline with invalid markdown syntax
in insight forms. Same goes to the investigation guide in rule creation
-> About.
### Before
**In timeline -> Notes**
**On rules -> about**
### After
**Timeline -> Notes**
Add note button should be disabled if markdown is invalid
**On rules -> about**
Field is red if markdown is invalid
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Original ticket: https://github.com/elastic/kibana/issues/156706
These changes prevent user from creating/updating the rule when alert
filter is selected and query left blank on the rule's action page. We
gonna show an error saying "A custom query is required." in this case.
<img width="1739" alt="Screenshot 2023-06-14 at 14 36 35"
src="0456f211-603c-44d9-9271-9cfdf59f12b6">
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Filter out pipelines from the existing pipeline selection dropdown that
are already attached to the current index. This also updates the
evaluation of the "Existing pipeline" option in the configuration panel.
Example: 2 pipelines that are already attached are filtered from the
dropdown; when attaching the 3rd one, the existing pipelines option is
disabled.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Fixes the following issues:
1. Suggestions are not reachable by keyboard navigation (it was actually
broken when fixing the [accesibility axe
warning](https://github.com/elastic/kibana/issues/154787), I ran it now
and it doesn't show now)
46c9eb70-4df7-462b-a878-b7d190aaf8c9
2. Dimension removal button is not reachable by keyboard navigation
before:
5011679b-b05a-48ca-a877-bbc940607b56
after:
9d056512-a622-4c1c-aa5b-242e2762e9ae
3. The padding for the reordering groups got a distance between the
empty dimension button (btw I have more little fixes for drag and drop
look but want to submit it in a separate PR to not ping here Discover
team):
<img width="421" alt="Screenshot 2023-07-03 at 17 42 41"
src="a1d22267-07b6-4179-89f9-b4c7ec5da030">
<img width="459" alt="Screenshot 2023-07-03 at 17 39 11"
src="a7b697ca-4ff3-4c18-bf96-a8710615a9df">
4. The focus outline for dataview selector and layer chart selector got
a nicer rectangular shape
before:
<img width="343" alt="Screenshot 2023-07-03 at 10 50 52"
src="0ca305b8-cdbf-4c43-b73d-0058046ab919">
<img width="360" alt="Screenshot 2023-07-03 at 17 41 18"
src="54e61646-5d4e-4512-b6e1-6d758e2b7498">
after:
<img width="350" alt="Screenshot 2023-07-03 at 11 11 01"
src="8ddd81dd-8072-4cf8-a98b-ae5eb9998add">
<img width="415" alt="Screenshot 2023-07-03 at 17 39 51"
src="1c36b3bd-cb15-4cb8-858e-92fc19e3181f">
5. The focus outline was corrected for dimension buttons. I also
refactored code here to make margins, paddings and gaps the property of
the containers and not the items themselves. I think this way it's more
correct.
before:
<img width="418" alt="Screenshot 2023-07-03 at 17 40 54"
src="b8ce4a22-dc69-44a7-8689-70928340ac24">
after:
712e2ac8-f81d-4896-b384-164fc2854766
## Summary
Unskip test that was timing out at indexing test hosts. The timeout has
since been increased in elastic/kibana/pull/159518
It was added in elastic/kibana/pull/157777 and skipped in
elastic/kibana/pull/156933 thus should be backported to `8.8.1` and
`8.9.0`
## Summary
Previously the data plugin relied on the data view plugin to load saved
searches since the saved searches depend upon the data plugin and
circular dependencies needed to be avoided. This is innovative and
perhaps a bit crazy.
What this PR does
- Data view api no longer loads saved searches, removing browser saved
object client usage
- Moves `kibana_context` expression and getKibanaContext function from
data plugin to saved search plugin since it loads saved searches
- Rename data views `SavedObjectsClientCommon` to `PersistenceAPI` -
this is the abstraction around saved object loading that no longer is
exclusive to the saved objects api.
- Adds saved search server api (plugin contract) for loading saved
searches.
- Functional tests on browser and server for kibana_context expression
when loading saved searches
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The NativeRenderer component is currently used to mount another
component in a separate mounting point. As far as I recall, we
introduced <NativeRenderer/> to allow users to create visualizations in
non-React frameworks. The idea was that users could write their own Lens
visualizations or datasources code and integrate it with our system.
However, it seems that this concept hasn't gained traction and we don’t
have it prioritized. Even if users express interest in writing their
visualizations outside of React, it is still possible to do so with some
additional boilerplate code (which we could provide as an example
non-React visualization).
Pros:
1. Simplifies and shortens the code:
1.1. Testing and debugging become easier as we no longer need to check
separate React trees when integrating frame, data source, and
visualization components.
1.2. Components communicate using standard React patterns, making
maintenance and comprehension simpler.
1.3. Context providers no longer need to be passed to each separate
component since they are already within the context.
1.4. Easier propagation of events or any other form of inter-component
communication.
2. Greatly improves performance and facilitates maintenance:
2.1. Directly accessing context inside the DatasourcePanel eliminates
the need for context passing, resulting in better performance.
2.2. Removing the requirement for a separate React root also contributes
to improved performance.
3. The render method will be removed when we upgrade to React 18. While
we could replace it with the new createRoot method, it makes sense to
perform some cleanup ;)
Cons:
1. Setting up non-React visualization or data source code might become
slightly more complex.
Performance improvement for drag and drop action with these changes:
before:
<img width="1110" alt="Screenshot 2023-07-10 at 07 14 39"
src="45a1b09b-5189-46f5-af2b-7781fcf4e774">
after:
<img width="1117" alt="Screenshot 2023-07-10 at 07 16 24"
src="0e704da1-3220-4eb9-8fa0-cc3584a90090">
## Single render when dragging:
(the first image is 3 screenshots from 3 different react roots as they
have separate mounting point. The complete render time is ~380ms)
<img width="1117" alt="Screenshot 2023-07-10 at 07 16 24"
src="6d7f2d9f-a758-476e-8efb-38693ae90097">
After we have one common render tree. Because we don't have to pass
context down as a prop, we greatly reduced the number of components
rerendered. (I will be working on reducing the render time for workspace
panel as this seems to still be a bottleneck point)
<img width="732" alt="Screenshot 2023-07-10 at 14 52 41"
src="03ec97b3-8225-490e-8884-0fd4e69587e8">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Unskip test that was timing out at indexing test hosts.
The timeout has since been increased in elastic/kibana/pull/159518
Was added in elastic/kibana/pull/154505 and skipped in
elastic/kibana/pull/156104
## Summary
Part of https://github.com/elastic/kibana/issues/158802
We decided that in the case we don't render a Lens suggestion (for
example when we have the SELECT * case) to render the histogram.
**Reminder**: Histogram makes sense only of there is a time field. For
text based mode, time field exists **ONLY** if there is the @timestamp
field.
I don't allow navigation to Lens or open the edit flyout in this case.
When the edit flyout allows the editing of the form based visualizations
(such as histogram) I will enable it then.
FTs have been changed to accomodate this change.
<img width="1751" alt="image"
src="3d28d881-bc60-43de-acf8-8cbcd172a3df">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
When I created drag and drop for Lens, the API I went for was not the
most readable one. It was designed this way because I wanted to gain
some performance, but it was very hard to maintain the performance gain
with a lot of changes in the drag and drop area because all the pieces
of the code needed to memoized in a tricky way and it wasn't
communicated well.
In the end it works even without these tricks so I decided to simplify
it in this PR.
The main changes include:
1. Instead of multiple `useState` per parameter, we keep all the state
in reducer both for `ReorderProvider` and `RootDragDropProvider`. Thanks
to that we get multiple improvements:
2. The code in `DragDrop` component becomes more descriptive as we don't
run multiple state updates when user executes an action but one state
update describing what actually happens (eg. `dispatchDnd({type:
'selectDropTarget' ....})`. The internal logic of the update lives in
the reducer.
3. We don't have to pass `trackUiCounterEvents` as another prop to
`DragDrop` and run it wherever we need - instead we pass it as a
middleware to the context and run before dispatching (and it's very easy
to add more middlewares if we need extra integrations at some point!)
4. We also run a11y announcements as a middleware instead of inside
`DragDrop` component
5. The `ChildDragDropProvider` props look much cleaner:
before:
```
<ChildDragDropProvider
keyboardMode={keyboardModeState}
setKeyboardMode={setKeyboardModeState}
dragging={draggingState.dragging}
setA11yMessage={setA11yMessage}
setDragging={setDragging}
activeDropTarget={activeDropTargetState}
setActiveDropTarget={setActiveDropTarget}
registerDropTarget={registerDropTarget}
dropTargetsByOrder={dropTargetsByOrderState}
dataTestSubjPrefix={dataTestSubj}
onTrackUICounterEvent={onTrackUICounterEvent}
>
{children}
</ChildDragDropProvider>
```
after:
```
<ChildDragDropProvider value={[state, dispatch]}>{children}</ChildDragDropProvider>
```
6. Created custom hook `useDragDropContext` instead of using
`useContext(DragContext)` and making DragContext private. This way we
will avoid potential problems with using context outside of root.
7. Bonus thing - if we ever decide to move to redux, the structure is
there already
What I am still not happy with is that the tests are very
domain-dependant instead of user-driven - instead of checking the store
actions, I should check the interface from the user perspective. I will
try to work on it once I find some time between more important tasks
though.
## Summary
Run Real Endpoint Cypress E2E on CI using Vagrant
---------
Co-authored-by: Tomasz Ciecierski <ciecierskitomek@gmail.com>
Co-authored-by: Ashokaditya <am.struktr@gmail.com>
**Related to:** https://github.com/elastic/security-team/issues/6867
(internal)
## Summary
As requested in
https://github.com/elastic/kibana/pull/160685#issuecomment-1621635262.
In test plans, we mention how scenarios are going to be automated --
whether a given scenario will be automated using Cypress for e2e tests,
Jest for integration tests and unit tests, etc. But currently, test
plans are under `x-pack/plugins/security_solution/cypress/`.
Since test plans are in fact technology agnostic, it does not make much
sense to keep them inside a specific framework technology folder. That's
why we're moving them to a generic
`x-pack/plugins/security_solution/docs/testing` folder.
Fixes#155737. This passed 350 flaky test runner runs, reenabling. These API
integration tests were missing from CODEOWNERS, adding that in this PR
too (`/x-pack/test/api_integration/apis/aiops/ @elastic/ml-ui`).
**Resolves: https://github.com/elastic/kibana/issues/138903**
## Summary
Adds an E2E Cypress test to check filtering by execution status in the
rules table.
<img width="953" alt="Screenshot 2023-06-26 at 14 10 10"
src="e1eb67ed-779c-42ad-8194-04a26598cfbc">
Created a separated PR in order setup a basic setup for cypress and test
https://github.com/elastic/kibana/pull/160620 for serverless.
Basic setup to run cypress for serverless-oblt
#### How to run it
from
`x-pack/test_serverless/functional/test_suites/observability/cypress`
```
yarn cypress:serverless:open
```
