Commit graph

1058 commits

Author SHA1 Message Date
Thomas Watson
531acda76c
GitHub create-deploy-tag workflow: Allow kibana-operations team to trigger (#165630) 2023-09-05 11:17:23 +02:00
Yuliia Naumenko
8fb5a651a0
Extracted DiscoverGrid to a package named @kbn/unified-data-table as UnifiedDataTable component (#163211)
## Summary

Current PR includes the next set of changes:

1. Moved `DiscoverGrid` component to a package `@kbn/unified-data-table`
and added `@elastic/kibana-data-discovery` as code owners.
2. Changed `@kbn/unified-data-table` package naming for data grid
related components and methods to correspond `UnifiedDataTable` instead
of `Discover`.

3. Moved hooks `useColumns` and `useRowHeightsOptions` to a package as
its logic belongs to `UnifiedDataTable`.
4. Renamed `DiscoverGridContext` to `UnifiedDataTableContext`.
5. Extended `UnifiedDataTable` interface and functionality with the next
customization options:
- `renderDocumentView?: (displayedRows:
DataTableRecord[],displayedColumns: string[]) => JSX.Element |
undefined;` - callback to render DocumentView when the document is
expanded
- `configRowHeight?: number;` - optional value for providing
configuration setting for UnifiedDataTable rows height
- `showMultiFields?: boolean;` - optional value for providing
configuration setting for enabling to display the complex fields in the
table. Default is true.
- `maxDocFieldsDisplayed?: number;` - optional value for providing
configuration setting for maximum number of document fields to display
in the table. Default is 50.
- `externalControlColumns?: EuiDataGridControlColumn[];` - optional
value for providing EuiDataGridControlColumn list of the additional
leading control columns. UnifiedDataTable includes two control columns:
Open Details and Select.
<img width="522" alt="Screenshot 2023-08-22 at 2 26 57 PM"
src="d796b9c8-2fef-4bcc-a3c9-9f5cc6349ab9">

- `externalAdditionalControls?: React.ReactNode;` - optional value for
providing the additional controls available in the UnifiedDataTable
toolbar to manage it's records or state. UnifiedDataTable includes
Columns, Sorting and Bulk Actions.
<img width="673" alt="Screenshot 2023-08-22 at 2 40 28 PM"
src="f7ac0c87-5310-49dd-9084-1ce01ca0f366">

- `rowsPerPageOptions?: number[];` - optional list of number type values
to set custom UnifiedDataTable paging options to display the records per
page.
- `renderCustomGridBody?: (args: EuiDataGridCustomBodyProps) =>
React.ReactNode;` - An optional function called to completely customize
and control the rendering of EuiDataGrid's body and cell placement.
<img width="1658" alt="Screenshot 2023-08-22 at 2 50 27 PM"
src="14adc18d-73af-40f5-9859-b3c708e265b1">

- `trailingControlColumns?: EuiDataGridControlColumn[];` - optional list
of the `EuiDataGridControlColumn` type for setting trailing control
columns standard for `EuiDataGrid`.
- `visibleCellActions?: number;` - optional value for a custom number of
the visible cell actions in the table
<img width="497" alt="Screenshot 2023-08-22 at 2 45 49 PM"
src="57ef3ad9-7401-46bb-9b38-cc8cca2e6a24">

- `externalCustomRenderers?: Record<string,(props:
EuiDataGridCellValueElementProps) => React.ReactNode>;` - an optional
settings for a specified fields rendering like links. Applied only for
the listed fields rendering:
<img width="1121" alt="Screenshot 2023-08-22 at 2 51 07 PM"
src="77501eae-3046-4a2c-90e1-2db487c21e2c">

- `consumer` - optional string value for the name of the
`UnifiedDataTable` consumer component or application.
6. Extended `UnifiedDataGrid` services with the two additional: 
    `storage: Storage;`
    `data: DataPublicPluginStart; `
replaced `core: CoreStart;` with `theme: ThemeServiceStart;`, because
`core` is used only to get `theme`
7. Replaced `DocumentView` property with `renderDocumentView?:
(displayedRows: DataTableRecord[],displayedColumns: string[]) =>
JSX.Element | undefined;` callback function, which allows not to use
`DiscoverGridFlyout` definition for the documents rendering.
```
    /**
   * Document detail view component
   */
  DocumentView?: typeof DiscoverGridFlyout;
```
8. Removed the next properties from the data table interface, because it
was used to render DiscoverGridFlyout:
```
   /**
   * Filters applied by saved search embeddable
   */
  filters?: Filter[];
  /**
   * Query applied by KQL bar or text based editor
   */
  query?: Query | AggregateQuery;
  /**
   * Saved search id used for links to single doc and surrounding docs in the flyout
   */
  savedSearchId?: string;
```
9. Added usage examples and interface description to README file.
10. Changed grid styles naming from `.dscDiscoverGrid*` to
`.udtDataTable*`
11. Migrated discover plugin to use `UnifiedDataTable` instead of
`DiscoverGrid`

Extra changes were needed to avoid the circular dependancies:
- moved `DocViewFilterFn` and `FieldMapping` from discover plugin to
`packages/kbn-discover-utils/src/types.ts`
- added own `export type SortOrder = [string, string];` to avoid deps
for saved-search plugin

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-09-01 23:02:53 -07:00
Thomas Watson
9a549a2fff
GitHub create-deploy-tag workflow: Show commit SHA (#165460) 2023-09-01 20:18:06 +02:00
Cauê Marcondes
acf89562c4
[Profiling] Profiling data access plugin (#165198)
This is part 1 of a series of PRs to expose the flamegraph to be used by
other plugins.

**The problem**
Currently for plugin-A to show data from plugin-B, it needs to add
dependency on plugin-B. If plugin-B wants to show data from plugin-A, it
also needs to add plugin-A as a dependency, and here is where the
problem happens. In such scenario, we have a cyclic dependency problem.

**The solution**
To solve this problem a new plugin is created, `profiling-data-access`.
This plugin exposes services that consumer plugins can call in other to
have the data they need to show on their end. The `profiling` plugin is
also using this new plugin. For now, only the flamegraph service is
available, The idea is to slowly migrate the data fetching from
profiling to this new plugin in other to facilitate the integration
across plugins.

**Why some many files?**
As I said, only the flamegraph logic was moved to the new plugin, but it
has many files that it needs to properly build the response of the
service call. I moved all these files to the `common` folder inside the
new plugin and adjusted the imports in the profiling plugin.

<img width="1032" alt="Screenshot 2023-08-31 at 09 29 27"
src="287bd28e-b834-45e0-8042-576d1fcff6cd">

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-09-01 09:09:41 +01:00
Lukas Olson
09cd69d386
[UnifiedDocViewer] Move Discover doc viewer into plugin/package (#162847)
## Summary

Replaces https://github.com/elastic/kibana/pull/154012.

Moves the Discover doc viewer component into a new plugin/package,
`@kbn/unified-doc-viewer` and `@kbn/unified-doc-viewer-plugin`.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### Risk Matrix

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Shahzad <shahzad31comp@gmail.com>
2023-08-31 11:46:59 -07:00
Davis McPhee
b4bfb2a2dd
[Data Discovery] Run example plugin functional tests in Serverless (#163411)
## Summary

> [!IMPORTANT]  
> These tests currently do not run in "real" Serverless and only run in
"local" Serverless (what we use in CI). Additional work will need to be
done to allow these tests to be run in an actual Serverless environment,
since they required `config.ts` changes which are only loaded when
running locally.

This PR copies the Data Discovery example plugin functional tests to
`test_serverless` and adds support for running them against the Search
project.

It also adds support for running functional tests against example
plugins in general in Serverless ("local" Serverless only currently).

In the future these should run as deployment-agnostic tests, but support
does not yet exist (see #161574), so in the meantime they've been
duplicated and modified in place. I've left `TODO` comments where test
files have been modified so we know what needs to be addressed once they
are converted to deployment-agnostic tests.

Part of #162347.

### Checklist

- [ ] ~Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)~
- [ ]
~[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials~
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] ~Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard
accessibility](https://webaim.org/techniques/keyboard/))~
- [ ] ~Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))~
- [ ] ~If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)~
- [ ] ~This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))~
- [ ] ~This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)~

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
2023-08-31 10:23:05 -03:00
Alejandro Fernández Haro
0ea37c1b42
[On-Week] Hot update of APM/EBT labels (#157093) 2023-08-31 14:36:20 +02:00
Felix Stürmer
ad59308b24
[Log Explorer] Convert log explorer profile into standalone app (#164493)
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani01@gmail.com>
2023-08-31 14:18:44 +02:00
Ash
6e367d94c9
[Serverless][Security Solution][Endpoint] Restrict endpoint exceptions on serverless via plugin sub-features (#164107)
### What this PR changes

branched from elastic/kibana/pull/163759

- Introduces new AppFeatures package `@kbn/security-solution-features`
with the common logic and `AppFeatureService` to apply offering specific
configurations for Security Solution features independently for
Serverless and ESS. This logic is replacing the earlier `AppFeatures` in
order to introduce new Kibana feature privileges for serverless PLIs so
that new Kibana privileges introduced for serverless PLIs do not
affect/show up as new Kibana feature privileges in ESS.
- Gates endpoint exceptions on alerts/rules based on serverless PLI
configurations. On serverless `Endpoint exceptions` should be
accessible/seen only on endpoint essentials/complete.

New AppFeatures logic architecture diagram:

![Security Solution Features
(Current)](f627406d-43bc-4db5-93b1-4e43eeb6d870)

**Note:** Corresponding API changes related to endpoint exceptions will
be in a new PR, along with the last set of UX changes for hiding the
`Endpoint exceptions` tab from the Rules details page.

### How to review

- Setup for _Servlerless_
  - Run `yarn es snapshot` on a terminal window to start ES.
- Copy `config/serverless.security.yml` to
`config/serverless.security.dev.yml`
- Run `yarn serverless-security --no-base-path` on another terminal
window to start kibana in serverless mode
- Run `node
x-pack/plugins/security_solution/scripts/endpoint/endpoint_agent_emulator.js
--asSuperuser` on a new window and then select `1` to `Load Endoints`
and then `1` to `Run` the loader script. This will load some fake
agents/alerts data to test with.

### Tests (Serverless)

- with 
`{ product_line: 'security', product_tier: 'essentials' }` or `{
product_line: 'security', product_tier: 'complete' }`
and
`{ product_line: 'endpoint', product_tier: 'essentials' }` or `{
product_line: 'endpoint', product_tier: 'complete' }`

1. Navigate to Rules>Shared exception lists via
`http://localhost:5601/app/security/exceptions`
2. Test that you can see `Endpoint Security Exception List` card on the
shared exception lists page.
3. Navigate to `Alerts` page via `app/security/alerts`, you should see
endpoint alerts. If not, then click on `Manage Rules` and then
disable/enable `Endpoint Security` rules. That should trigger alerts to
show up on the Alerts table.
4. Click on `View Details` button under `Actions` column. Once the
flyout is visible, click on `Take Action` and verify that `Add Endpoint
exception` is visible/enabled/clickable on the menu.
5. Click on `More actions` button under `Actions` column and verify that
`Add Endpoint exception` is visible/enabled/clickable on the menu.
6. Click on `Investigate in timeline` button under `Actions` column;
when the timeline view is visible and the alert item is displayed, click
on buttons mentioned in 4. and 5. above and verify the same.
7. Navigate to `Rules`>`DetectionRules`>`Endpoint Security` rule under
the `Rules` table. Select the `Alerts` tab.
8. Click and verify `View details`,`More actions` and `Investigate in
timeline` buttons same as in 4., 5., 6. above.
9. You should be able to see the `Endpoint exceptions` tab as well.
Click and verify that you can see the tab's content.

- with 
`{ product_line: 'security', product_tier: 'essentials' }` or `{
product_line: 'security', product_tier: 'complete' }`
1. Edit `config/serverless.security.dev.yml` so that `endpoint` product
line item is commented out.
2. Test that you can not see `Endpoint Security Exception List` card on
the shared exception lists page.
3. Items 4. 5. 6. as above but the menu items should be disabled. This
can be verified with fake data only as with a real endpoint, endpoint
alerts are actually not visible at all.


### Tests (ESS)
On the ESS side, endpoint exceptions are not affected by this change and
work as usual based on index privileges.

---------

Co-authored-by: semd <sergi.massaneda@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: YulNaumenko <jo.naumenko@gmail.com>
Co-authored-by: Pablo Neves Machado <pablo.nevesmachado@elastic.co>
Co-authored-by: Pablo Machado <machadoum@gmail.com>
2023-08-31 13:35:42 +02:00
Thomas Watson
672f38f9be
GitHub create-deploy-tag workflow: Improve Slack message 5 2023-08-31 12:15:49 +02:00
Thomas Watson
d45347cf68
GitHub create-deploy-tag workflow: Improve Slack message 4 2023-08-31 10:32:55 +02:00
Thomas Watson
d24a65ef94
GitHub create-deploy-tag workflow: Improve Slack message 3 2023-08-31 09:39:49 +02:00
Thomas Watson
84b038b2df
GitHub create-deploy-tag workflow: Improve Slack message 2 2023-08-31 09:35:10 +02:00
Thomas Watson
235b272a88
GitHub create-deploy-tag workflow: Improve Slack message 2023-08-31 09:21:13 +02:00
Thomas Watson
a6aa865dee
GitHub create-deploy-tag workflow: Fix protected tag issue 2 2023-08-30 23:49:41 +02:00
Thomas Watson
350cd55823
GitHub create-deploy-tag workflow: Fix protected tag issue 2023-08-30 23:39:01 +02:00
Thomas Watson
927cb221ad
GitHub create-deploy-tag workflow: Fix slack posting bug 2 2023-08-30 23:19:50 +02:00
Thomas Watson
fbbdef8d05
GitHub create-deploy-tag workflow: Fix slack posting bug 2023-08-30 22:56:47 +02:00
Thomas Watson
681c2e9cf7
Trying to fix slack posting bug (#165272) 2023-08-30 22:27:54 +02:00
Thomas Watson
991d2cd552
Trying to fix slack posting bug (#165271) 2023-08-30 22:18:55 +02:00
Thomas Watson
974b492ab0
Fix create-deploy-tag workflow with correct Slack channel-id (#165268) 2023-08-30 21:55:23 +02:00
Thomas Watson
cf16ebd3a0
Improve create-deploy-tag workflow (#165244) 2023-08-30 20:24:45 +02:00
Thomas Watson
3128c46c70
Add GitHub Action Workflow: create-deploy-tag (#165213) 2023-08-30 15:38:22 +02:00
Davis McPhee
c288c8b9bb
[Data Discovery] Run API integration tests in Serverless (#163550)
## Summary

This PR copies the Data Discovery API integration tests to
`test_serverless`. They are currently in the common folder and will run
once for each project type in CI, but we could instead move them to a
specific project if we don't want to run them three times each.

In the future these should run as deployment-agnostic tests, but support
does not yet exist (see #161574), so in the meantime they've been
duplicated and modified in place. I've left `TODO` comments where test
files have been modified so we know what needs to be addressed once they
are converted to deployment-agnostic tests.

Part of #162347.

### Checklist

- [ ] ~Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)~
- [ ]
~[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials~
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] ~Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard
accessibility](https://webaim.org/techniques/keyboard/))~
- [ ] ~Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))~
- [ ] ~If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)~
- [ ] ~This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))~
- [ ] ~This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)~

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-29 10:22:48 -03:00
Andrew Macri
3935548f36
[Security Solution] [Elastic AI Assistant] LangChain integration (experimental) (#164908)
## [Security Solution] [Elastic AI Assistant] LangChain integration (experimental)

This PR integrates [LangChain](https://www.langchain.com/) with the [Elastic AI Assistant](https://www.elastic.co/blog/introducing-elastic-ai-assistant) as an experimental, alternative execution path.

### How it works

- There are virtually no client side changes to the assistant, apart from a new branch in `x-pack/packages/kbn-elastic-assistant/impl/assistant/api.tsx` that chooses a path based on the value of the `assistantLangChain` flag:

```typescript
    const path = assistantLangChain
      ? `/internal/elastic_assistant/actions/connector/${apiConfig?.connectorId}/_execute`
      : `/api/actions/connector/${apiConfig?.connectorId}/_execute`;
```

Execution of the LangChain chain happens server-side. The new route still executes the request via the `connectorId` in the route, but the connector won't execute the request exactly as it was sent by the client. Instead, the connector will execute one (or more) prompts that are generated by LangChain.

Requests routed to `/internal/elastic_assistant/actions/connector/${apiConfig?.connectorId}/_execute` will be processed by a new Kibana plugin located in:

```
x-pack/plugins/elastic_assistant
```

- Requests are processed in the `postActionsConnectorExecuteRoute` handler in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts`.

The `postActionsConnectorExecuteRoute` route handler:

1. Extracts the chat messages sent by the assistant
2. Converts the extracted messages to the format expected by LangChain
3. Passes the converted messages to `executeCustomLlmChain`

- The `executeCustomLlmChain` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts`:

1. Splits the messages into `pastMessages` and `latestMessage`, where the latter contains only the last message sent by the user
2. Wraps the conversation history in the `BufferMemory` LangChain abstraction
3. Executes the chain, kicking it off with `latestMessage`

```typescript
  const llm = new ActionsClientLlm({ actions, connectorId, request });

  const pastMessages = langchainMessages.slice(0, -1); // all but the last message
  const latestMessage = langchainMessages.slice(-1); // the last message

  const memory = new BufferMemory({
    chatHistory: new ChatMessageHistory(pastMessages),
  });

  const chain = new ConversationChain({ llm, memory });

  await chain.call({ input: latestMessage[0].content }); // kick off the chain with the last message
};
```

- When LangChain executes the chain, it will invoke `ActionsClientLlm`'s `_call` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/llm/actions_client_llm.ts` one or more times.

The `_call` function's signature is defined by LangChain:

```
async _call(prompt: string): Promise<string>
```

- The contents of `prompt` are completely determined by LangChain.
- The string returned by the promise is the "answer" from the LLM

The `ActionsClientLlm` extends LangChain's LLM interface:

```typescript
export class ActionsClientLlm extends LLM
```

This let's us do additional "work" in the `_call` function:

1. Create a new assistant message using the contents of the `prompt` (`string`) argument to `_call`
2. Create a request body in the format expected by the connector
3. Create an actions client from the authenticated request context
4. Execute the actions client with the request body
5. Save the raw response from the connector, because that's what the assistant expects
6. Return the result as a plain string, as per the contact of `_call`

## Desk testing

This experimental LangChain integration may NOT be enabled via a feature flag (yet).

Set

```typescript
assistantLangChain={true}
```

in `x-pack/plugins/security_solution/public/app/app.tsx` to enable this experimental feature in development environments.
2023-08-28 10:30:05 -06:00
Anton Dosov
243142d9c1
[Serverless Elasticsearch] Fix user is blocked from moving forward when opening Discover, Dashboard, or Visualize Library 2023-08-28 15:24:34 +01:00
Pablo Machado
75644797c3
Create upselling package and implement EntityAnalytics serverless upselling (#164136)
UX: https://github.com/elastic/security-team/issues/7310

## Summary

* It creates an Upselling package to share the service and components
between ESS and Serverless plugins
* It implements upselling for entity analytics on serverless by
replicating the ESS approach

ESS
<img width="1520" alt="Screenshot 2023-08-17 at 13 34 59"
src="95c2c94e-7ab3-4d9f-aa24-b3e9c00eb3ed">

Serverless
<img width="1523" alt="Screenshot 2023-08-17 at 13 39 25"
src="618ce9dc-ef4e-469d-884a-dfb09834d0b0">

We are not displaying the upgrade button because we still don't know how
to link to the cloud settings page.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-24 04:18:19 -07:00
Joey F. Poon
a3f6403033
[Security Solution] update codeowner for serverless security subdir (#164640) 2023-08-23 14:31:48 -07:00
Karl Godard
a95abde2be
[Cloud Security] CODEOWNERS update (#164522)
## Summary

This fixes some missed spots when I changed CODEOWNERS in this PR:
https://github.com/elastic/kibana/pull/164122

Forgot to update all the kibana.jsonc files, so CI overwrote my changes
in CODEOWNERS :)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-23 09:57:45 -07:00
Aleh Zasypkin
5aee5da843
Allow Kibana to restrict the usage of JWT for a predefined set of routes only. (#163806)
## Summary

Allow Kibana to restrict the usage of JWT for a predefined set of routes
only in Serverless environment by default. This capability is not
available in non-Serverless environment.

Any route that needs to be accessed in Serverless environemnt using JWT
as a means of authentication should include `security:acceptJWT` tag.

## How to test

If you'd like to generate your own JWT to test the PR, please follow the
steps outlined in
https://github.com/elastic/kibana/pull/159117#issue-1743796706 or just
run functional test server and use static JWT from the Serverless test.

This PR also generated a Serverless Docker image that you can use in
your Dev/QA MKI cluster.

- [x] Implementation functionality and add unit tests
- [x] Update metrics/status routes to include new `security:acceptJWT`
tag
- [x] Update serverless test suite to include a test for
`security:acceptJWT`

__Fixes: https://github.com/elastic/kibana/issues/162632__

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-23 14:57:32 +02:00
Gloria Hornero
10ab42692d
[Security Solution] Updating codeowners file (#164489) 2023-08-22 11:41:18 -07:00
Isaac Karrer
58334b54ac
Kibana QualityGate Scaffold (#163831)
The main question is here did I get the github team names correct for
fleet and security? Are there any other groups I am missing who will
have quality gates they want to see execute on every kibana release?

- The PR which registered the repo and controls who can write secrets to
vault was merged
[here](https://github.com/elastic/catalog-info/pull/488/files)
- Quality gate registration with argo deploy pipeline
[here](https://github.com/elastic/serverless-gitops/pull/586). Should
merge after this and pipelines are working.
- [this one](https://github.com/elastic/catalog-info/pull/485/files)
merges last and tells terrazzo about the catalog-info.yaml file with our
pipeline. Other pipelines should be migrated over but are not included
here.


Rel: https://github.com/elastic/ingest-dev/issues/2201
Rel: https://elasticco.atlassian.net/browse/QX-282

---------

Co-authored-by: Ramon Butter <ramon.butter@elastic.co>
2023-08-22 14:30:08 +02:00
Søren Louv-Jansen
7df1cee163
[APM] Add data access plugin (#162367)
Closes https://github.com/elastic/kibana/issues/161906
Related to:
https://github.com/elastic/observability-dev/discussions/2787
(_internal_)

This add a new plugin `apm_data_access` that contains the APM query
targets (indices to query for APM data).
This plugin can be consumed by apm and any other plugin, making it
possible for other plugins to know about the configured APM query
targets.



## Example:

APM query targets can be specified in kibana[.dev].yml using
`xpack.apm.indices.{dataset}: some-index-*` for instances:

```yml
xpack.apm.indices.transaction: apm-*
```

See all config options on:
https://www.elastic.co/guide/en/kibana/current/apm-settings-kb.html#general-apm-settings-kb

Query targets can also be specified via the UI (and persisted in a saved
object) via the settings page: `/app/apm/settings/apm-indices`

**Retrieving the query targets**
Query targets can be retrieved from other plugins via `getApmIndices`:
```ts
const apmIndices = await plugins.apmDataAccess.setup.getApmIndices(savedObjects.client); 
```

TODO:

- [x] Add SO client and fetch space aware index config (see
4d7f626da0/x-pack/plugins/apm/server/routes/settings/apm_indices/get_apm_indices.ts (L32-L44))
- [ ] Add simple APM client for querying apm data

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-22 02:19:09 -07:00
Georgii Gorbachev
8238900340
[Security Solution] Update CODEOWNERS for the Detection Engine team (#164359)
## Summary

This PR updates the COEOWNERS file by adding missing Cypress tests
folders owned by the @elastic/security-detection-engine team.
2023-08-21 22:47:54 +02:00
Karl Godard
4ce54648f8
[Cloud Security] removed beta label from tty setting (#164122)
## Summary

Removes a beta label from the "terminal output" feature in the linux
settings for the endpoint integration. The beta tag was previously
removed from the TTY player itself, but this spot was missed.

Also, this PR changes codeowners of session_view, kubernetes_security
and cloud_defend plugins to the **kibana-cloud-security-posture** team

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-17 10:15:32 -07:00
Drew Tate
276cf86858
[Event annotations] add types to integration tests (#164103) 2023-08-17 09:17:25 -05:00
Drew Tate
08a48f3cf1
[Lens] allow removing ad-hoc data view from event annotation group (#163976) 2023-08-16 10:24:15 -05:00
Georgii Gorbachev
a3e51bf3fe
[Security Solution] Fix CODEOWNERS for Detection and Response teams (#163964)
## Summary

According to @oatkiller, we have many CODEOWNERS rules that match no
files.

I updated the file according to this and my own audit:

```
$ github-codeowners validate
Found duplicate rules [
  'x-pack/packages/observability/alert_details @elastic/actionable-observability'
]
Found rules which did not match any files [
  'x-pack/test/alerting_api_integration/common/plugins/aad @elastic/response-ops',
  'x-pack/test/alerting_api_integration/common/plugins/actions_simulators @elastic/response-ops',
  'x-pack/test/alerting_api_integration/packages/helpers @elastic/response-ops',
  'x-pack/test/alerting_api_integration/common/plugins/alerts @elastic/response-ops',
  'x-pack/test/alerting_api_integration/common/plugins/alerts_restricted @elastic/response-ops',
  'test/analytics/plugins/analytics_ftr_helpers @elastic/kibana-core',
  'test/analytics/plugins/analytics_plugin_a @elastic/kibana-core',
  'x-pack/test/security_api_integration/plugins/audit_log @elastic/kibana-security',
  'x-pack/test/cases_api_integration/common/plugins/cases @elastic/response-ops',
  'x-pack/plugins/cloud_defend @elastic/sec-cloudnative-integrations',
  'src/plugins/data @elastic/kibana-visualizations @elastic/kibana-data-discovery',
  'x-pack/test/encrypted_saved_objects_api_integration/plugins/api_consumer_plugin @elastic/kibana-security',
  'x-pack/test/ui_capabilities/common/plugins/foo_plugin @elastic/kibana-security',
  'test/interactive_setup_api_integration/plugins/test_endpoints @elastic/kibana-security',
  'x-pack/test/cases_api_integration/common/plugins/observability @elastic/response-ops',
  'x-pack/test/security_api_integration/plugins/oidc_provider @elastic/kibana-security',
  'packages/kbn-performance-testing-dataset-extractor @elastic/kibana-performance-testing',
  'packages/react/kibana_context/common @elastic/appex-sharedux',
  'packages/react/kibana_context/render @elastic/appex-sharedux',
  'packages/react/kibana_context/root @elastic/appex-sharedux',
  'packages/react/kibana_context/styled @elastic/appex-sharedux',
  'packages/react/kibana_context/theme @elastic/appex-sharedux',
  'packages/react/kibana_mount @elastic/appex-sharedux',
  'x-pack/test/security_api_integration/plugins/saml_provider @elastic/kibana-security',
  'x-pack/test/saved_object_api_integration/common/plugins/saved_object_test_plugin @elastic/kibana-security',
  'x-pack/test/security_api_integration/packages/helpers @elastic/kibana-core',
  'x-pack/plugins/security_solution_ess @elastic/security-solution',
  'x-pack/test/cases_api_integration/common/plugins/security_solution @elastic/response-ops',
  'x-pack/plugins/security_solution_serverless @elastic/security-solution',
  'x-pack/test/security_functional/plugins/test_endpoints @elastic/kibana-security',
  'x-pack/test/spaces_api_integration/common/plugins/spaces_test_plugin @elastic/kibana-security',
  'x-pack/test/alerting_api_integration/common/plugins/task_manager_fixture @elastic/response-ops',
  'x-pack/test/security_api_integration/plugins/user_profiles_consumer @elastic/kibana-security',
  '/src/plugins/visualize/ @elastic/kibana-visualizations',
  '/x-pack/test/api_integration/apis/lens/ @elastic/kibana-visualizations',
  '/x-pack/test/functional/apps/dashboard/reporting/ @elastic/appex-sharedux',
  '/x-pack/test/functional/es_archives/lens/reporting/ @elastic/appex-sharedux',
  '/src/plugins/kibana_react/public/@elastic/appex-sharedux @elastic/kibana-presentation',
  '/.github/workflows/deploy-my-kibana.yml @elastic/observablt-robots',
  '/.github/workflows/oblt-github-commands @elastic/observablt-robots',
  '/x-pack/test/api_integration/apis/infra @elastic/infra-monitoring-ui',
  '/src/dev/build/tasks/bundle_fleet_packages.ts @elastic/fleet @elastic/kibana-operations',
  '/src/apm.js @elastic/kibana-core @vigneshshanmugam',
  '/src/core/types/elasticsearch @elastic/apm-ui',
  '/packages/kbn-utility-types/src/dot_test.ts @dgieselaar',
  '/x-pack/test/alerting_api_integration/observability/synthetics_rule.ts @elastic/uptime',
  '/x-pack/test/alerting_api_integration/observability/index.ts @elastic/uptime',
  '/x-pack/plugins/apm/public/application/uxApp.tsx @elastic/uptime',
  '/x-pack/plugins/apm/public/components/app/rum_dashboard @elastic/uptime',
  '/x-pack/test/apm_api_integration/tests/csm/ @elastic/uptime',
  '/x-pack/test/api_integration_basic/apis/ml/ @elastic/ml-ui',
  '/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/ml_rule_types/ @elastic/ml-ui',
  '/x-pack/test/alerting_api_integration/spaces_only/tests/alerting/transform_rule_types/ @elastic/ml-ui',
  '/x-pack/test/screenshot_creation/services/ml_screenshots.ts @elastic/ml-ui',
  '/x-pack/test/functional/es_archives/maps/ @elastic/kibana-gis',
  '/src/dev/license_checker/config.ts @elastic/kibana-operations',
  '/test/visual_regression @elastic/appex-qa',
  '/x-pack/test/visual_regression @elastic/appex-qa',
  '/src/core/public/i18n/  @elastic/kibana-localization @elastic/kibana-core',
  '/x-pack/plugins/security_solution/public/common/components/alerts_viewer @elastic/security-threat-hunting-investigations',
  '/x-pack/plugins/security_solution/public/detections/components/alerts_table/timeline_action @elastic/security-threat-hunting-investigations',
  '/x-pack/plugins/security_solution/public/detections/components/alerts_info @elastic/security-threat-hunting-investigations',
  '/x-pack/plugins/security_solution/common/search_strategy/security_solution/user @elastic/security-threat-hunting-explore',
  '/x-pack/plugins/security_solution/cypress/upgrade_e2e/threat_hunting/cases @elastic/security-threat-hunting-explore',
  '/x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema @elastic/security-detection-rule-management @elastic/security-detection-engine',
  '/x-pack/plugins/security_solution/common/api/detection_engine/fleet_integrations @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/api/detection_engine/rule_management @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/api/detection_engine/rule_monitoring @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/detection_engine/fleet_integrations @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/detection_engine/prebuilt_rules @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/detection_engine/rule_monitoring @elastic/security-detection-rule-management',
  '/x-pack/plugins/security_solution/common/detection_engine/rule_exceptions @elastic/security-detection-engine',
  '/x-pack/plugins/security_solution/public/detection_engine/rule_exceptions_ui @elastic/security-detection-engine',
  '/x-pack/plugins/security_solution/public/common/components/exceptions @elastic/security-detection-engine',
  '/x-pack/plugins/security_solution/server/lib/sourcerer @elastic/security-detection-engine',
  '/x-pack/plugins/security_solution/cypress/e2e/detection_rules @elastic/security-detection-rule-management @elastic/security-detection-engine',
  '/x-pack/plugins/security_solution/cypress/plugins @elastic/security-detections-response @elastic/security-threat-hunting',
  '/x-pack/plugins/security_solution/common/ecs @elastic/security-threat-hunting-investigations',
  '/x-pack/plugins/security_solution/server/lib/detection_engine/rule_actions @elastic/security-detection-engine @elastic/security-detection-rule-management',
  'x-pack/plugins/security_solution/cypress/ccs_e2e @elastic/security-engineering-productivity',
  'x-pack/plugins/security_solution/cypress/upgrade_e2e @elastic/security-engineering-productivity',
  '/x-pack/plugins/security_solution/common/detection_engine/rule_response_actions @elastic/security-defend-workflows',
  '/x-pack/plugins/security_solution/public/common/components/guided_onboarding @elastic/security-threat-hunting-explore',
  '/x-pack/plugins/apm/**/*.scss @elastic/observability-design',
  '/x-pack/plugins/endpoint/**/*.scss @elastic/security-design',
  '/x-pack/plugins/security_solution_ess/**/*.scss @elastic/security-design',
  '/x-pack/plugins/security_solution_serverless/**/*.scss @elastic/security-design'
]
```
2023-08-16 07:27:53 -07:00
Gloria Hornero
e78d61789f
[Security Solution] Updates codeowners file (#164026) 2023-08-16 05:38:11 -07:00
Gloria Hornero
f9c9722c6f
[Security Solution][Serverless] Reusing Cypress tests for Serverless infrastructure (#162698)
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
2023-08-15 19:50:54 +02:00
Dzmitry Lemechko
12a10d9855
Add appex-qa as codeowner for FTR serverless base config files (#163914)
## Summary

This way Appex-QA can keep track on config changes and make sure folks
aware that some changes might not work for real MKI-hosted projects.
2023-08-15 02:13:56 -07:00
Clint Andrew Hall
1546490e98
[settings] Extract and fix Section Registry (#163502)
## Summary

While working to extract various portions of the `advancedSettings`
plugin into packages, I found the `ComponentRegistry` in the plugin to
have a number of issues that contributed to a fairly bad UX:

- the API allows for adding/overriding the title, subtitle and footer of
the Advanced Settings page, but only the footer is rendered.
- the API is available to all plugins, but only renders a single
entry... so depending on the plugin load order, the render is not
guaranteed.
- filtering the footer in or out of the display is delegated to the
component itself, so:
  - it only takes effect on render.
- the count is only updated if you click on the page that contains it,
but that logic is currently broken.
  - the error message is inaccurate.

![Aug-09-2023
11-19-06](494aba14-f2c0-4ce7-b3f0-1910824aeb0e)

This PR fixes those issues and more:

- extracts the registry into its own package.
- changes the API to allow for multiple sections from multiple plugins.
- changes the API to filter these sections from the plugin, rather than
from each individual component.
- fixes state management to show sections, keep counts accurate, etc.

![Aug-09-2023
11-02-11](d8e8033c-f9ed-4615-b954-b5c23fda4d7a)

---------

Co-authored-by: Vadim Kibana <82822460+vadimkibana@users.noreply.github.com>
2023-08-14 11:00:33 -07:00
Dmitrii Shevchenko
bc37dc2c5a
[Security Solution] Initial OpenAPI codegen implementation (#163186)
**Resolves: https://github.com/elastic/security-team/issues/7134**

## Summary

Implemented request and response schema generation from OpenAPI
specifications.

The code generator script scans the
`x-pack/plugins/security_solution/common/api` directory, locates all
`*.schema.yaml` files, and generates a corresponding `*.gen.ts` artifact
for each, containing `zod` schema definitions.

<hr/>

Right now, all generation sources are set to `x-codegen-enabled: false`
to prevent the creation of duplicate schemas. Maintaining the old
`io-ts` schemas alongside the new `zod` ones could potentially lead to
confusion among developers. Thus, the recommended migration strategy is
to incrementally replace old schema usages with new ones, subsequently
removing outdated ones. I'll be implementing this approach in the
upcoming PRs.

### How to use the generator

If you need to test the generator locally, enable several sources and
run the generator script to see the results.

Navigate to `x-pack/plugins/security_solution` and run `yarn
openapi:generate`

<img width="916" alt="image"
src="be1a8a61-b9ed-4359-bc3e-bf393f256859">

Important note: if you want to enable route schemas, ensure you also
enable all their dependencies, such as common schemas. Failing to do so
will result in the generated code importing non-existent files.

### Example

Input file
(`x-pack/plugins/security_solution/common/api/detection_engine/model/error_schema.schema.yaml`):

```yaml
openapi: 3.0.0
info:
  title: Error Schema
  version: 'not applicable'
paths: {}
components:
  schemas:
    ErrorSchema:
      type: object
      required:
        - error
      properties:
        id:
          type: string
        rule_id:
          $ref: './rule_schema/common_attributes.schema.yaml#/components/schemas/RuleSignatureId'
        list_id:
          type: string
          minLength: 1
        item_id:
          type: string
          minLength: 1
        error:
          type: object
          required:
            - status_code
            - message
          properties:
            status_code:
              type: integer
              minimum: 400
            message:
              type: string
```

Generated output file
(`x-pack/plugins/security_solution/common/api/detection_engine/model/error_schema.gen.ts`):

```ts
/*
 * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
 * or more contributor license agreements. Licensed under the Elastic License
 * 2.0; you may not use this file except in compliance with the Elastic License
 * 2.0.
 */

import { z } from 'zod';

/*
 * NOTICE: Do not edit this file manually.
 * This file is automatically generated by the OpenAPI Generator `yarn openapi:generate`.
 */

import { RuleSignatureId } from './rule_schema/common_attributes.gen';

export type ErrorSchema = z.infer<typeof ErrorSchema>;
export const ErrorSchema = z.object({
  id: z.string().optional(),
  rule_id: RuleSignatureId.optional(),
  list_id: z.string().min(1).optional(),
  item_id: z.string().min(1).optional(),
  error: z.object({
    status_code: z.number().min(400),
    message: z.string(),
  }),
});
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-14 18:35:57 +02:00
Carlos Crespo
281cc224c9
Move Lens attribute builder to a package (#163422)
closes [#163491](https://github.com/elastic/kibana/issues/163491)

## Summary

This PR creates a new package that contains a utility API that helps to
generate the JSON with the attributes required to render a Lens chart
with the `EmbeddableComponent`.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-14 11:46:47 +02:00
Efe Gürkan YALAMAN
1e7efae56a
[Enterprise Search] Reuse serverless panels on Enterprise Search (#163179)
## Summary

Reuse Serverless panels for API index overview. 
Updated Enterprise Search overview to latests designs.

Note: There will be another PR for layout changes on overview
afterwards.





51537e57-e822-4b9f-b9ed-49d82d192690


a3696897-40a5-4cb3-9fe6-53ce5b8f560f


de752063-04ff-42a3-8538-7fb10a9df1ca

![Screenshot 2023-08-09 at 01 55
33](083504c4-7fa6-424b-a833-89dd73c61e49)



### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
2023-08-11 19:00:40 +02:00
Kerry Gallagher
00ffe1d791
[Logs+] Create an integration while on-boarding logs (#163219)
## Summary

This closes https://github.com/elastic/kibana/issues/161960, a basic
integration will now be created whilst onboarding logs (though the
custom logs flow).

This implements the *initial* version of this work, and does not include
things like adding a dataset to an existing integration.

## UI / UX

General:

![Screenshot 2023-08-07 at 15 20
21](3ca4e300-41c3-4554-a095-0f3dcf9e9523)

Naming conflict errors:

![Screenshot 2023-08-11 at 13 34
45](2a138eac-73e2-4cc9-b1e8-56c586b852ee)

![Screenshot 2023-08-11 at 13 34
59](6e651de9-debd-46aa-a3d5-2b6eb4e3bb4f)

Lack of permissions error:

![Screenshot 2023-08-09 at 17 10
35](d47b40c8-fe4a-4b86-abf8-d8fda51515fd)

General errors:

![Screenshot 2023-08-07 at 16 49
40](346c28d0-ec3e-4f7e-ae16-3f1adf440c21)

Success callout on the next panel:

![Screenshot 2023-08-07 at 17 20
45](03e78e45-871b-4224-9999-5b3d7e2ccdf0)

Delete previous flow (happens in the background):


![delete_process](44c18793-9df7-4228-b351-5668f098e138)


## Pointers for reviewers  / next steps

- This PR also creates a new package for the `useTrackedPromise` hook,
as this is used in several places and I didn't want to just duplicate it
again (I haven't replaced other current uses in this PR, but will as a
followup).

- `useFetcher` was avoided as A) it's very tightly coupled with the
observability onboarding server route repository (and `callApi` is
scoped to this) and I wanted to call an "external" API in Fleet and B) I
wanted explicit control over when the request is dispatched (not on
mount), and whilst this can sort of be achieved by not returning a
promise from the callback it gets quite messy. I also wanted more
granular error handling control.

- Moving forward I think we'll need to enhance the state management of
the plugin. We'll want to add the ability to "add to existing
integration" and this is going to make the state more complex (even with
chunks of this functionality likely moved to it's own package). I did
actually have the Wizard state moved in to a constate container at one
point (as a starter) but I reverted this commit to make the changeset
less intrusive. It's for this same reason that, for now, I haven't
focussed too closely on extracting things like generating the friendly
error messages etc as we'll likely want to extract some of the "create
integration" hooks / UI in to a standalone package so they can be used
elsewhere (not just onboarding). There are also quite a few `
eslint-disable-next-line react-hooks/exhaustive-deps` rules in the
plugin at the moment due to the references not being stable, we could
improve that at the same time as any state changes.

- You can technically navigate directly to
`/fox/app/observabilityOnboarding/customLogs/installElasticAgent`, but
no state is stored in the URL, so nothing is rehydrated resulting in a
very empty configuration. I'm not entirely sure this is a behaviour we
want, but for now I've just made the callout conditional on state
existing (so coming from the previous panel).

- The Fleet custom integrations API now throws a 409 (conflict) when
using a name that already exists.

## Testing

- Head to `/app/observabilityOnboarding` to trigger the onboarding flow
- Select "Stream log files"
- When hitting "continue" an integration should be created in the
background (check the network requests for
`api/fleet/epm/custom_integrations`)
- When continuing (to install shipper), then going back **and** making
changes to your integration options, when clicking continue again there
should be a network request that deletes the previously created
integration (to clean things up). This should be seamless to the user.
- You should not be able to use a name that already exists (for an
existing custom integration)
- General errors (like permission issues, asset installation issues)
should display at the bottom
- When you hit the next panel (install shipper) there should be a
success callout that also contains the name of the integration that was
created

## In progress

~Two changes still in progress, but they don't need to hold up the
review (8.10 coming soon 👀):~

- ~To have a friendlier error for permissions issues (not just
"forbidden")~
- ~Fleet API integration test for the naming collision~

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-11 17:18:55 +01:00
Julia Rechkunova
c97d4960bf
[Discover] Inline shard failures warnings (#161271)
- Closes https://github.com/elastic/kibana/issues/155216

## Summary

This PR replaces shard falures toasts with inline warnings in Discover.

- [x] Intercept shard failures in Discover main app
- [x] Show inline warnings above the grid instead
- [x] Handle NoResultsFound case too
- [x] Implement for Discover context app
- [x] Implement for saved search embeddable on Dashboard
- [x] Can we inline timeouts too?
- [x] Check SQL view
- [x] Add tests


Discover view with shard failures
<img width="400" alt="Screenshot 2023-07-06 at 14 23 48"
src="a0799aa0-9d2e-42ee-b89b-e0e1180220a5">

Discover view with shard failures (and no results)
<img width="400" alt="Screenshot 2023-07-07 at 13 24 50"
src="28dc2bad-9776-4aa9-8f51-219d1c87487a">

Dashboard view with shard failures
<img width="400" alt="Screenshot 2023-07-06 at 16 15 49"
src="2a68c19a-1ca8-4f10-a9f5-4aa56f9160b0">

Surrounding documents view with shard failures
<img width="400" alt="Screenshot 2023-07-10 at 17 26 31"
src="ade63cfe-a1c2-4c22-8823-58dcfef9357f">

Discover view with timeouts
<img width="400" alt="Screenshot 2023-07-07 at 16 47 27"
src="0101be19-a555-4f96-b963-7fe418d51fb5">

Dashboard view with timeouts
<img width="400" alt="Screenshot 2023-07-07 at 16 48 18"
src="82979365-4129-4385-8a13-9c139e1acbf1">

Surrounding documents view with timeouts
<img width="400" alt="Screenshot 2023-07-11 at 15 03 41"
src="0ea41b79-ac6b-4456-9cfa-0d038b10da7d">

## Testing

For testing please uncomment
3f102cf688/src/plugins/data/common/search/search_source/search_source.ts (L922)
or
3f102cf688/src/plugins/data/common/search/search_source/search_source.ts (L547)
and switch to `kibana*` data view.


### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

---------

Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-10 13:49:45 +02:00
Patryk Kopyciński
4637b744d8
Add SentinelOne connector (#159157)
## Summary

Adds new connector type to support https://www.sentinelone.com/

The scope of this PR was limited to the Connector logic, schemas, and
types to make PR more digestible.
In the current PR, the connector is NOT registered, so it's not going to
be available to the users.
In the follow-up PR I'm going to improve the UX of Param's form and then
enable the connector

<img width="1685" alt="Zrzut ekranu 2023-08-3 o 11 18 54"
src="965ef8ef-497f-42a8-983e-38fd0370cba8">
 visual changes include a screenshot or gif.

<img width="1685" alt="image"
src="119d2255-ed9f-4923-886d-eb139223a47d">

<img width="1690" alt="image"
src="e2c569d2-b497-4641-a6a6-454494223ffc">
2023-08-09 11:02:11 -07:00
Marshall Main
db0996f4a3
[Security Solution] Move remaining timeline route schemas to /common/api (#162857)
Closes https://github.com/elastic/security-team/issues/7099
Follow up to https://github.com/elastic/kibana/pull/162314

I mislabeled 3 timeline-related internal APIs as detection engine APIs
on [this
spreadsheet](https://docs.google.com/spreadsheets/d/1VCoJ74EkyGuj59VwWj_3v2ecB84pNCpzGqkYnS0SUKw/edit?pli=1#gid=1102015677)
(create_tags, get_tags_by_name, get_dashboards_by_tags). The APIs are
now correctly categorized on the spreadsheet and this PR establishes
schemas for them in `/common/api`.

I also converted these 3 small schemas to io-ts to make it easier to
avoid pulling in `@kbn/config-schema` to `public`, as that increased the
async chunk size by a full 840KB.
2023-08-09 07:26:08 -07:00
Marshall Main
30ca22c254
[Security Solution] Move risk score API schemas to /common/api (#163032)
Closes https://github.com/elastic/security-team/issues/7101
2023-08-07 07:33:08 -07:00