[Logs+] Create an integration while on-boarding logs (#163219)

## Summary

This closes https://github.com/elastic/kibana/issues/161960, a basic
integration will now be created whilst onboarding logs (though the
custom logs flow).

This implements the *initial* version of this work, and does not include
things like adding a dataset to an existing integration.

## UI / UX

General:

![Screenshot 2023-08-07 at 15 20
21](3ca4e300-41c3-4554-a095-0f3dcf9e9523)

Naming conflict errors:

![Screenshot 2023-08-11 at 13 34
45](2a138eac-73e2-4cc9-b1e8-56c586b852ee)

![Screenshot 2023-08-11 at 13 34
59](6e651de9-debd-46aa-a3d5-2b6eb4e3bb4f)

Lack of permissions error:

![Screenshot 2023-08-09 at 17 10
35](d47b40c8-fe4a-4b86-abf8-d8fda51515fd)

General errors:

![Screenshot 2023-08-07 at 16 49
40](346c28d0-ec3e-4f7e-ae16-3f1adf440c21)

Success callout on the next panel:

![Screenshot 2023-08-07 at 17 20
45](03e78e45-871b-4224-9999-5b3d7e2ccdf0)

Delete previous flow (happens in the background):


![delete_process](44c18793-9df7-4228-b351-5668f098e138)


## Pointers for reviewers  / next steps

- This PR also creates a new package for the `useTrackedPromise` hook,
as this is used in several places and I didn't want to just duplicate it
again (I haven't replaced other current uses in this PR, but will as a
followup).

- `useFetcher` was avoided as A) it's very tightly coupled with the
observability onboarding server route repository (and `callApi` is
scoped to this) and I wanted to call an "external" API in Fleet and B) I
wanted explicit control over when the request is dispatched (not on
mount), and whilst this can sort of be achieved by not returning a
promise from the callback it gets quite messy. I also wanted more
granular error handling control.

- Moving forward I think we'll need to enhance the state management of
the plugin. We'll want to add the ability to "add to existing
integration" and this is going to make the state more complex (even with
chunks of this functionality likely moved to it's own package). I did
actually have the Wizard state moved in to a constate container at one
point (as a starter) but I reverted this commit to make the changeset
less intrusive. It's for this same reason that, for now, I haven't
focussed too closely on extracting things like generating the friendly
error messages etc as we'll likely want to extract some of the "create
integration" hooks / UI in to a standalone package so they can be used
elsewhere (not just onboarding). There are also quite a few `
eslint-disable-next-line react-hooks/exhaustive-deps` rules in the
plugin at the moment due to the references not being stable, we could
improve that at the same time as any state changes.

- You can technically navigate directly to
`/fox/app/observabilityOnboarding/customLogs/installElasticAgent`, but
no state is stored in the URL, so nothing is rehydrated resulting in a
very empty configuration. I'm not entirely sure this is a behaviour we
want, but for now I've just made the callout conditional on state
existing (so coming from the previous panel).

- The Fleet custom integrations API now throws a 409 (conflict) when
using a name that already exists.

## Testing

- Head to `/app/observabilityOnboarding` to trigger the onboarding flow
- Select "Stream log files"
- When hitting "continue" an integration should be created in the
background (check the network requests for
`api/fleet/epm/custom_integrations`)
- When continuing (to install shipper), then going back **and** making
changes to your integration options, when clicking continue again there
should be a network request that deletes the previously created
integration (to clean things up). This should be seamless to the user.
- You should not be able to use a name that already exists (for an
existing custom integration)
- General errors (like permission issues, asset installation issues)
should display at the bottom
- When you hit the next panel (install shipper) there should be a
success callout that also contains the name of the integration that was
created

## In progress

~Two changes still in progress, but they don't need to hold up the
review (8.10 coming soon 👀):~

- ~To have a friendlier error for permissions issues (not just
"forbidden")~
- ~Fleet API integration test for the naming collision~

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Kerry Gallagher 2023-08-11 17:18:55 +01:00 committed by GitHub
parent 4de61111b1
commit 00ffe1d791
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
20 changed files with 1055 additions and 106 deletions

1
.github/CODEOWNERS vendored
View file

@ -752,6 +752,7 @@ src/plugins/url_forwarding @elastic/kibana-visualizations
packages/kbn-url-state @elastic/security-threat-hunting-investigations
src/plugins/usage_collection @elastic/kibana-core
test/plugin_functional/plugins/usage_collection @elastic/kibana-core
packages/kbn-use-tracked-promise @elastic/infra-monitoring-ui
packages/kbn-user-profile-components @elastic/kibana-security
examples/user_profile_examples @elastic/kibana-security
x-pack/test/security_api_integration/plugins/user_profiles_consumer @elastic/kibana-security

View file

@ -741,6 +741,7 @@
"@kbn/url-state": "link:packages/kbn-url-state",
"@kbn/usage-collection-plugin": "link:src/plugins/usage_collection",
"@kbn/usage-collection-test-plugin": "link:test/plugin_functional/plugins/usage_collection",
"@kbn/use-tracked-promise": "link:packages/kbn-use-tracked-promise",
"@kbn/user-profile-components": "link:packages/kbn-user-profile-components",
"@kbn/user-profile-examples-plugin": "link:examples/user_profile_examples",
"@kbn/user-profiles-consumer-plugin": "link:x-pack/test/security_api_integration/plugins/user_profiles_consumer",

View file

@ -0,0 +1,62 @@
# @kbn/use-tracked-promise
/**
* This hook manages a Promise factory and can create new Promises from it. The
* state of these Promises is tracked and they can be canceled when superseded
* to avoid race conditions.
*
* ```
* const [requestState, performRequest] = useTrackedPromise(
* {
* cancelPreviousOn: 'resolution',
* createPromise: async (url: string) => {
* return await fetchSomething(url)
* },
* onResolve: response => {
* setSomeState(response.data);
* },
* onReject: response => {
* setSomeError(response);
* },
* },
* [fetchSomething]
* );
* ```
*
* The `onResolve` and `onReject` handlers are registered separately, because
* the hook will inject a rejection when in case of a canellation. The
* `cancelPreviousOn` attribute can be used to indicate when the preceding
* pending promises should be canceled:
*
* 'never': No preceding promises will be canceled.
*
* 'creation': Any preceding promises will be canceled as soon as a new one is
* created.
*
* 'settlement': Any preceding promise will be canceled when a newer promise is
* resolved or rejected.
*
* 'resolution': Any preceding promise will be canceled when a newer promise is
* resolved.
*
* 'rejection': Any preceding promise will be canceled when a newer promise is
* rejected.
*
* Any pending promises will be canceled when the component using the hook is
* unmounted, but their status will not be tracked to avoid React warnings
* about memory leaks.
*
* The last argument is a normal React hook dependency list that indicates
* under which conditions a new reference to the configuration object should be
* used.
*
* The `onResolve`, `onReject` and possible uncatched errors are only triggered
* if the underlying component is mounted. To ensure they always trigger (i.e.
* if the promise is called in a `useLayoutEffect`) use the `triggerOrThrow`
* attribute:
*
* 'whenMounted': (default) they are called only if the component is mounted.
*
* 'always': they always call. The consumer is then responsible of ensuring no
* side effects happen if the underlying component is not mounted.
*/

View file

@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export { useTrackedPromise } from './use_tracked_promise';

View file

@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
module.exports = {
preset: '@kbn/test/jest_node',
rootDir: '../..',
roots: ['<rootDir>/packages/kbn-use-tracked-promise'],
};

View file

@ -0,0 +1,5 @@
{
"type": "shared-common",
"id": "@kbn/use-tracked-promise",
"owner": "@elastic/infra-monitoring-ui"
}

View file

@ -0,0 +1,6 @@
{
"name": "@kbn/use-tracked-promise",
"private": true,
"version": "1.0.0",
"license": "SSPL-1.0 OR Elastic License 2.0"
}

View file

@ -0,0 +1,17 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "target/types",
"types": [
"jest",
"node"
]
},
"include": [
"**/*.ts",
],
"exclude": [
"target/**/*"
],
"kbn_references": []
}

View file

@ -0,0 +1,300 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
/* eslint-disable max-classes-per-file */
import { DependencyList, useEffect, useMemo, useRef, useState, useCallback } from 'react';
import useMountedState from 'react-use/lib/useMountedState';
interface UseTrackedPromiseArgs<Arguments extends any[], Result> {
createPromise: (...args: Arguments) => Promise<Result>;
onResolve?: (result: Result) => void;
onReject?: (value: unknown) => void;
cancelPreviousOn?: 'creation' | 'settlement' | 'resolution' | 'rejection' | 'never';
triggerOrThrow?: 'always' | 'whenMounted';
}
/**
* This hook manages a Promise factory and can create new Promises from it. The
* state of these Promises is tracked and they can be canceled when superseded
* to avoid race conditions.
*
* ```
* const [requestState, performRequest] = useTrackedPromise(
* {
* cancelPreviousOn: 'resolution',
* createPromise: async (url: string) => {
* return await fetchSomething(url)
* },
* onResolve: response => {
* setSomeState(response.data);
* },
* onReject: response => {
* setSomeError(response);
* },
* },
* [fetchSomething]
* );
* ```
*
* The `onResolve` and `onReject` handlers are registered separately, because
* the hook will inject a rejection when in case of a canellation. The
* `cancelPreviousOn` attribute can be used to indicate when the preceding
* pending promises should be canceled:
*
* 'never': No preceding promises will be canceled.
*
* 'creation': Any preceding promises will be canceled as soon as a new one is
* created.
*
* 'settlement': Any preceding promise will be canceled when a newer promise is
* resolved or rejected.
*
* 'resolution': Any preceding promise will be canceled when a newer promise is
* resolved.
*
* 'rejection': Any preceding promise will be canceled when a newer promise is
* rejected.
*
* Any pending promises will be canceled when the component using the hook is
* unmounted, but their status will not be tracked to avoid React warnings
* about memory leaks.
*
* The last argument is a normal React hook dependency list that indicates
* under which conditions a new reference to the configuration object should be
* used.
*
* The `onResolve`, `onReject` and possible uncatched errors are only triggered
* if the underlying component is mounted. To ensure they always trigger (i.e.
* if the promise is called in a `useLayoutEffect`) use the `triggerOrThrow`
* attribute:
*
* 'whenMounted': (default) they are called only if the component is mounted.
*
* 'always': they always call. The consumer is then responsible of ensuring no
* side effects happen if the underlying component is not mounted.
*/
export const useTrackedPromise = <Arguments extends any[], Result>(
{
createPromise,
onResolve = noOp,
onReject = noOp,
cancelPreviousOn = 'never',
triggerOrThrow = 'whenMounted',
}: UseTrackedPromiseArgs<Arguments, Result>,
dependencies: DependencyList
) => {
const isComponentMounted = useMountedState();
const shouldTriggerOrThrow = useCallback(() => {
switch (triggerOrThrow) {
case 'always':
return true;
case 'whenMounted':
return isComponentMounted();
}
}, [isComponentMounted, triggerOrThrow]);
/**
* If a promise is currently pending, this holds a reference to it and its
* cancellation function.
*/
const pendingPromises = useRef<ReadonlyArray<CancelablePromise<Result>>>([]);
/**
* The state of the promise most recently created by the `createPromise`
* factory. It could be uninitialized, pending, resolved or rejected.
*/
const [promiseState, setPromiseState] = useState<PromiseState<Result>>({
state: 'uninitialized',
});
const reset = useCallback(() => {
setPromiseState({
state: 'uninitialized',
});
}, []);
const execute = useMemo(
() =>
(...args: Arguments) => {
let rejectCancellationPromise!: (value: any) => void;
const cancellationPromise = new Promise<any>((_, reject) => {
rejectCancellationPromise = reject;
});
// remember the list of prior pending promises for cancellation
const previousPendingPromises = pendingPromises.current;
const cancelPreviousPendingPromises = () => {
previousPendingPromises.forEach((promise) => promise.cancel());
};
const newPromise = createPromise(...args);
const newCancelablePromise = Promise.race([newPromise, cancellationPromise]);
// track this new state
setPromiseState({
state: 'pending',
promise: newCancelablePromise,
});
if (cancelPreviousOn === 'creation') {
cancelPreviousPendingPromises();
}
const newPendingPromise: CancelablePromise<Result> = {
cancel: () => {
rejectCancellationPromise(new CanceledPromiseError());
},
cancelSilently: () => {
rejectCancellationPromise(new SilentCanceledPromiseError());
},
promise: newCancelablePromise.then(
(value) => {
if (['settlement', 'resolution'].includes(cancelPreviousOn)) {
cancelPreviousPendingPromises();
}
// remove itself from the list of pending promises
pendingPromises.current = pendingPromises.current.filter(
(pendingPromise) => pendingPromise.promise !== newPendingPromise.promise
);
if (onResolve && shouldTriggerOrThrow()) {
onResolve(value);
}
setPromiseState((previousPromiseState) =>
previousPromiseState.state === 'pending' &&
previousPromiseState.promise === newCancelablePromise
? {
state: 'resolved',
promise: newPendingPromise.promise,
value,
}
: previousPromiseState
);
return value;
},
(value) => {
if (!(value instanceof SilentCanceledPromiseError)) {
if (['settlement', 'rejection'].includes(cancelPreviousOn)) {
cancelPreviousPendingPromises();
}
// remove itself from the list of pending promises
pendingPromises.current = pendingPromises.current.filter(
(pendingPromise) => pendingPromise.promise !== newPendingPromise.promise
);
if (shouldTriggerOrThrow()) {
if (onReject) {
onReject(value);
} else {
throw value;
}
}
setPromiseState((previousPromiseState) =>
previousPromiseState.state === 'pending' &&
previousPromiseState.promise === newCancelablePromise
? {
state: 'rejected',
promise: newCancelablePromise,
value,
}
: previousPromiseState
);
}
}
),
};
// add the new promise to the list of pending promises
pendingPromises.current = [...pendingPromises.current, newPendingPromise];
// silence "unhandled rejection" warnings
newPendingPromise.promise.catch(noOp);
return newPendingPromise.promise;
},
// the dependencies are managed by the caller
// eslint-disable-next-line react-hooks/exhaustive-deps
dependencies
);
/**
* Cancel any pending promises silently to avoid memory leaks and race
* conditions.
*/
useEffect(
() => () => {
pendingPromises.current.forEach((promise) => promise.cancelSilently());
},
[]
);
return [promiseState, execute, reset] as [typeof promiseState, typeof execute, typeof reset];
};
export interface UninitializedPromiseState {
state: 'uninitialized';
}
export interface PendingPromiseState<ResolvedValue> {
state: 'pending';
promise: Promise<ResolvedValue>;
}
export interface ResolvedPromiseState<ResolvedValue> {
state: 'resolved';
promise: Promise<ResolvedValue>;
value: ResolvedValue;
}
export interface RejectedPromiseState<ResolvedValue, RejectedValue> {
state: 'rejected';
promise: Promise<ResolvedValue>;
value: RejectedValue;
}
export type SettledPromiseState<ResolvedValue, RejectedValue> =
| ResolvedPromiseState<ResolvedValue>
| RejectedPromiseState<ResolvedValue, RejectedValue>;
export type PromiseState<ResolvedValue, RejectedValue = unknown> =
| UninitializedPromiseState
| PendingPromiseState<ResolvedValue>
| SettledPromiseState<ResolvedValue, RejectedValue>;
export const isRejectedPromiseState = (
promiseState: PromiseState<any, any>
): promiseState is RejectedPromiseState<any, any> => promiseState.state === 'rejected';
interface CancelablePromise<ResolvedValue> {
// reject the promise prematurely with a CanceledPromiseError
cancel: () => void;
// reject the promise prematurely with a SilentCanceledPromiseError
cancelSilently: () => void;
// the tracked promise
promise: Promise<ResolvedValue>;
}
export class CanceledPromiseError extends Error {
public isCanceled = true;
constructor(message?: string) {
super(message);
Object.setPrototypeOf(this, new.target.prototype);
}
}
export class SilentCanceledPromiseError extends CanceledPromiseError {}
const noOp = () => undefined;

View file

@ -1498,6 +1498,8 @@
"@kbn/usage-collection-plugin/*": ["src/plugins/usage_collection/*"],
"@kbn/usage-collection-test-plugin": ["test/plugin_functional/plugins/usage_collection"],
"@kbn/usage-collection-test-plugin/*": ["test/plugin_functional/plugins/usage_collection/*"],
"@kbn/use-tracked-promise": ["packages/kbn-use-tracked-promise"],
"@kbn/use-tracked-promise/*": ["packages/kbn-use-tracked-promise/*"],
"@kbn/user-profile-components": ["packages/kbn-user-profile-components"],
"@kbn/user-profile-components/*": ["packages/kbn-user-profile-components/*"],
"@kbn/user-profile-examples-plugin": ["examples/user_profile_examples"],

View file

@ -84,6 +84,7 @@ import type {
InstallationInfo,
} from '../../types';
import { getDataStreams } from '../../services/epm/data_streams';
import { NamingCollisionError } from '../../services/epm/packages/custom_integrations/validation/check_naming_collision';
const CACHE_CONTROL_10_MINUTES_HEADER: HttpResponseOptions['headers'] = {
'cache-control': 'max-age=600',
@ -419,28 +420,40 @@ export const createCustomIntegrationHandler: FleetRequestHandler<
const spaceId = fleetContext.spaceId;
const { integrationName, force, datasets } = request.body;
const res = await installPackage({
installSource: 'custom',
savedObjectsClient,
pkgName: integrationName,
datasets,
esClient,
spaceId,
force,
authorizationHeader,
kibanaVersion,
});
try {
const res = await installPackage({
installSource: 'custom',
savedObjectsClient,
pkgName: integrationName,
datasets,
esClient,
spaceId,
force,
authorizationHeader,
kibanaVersion,
});
if (!res.error) {
const body: InstallPackageResponse = {
items: res.assets || [],
_meta: {
install_source: res.installSource,
},
};
return response.ok({ body });
} else {
return await defaultFleetErrorHandler({ error: res.error, response });
if (!res.error) {
const body: InstallPackageResponse = {
items: res.assets || [],
_meta: {
install_source: res.installSource,
},
};
return response.ok({ body });
} else {
return await defaultFleetErrorHandler({ error: res.error, response });
}
} catch (error) {
if (error instanceof NamingCollisionError) {
return response.customError({
statusCode: 409,
body: {
message: error.message,
},
});
}
return await defaultFleetErrorHandler({ error, response });
}
};

View file

@ -0,0 +1,88 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { nodeBuilder } from '@kbn/es-query';
import type { SavedObjectsClientContract } from '@kbn/core-saved-objects-api-server';
import { i18n } from '@kbn/i18n';
import { auditLoggingService } from '../../../../audit_logging';
import { PACKAGES_SAVED_OBJECT_TYPE, type Installation } from '../../../../../../common';
import * as Registry from '../../../registry';
export const checkForNamingCollision = async (
savedObjectsClient: SavedObjectsClientContract,
integrationName: string
) => {
await checkForRegistryNamingCollision(savedObjectsClient, integrationName);
await checkForInstallationNamingCollision(savedObjectsClient, integrationName);
};
export const checkForRegistryNamingCollision = async (
savedObjectsClient: SavedObjectsClientContract,
integrationName: string
) => {
const registryOrBundledPackage = await Registry.fetchFindLatestPackageOrUndefined(
integrationName
);
if (registryOrBundledPackage) {
const registryConflictMessage = i18n.translate(
'xpack.fleet.customIntegrations.namingCollisionError.registryOrBundle',
{
defaultMessage:
'Failed to create the integration as an integration with the name {integrationName} already exists in the package registry or as a bundled package.',
values: {
integrationName,
},
}
);
throw new NamingCollisionError(registryConflictMessage);
}
};
export const checkForInstallationNamingCollision = async (
savedObjectsClient: SavedObjectsClientContract,
integrationName: string
) => {
const result = await savedObjectsClient.find<Installation>({
type: PACKAGES_SAVED_OBJECT_TYPE,
perPage: 1,
filter: nodeBuilder.and([
nodeBuilder.is(`${PACKAGES_SAVED_OBJECT_TYPE}.attributes.name`, integrationName),
]),
});
if (result.saved_objects.length > 0) {
const installationConflictMessage = i18n.translate(
'xpack.fleet.customIntegrations.namingCollisionError.installationConflictMessage',
{
defaultMessage:
'Failed to create the integration as an installation with the name {integrationName} already exists.',
values: {
integrationName,
},
}
);
throw new NamingCollisionError(installationConflictMessage);
}
for (const savedObject of result.saved_objects) {
auditLoggingService.writeCustomSoAuditLog({
action: 'find',
id: savedObject.id,
savedObjectType: PACKAGES_SAVED_OBJECT_TYPE,
});
}
};
export class NamingCollisionError extends Error {
constructor(message?: string) {
super(message);
Object.setPrototypeOf(this, new.target.prototype);
this.name = 'NamingCollisionError';
}
}

View file

@ -102,6 +102,7 @@ import { INITIAL_VERSION } from './custom_integrations/constants';
import { createAssets } from './custom_integrations';
import { cacheAssets } from './custom_integrations/assets/cache';
import { generateDatastreamEntries } from './custom_integrations/assets/dataset/utils';
import { checkForNamingCollision } from './custom_integrations/validation/check_naming_collision';
export async function isPackageInstalled(options: {
savedObjectsClient: SavedObjectsClientContract;
@ -781,6 +782,9 @@ export async function installCustomPackage(
kibanaVersion,
} = args;
// Validate that we can create this package, validations will throw if they don't pass
await checkForNamingCollision(savedObjectsClient, pkgName);
// Compose a packageInfo
const packageInfo = {
format_version: CUSTOM_INTEGRATION_PACKAGE_SPEC_VERSION,

View file

@ -10,6 +10,7 @@ import {
EuiButton,
EuiButtonEmpty,
EuiButtonIcon,
EuiCallOut,
EuiFieldText,
EuiFlexGroup,
EuiFlexItem,
@ -27,7 +28,12 @@ import {
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n-react';
import { isEmpty } from 'lodash';
import React, { useState } from 'react';
import React, { useCallback, useState } from 'react';
import {
IntegrationError,
IntegrationOptions,
useCreateIntegration,
} from '../../../../hooks/use_create_integration';
import { useWizard } from '.';
import { OptionalFormRow } from '../../../shared/optional_form_row';
import {
@ -45,6 +51,13 @@ export function ConfigureLogs() {
const { goToStep, goBack, getState, setState } = useWizard();
const wizardState = getState();
const [integrationName, setIntegrationName] = useState(
wizardState.integrationName
);
const [integrationNameTouched, setIntegrationNameTouched] = useState(false);
const [integrationError, setIntegrationError] = useState<
IntegrationError | undefined
>();
const [datasetName, setDatasetName] = useState(wizardState.datasetName);
const [serviceName, setServiceName] = useState(wizardState.serviceName);
const [logFilePaths, setLogFilePaths] = useState(wizardState.logFilePaths);
@ -52,20 +65,65 @@ export function ConfigureLogs() {
const [customConfigurations, setCustomConfigurations] = useState(
wizardState.customConfigurations
);
const logFilePathNotConfigured = logFilePaths.every((filepath) => !filepath);
function onContinue() {
const onIntegrationCreationSuccess = useCallback(
(integration: IntegrationOptions) => {
setState((state) => ({
...state,
lastCreatedIntegration: integration,
}));
goToStep('installElasticAgent');
},
[goToStep, setState]
);
const onIntegrationCreationFailure = useCallback(
(error: IntegrationError) => {
setIntegrationError(error);
},
[setIntegrationError]
);
const { createIntegration, createIntegrationRequest } = useCreateIntegration({
onIntegrationCreationSuccess,
onIntegrationCreationFailure,
initialLastCreatedIntegration: wizardState.lastCreatedIntegration,
});
const isCreatingIntegration = createIntegrationRequest.state === 'pending';
const hasFailedCreatingIntegration =
createIntegrationRequest.state === 'rejected';
const onContinue = useCallback(() => {
setState((state) => ({
...state,
datasetName,
integrationName,
serviceName,
logFilePaths: logFilePaths.filter((filepath) => !!filepath),
namespace,
customConfigurations,
}));
goToStep('installElasticAgent');
}
createIntegration({
integrationName,
datasets: [
{
name: datasetName,
type: 'logs' as const,
},
],
});
}, [
createIntegration,
customConfigurations,
datasetName,
integrationName,
logFilePaths,
namespace,
serviceName,
setState,
]);
function addLogFilePath() {
setLogFilePaths((prev) => [...prev, '']);
@ -85,17 +143,30 @@ export function ConfigureLogs() {
);
if (index === 0) {
setIntegrationName(getFilename(filepath));
setDatasetName(getFilename(filepath));
}
}
const isDatasetNameInvalid = datasetNameTouched && isEmpty(datasetName);
const hasNamingCollision =
integrationError && integrationError.type === 'NamingCollision';
const datasetNameError = i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.error',
{ defaultMessage: 'A dataset name is required.' }
const isIntegrationNameInvalid =
(integrationNameTouched &&
(isEmpty(integrationName) || !isLowerCase(integrationName))) ||
hasNamingCollision;
const integrationNameError = getIntegrationNameError(
integrationName,
integrationNameTouched,
integrationError
);
const isDatasetNameInvalid =
datasetNameTouched && (isEmpty(datasetName) || !isLowerCase(datasetName));
const datasetNameError = getDatasetNameError(datasetName, datasetNameTouched);
return (
<StepPanel
panelFooter={
@ -106,13 +177,24 @@ export function ConfigureLogs() {
color="primary"
fill
onClick={onContinue}
isLoading={isCreatingIntegration}
isDisabled={
logFilePathNotConfigured || !datasetName || !namespace
}
>
{i18n.translate('xpack.observability_onboarding.steps.continue', {
defaultMessage: 'Continue',
})}
{isCreatingIntegration
? i18n.translate(
'xpack.observability_onboarding.steps.loading',
{
defaultMessage: 'Creating integration...',
}
)
: i18n.translate(
'xpack.observability_onboarding.steps.continue',
{
defaultMessage: 'Continue',
}
)}
</EuiButton>,
]}
/>
@ -124,8 +206,7 @@ export function ConfigureLogs() {
{i18n.translate(
'xpack.observability_onboarding.configureLogs.description',
{
defaultMessage:
'Fill the paths to the log files on your hosts.',
defaultMessage: 'Configure inputs',
}
)}
</p>
@ -195,61 +276,6 @@ export function ConfigureLogs() {
</EuiFlexItem>
</EuiFlexGroup>
<EuiSpacer size="s" />
<EuiFormRow
label={
<EuiFlexGroup
alignItems="center"
gutterSize="xs"
responsive={false}
>
<EuiFlexItem grow={false}>
{i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.name',
{
defaultMessage: 'Dataset name',
}
)}
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EuiIconTip
content={i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.name.tooltip',
{
defaultMessage:
'Provide a dataset name to help identify the source of your logs in future uses. Defaults to the name of the log file.',
}
)}
position="right"
/>
</EuiFlexItem>
</EuiFlexGroup>
}
helpText={i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.helper',
{
defaultMessage:
"All lowercase, max 100 chars, special characters will be replaced with '_'.",
}
)}
isInvalid={isDatasetNameInvalid}
error={datasetNameError}
>
<EuiFieldText
placeholder={i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.placeholder',
{
defaultMessage: 'Give your logs a name',
}
)}
value={datasetName}
onChange={(event) =>
setDatasetName(replaceSpecialChars(event.target.value))
}
isInvalid={isDatasetNameInvalid}
onInput={() => setDatasetNameTouched(true)}
/>
</EuiFormRow>
<EuiSpacer size="m" />
<OptionalFormRow
label={
<EuiFlexGroup
@ -435,7 +461,203 @@ export function ConfigureLogs() {
<EuiSpacer size="s" />
</EuiFlexGroup>
</EuiForm>
<EuiSpacer size="l" />
<EuiText color="subdued">
<p>
{i18n.translate(
'xpack.observability_onboarding.configureLogs.configureIntegrationDescription',
{
defaultMessage: 'Configure integration',
}
)}
</p>
</EuiText>
<EuiSpacer size="l" />
<EuiForm fullWidth>
<EuiFormRow
label={
<EuiFlexGroup
alignItems="center"
gutterSize="xs"
responsive={false}
>
<EuiFlexItem grow={false}>
{i18n.translate(
'xpack.observability_onboarding.configureLogs.integration.name',
{
defaultMessage: 'Integration name',
}
)}
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EuiIconTip
content={i18n.translate(
'xpack.observability_onboarding.configureLogs.integration.name.tooltip',
{
defaultMessage:
'Provide an integration name for the integration that will be created to organise these custom logs. Defaults to the name of the log file.',
}
)}
position="right"
/>
</EuiFlexItem>
</EuiFlexGroup>
}
helpText={i18n.translate(
'xpack.observability_onboarding.configureLogs.integration.helper',
{
defaultMessage:
"All lowercase, max 100 chars, special characters will be replaced with '_'.",
}
)}
isInvalid={isIntegrationNameInvalid}
error={integrationNameError}
>
<EuiFieldText
placeholder={i18n.translate(
'xpack.observability_onboarding.configureLogs.integration.placeholder',
{
defaultMessage: 'Give your integration a name',
}
)}
value={integrationName}
onChange={(event) =>
setIntegrationName(replaceSpecialChars(event.target.value))
}
isInvalid={isIntegrationNameInvalid}
onInput={() => setIntegrationNameTouched(true)}
/>
</EuiFormRow>
<EuiFormRow
label={
<EuiFlexGroup
alignItems="center"
gutterSize="xs"
responsive={false}
>
<EuiFlexItem grow={false}>
{i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.name',
{
defaultMessage: 'Dataset name',
}
)}
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EuiIconTip
content={i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.name.tooltip',
{
defaultMessage:
'Provide a dataset name to help organise these custom logs. This dataset will be associated with the integration. Defaults to the name of the log file.',
}
)}
position="right"
/>
</EuiFlexItem>
</EuiFlexGroup>
}
helpText={i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.helper',
{
defaultMessage:
"All lowercase, max 100 chars, special characters will be replaced with '_'.",
}
)}
isInvalid={isDatasetNameInvalid}
error={datasetNameError}
>
<EuiFieldText
placeholder={i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.placeholder',
{
defaultMessage: "Give your integration's dataset a name",
}
)}
value={datasetName}
onChange={(event) =>
setDatasetName(replaceSpecialChars(event.target.value))
}
isInvalid={isDatasetNameInvalid}
onInput={() => setDatasetNameTouched(true)}
/>
</EuiFormRow>
</EuiForm>
{hasFailedCreatingIntegration && integrationError && (
<>
<EuiSpacer size="l" />
{getIntegrationErrorCallout(integrationError)}
</>
)}
</StepPanelContent>
</StepPanel>
);
}
const getIntegrationErrorCallout = (integrationError: IntegrationError) => {
const title = i18n.translate(
'xpack.observability_onboarding.configureLogs.integrationCreation.error.title',
{ defaultMessage: 'Sorry, there was an error' }
);
switch (integrationError.type) {
case 'AuthorizationError':
const authorizationDescription = i18n.translate(
'xpack.observability_onboarding.configureLogs.integrationCreation.error.authorization.description',
{
defaultMessage:
'This user does not have permissions to create an integration.',
}
);
return (
<EuiCallOut title={title} color="danger" iconType="error">
<p>{authorizationDescription}</p>
</EuiCallOut>
);
case 'UnknownError':
return (
<EuiCallOut title={title} color="danger" iconType="error">
<p>{integrationError.message}</p>
</EuiCallOut>
);
}
};
const isLowerCase = (str: string) => str.toLowerCase() === str;
const getIntegrationNameError = (
integrationName: string,
touched: boolean,
integrationError?: IntegrationError
) => {
if (touched && isEmpty(integrationName)) {
return i18n.translate(
'xpack.observability_onboarding.configureLogs.integration.emptyError',
{ defaultMessage: 'An integration name is required.' }
);
}
if (touched && !isLowerCase(integrationName)) {
return i18n.translate(
'xpack.observability_onboarding.configureLogs.integration.lowercaseError',
{ defaultMessage: 'An integration name should be lowercase.' }
);
}
if (integrationError && integrationError.type === 'NamingCollision') {
return integrationError.message;
}
};
const getDatasetNameError = (datasetName: string, touched: boolean) => {
if (touched && isEmpty(datasetName)) {
return i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.emptyError',
{ defaultMessage: 'A dataset name is required.' }
);
}
if (touched && !isLowerCase(datasetName)) {
return i18n.translate(
'xpack.observability_onboarding.configureLogs.dataset.lowercaseError',
{ defaultMessage: 'A dataset name should be lowercase.' }
);
}
};

View file

@ -6,6 +6,7 @@
*/
import { i18n } from '@kbn/i18n';
import { IntegrationOptions } from '../../../../hooks/use_create_integration';
import {
createWizardContext,
Step,
@ -16,6 +17,8 @@ import { InstallElasticAgent } from './install_elastic_agent';
import { SelectLogs } from './select_logs';
interface WizardState {
integrationName: string;
lastCreatedIntegration?: IntegrationOptions;
datasetName: string;
serviceName: string;
logFilePaths: string[];
@ -37,6 +40,7 @@ interface WizardState {
}
const initialState: WizardState = {
integrationName: '',
datasetName: '',
serviceName: '',
logFilePaths: [''],

View file

@ -8,6 +8,7 @@
import {
EuiButton,
EuiButtonEmpty,
EuiCallOut,
EuiFlexGroup,
EuiFlexItem,
EuiSpacer,
@ -267,6 +268,24 @@ export function InstallElasticAgent() {
</p>
</EuiText>
<EuiSpacer size="m" />
{wizardState.integrationName && (
<>
<EuiCallOut
title={i18n.translate(
'xpack.observability_onboarding.installElasticAgent.integrationSuccessCallout.title',
{
defaultMessage: '{integrationName} integration installed.',
values: {
integrationName: wizardState.integrationName,
},
}
)}
color="success"
iconType="check"
/>
<EuiSpacer size="m" />
</>
)}
{apiKeyEncoded && onboardingId ? (
<ApiKeyBanner
payload={{ apiKeyEncoded, onboardingId }}

View file

@ -0,0 +1,123 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { useCallback, useState } from 'react';
import deepEqual from 'react-fast-compare';
import { useKibana } from '@kbn/kibana-react-plugin/public';
import { useTrackedPromise } from '@kbn/use-tracked-promise';
import { i18n } from '@kbn/i18n';
export interface IntegrationOptions {
integrationName: string;
datasets: Array<{
name: string;
type: 'logs';
}>;
}
// Errors
const GENERIC_ERROR_MESSAGE = i18n.translate(
'xpack.observability_onboarding.useCreateIntegration.integrationError.genericError',
{
defaultMessage: 'Unable to create an integration',
}
);
type ErrorType = 'NamingCollision' | 'AuthorizationError' | 'UnknownError';
export interface IntegrationError {
type: ErrorType;
message: string;
}
export const useCreateIntegration = ({
onIntegrationCreationSuccess,
onIntegrationCreationFailure,
initialLastCreatedIntegration,
deletePreviousIntegration = true,
}: {
integrationOptions?: IntegrationOptions;
onIntegrationCreationSuccess: (integration: IntegrationOptions) => void;
onIntegrationCreationFailure: (error: IntegrationError) => void;
initialLastCreatedIntegration?: IntegrationOptions;
deletePreviousIntegration?: boolean;
}) => {
const {
services: { http },
} = useKibana();
const [lastCreatedIntegration, setLastCreatedIntegration] = useState<
IntegrationOptions | undefined
>(initialLastCreatedIntegration);
const [createIntegrationRequest, callCreateIntegration] = useTrackedPromise(
{
cancelPreviousOn: 'creation',
createPromise: async (integrationOptions) => {
if (lastCreatedIntegration && deletePreviousIntegration) {
await http?.delete(
`/api/fleet/epm/packages/${lastCreatedIntegration.integrationName}/1.0.0`,
{}
);
}
await http?.post('/api/fleet/epm/custom_integrations', {
body: JSON.stringify(integrationOptions),
});
return integrationOptions;
},
onResolve: (integrationOptions: IntegrationOptions) => {
setLastCreatedIntegration(integrationOptions);
onIntegrationCreationSuccess(integrationOptions!);
},
onReject: (requestError: any) => {
if (requestError?.body?.statusCode === 409) {
onIntegrationCreationFailure({
type: 'NamingCollision' as const,
message: requestError.body.message,
});
} else if (requestError?.body?.statusCode === 403) {
onIntegrationCreationFailure({
type: 'AuthorizationError' as const,
message: requestError?.body?.message,
});
} else {
onIntegrationCreationFailure({
type: 'UnknownError' as const,
message: requestError?.body?.message ?? GENERIC_ERROR_MESSAGE,
});
}
},
},
[
lastCreatedIntegration,
deletePreviousIntegration,
onIntegrationCreationSuccess,
onIntegrationCreationFailure,
setLastCreatedIntegration,
]
);
const createIntegration = useCallback(
(integrationOptions: IntegrationOptions) => {
// Bypass creating the integration again
if (deepEqual(integrationOptions, lastCreatedIntegration)) {
onIntegrationCreationSuccess(integrationOptions);
} else {
callCreateIntegration(integrationOptions);
}
},
[
callCreateIntegration,
lastCreatedIntegration,
onIntegrationCreationSuccess,
]
);
return {
createIntegration,
createIntegrationRequest,
};
};

View file

@ -32,6 +32,7 @@
"@kbn/std",
"@kbn/data-views-plugin",
"@kbn/es-query",
"@kbn/use-tracked-promise",
],
"exclude": [
"target/**/*",

View file

@ -10,7 +10,7 @@ import { PACKAGES_SAVED_OBJECT_TYPE } from '@kbn/fleet-plugin/common';
import { FtrProviderContext } from '../../../api_integration/ftr_provider_context';
const INTEGRATION_NAME = 'my_custom_nginx';
const INTEGRATION_NAME = 'my_nginx';
const INTEGRATION_VERSION = '1.0.0';
export default function (providerContext: FtrProviderContext) {
@ -21,7 +21,8 @@ export default function (providerContext: FtrProviderContext) {
const uninstallPackage = async () => {
await supertest
.delete(`/api/fleet/epm/packages/${INTEGRATION_NAME}/${INTEGRATION_VERSION}`)
.set('kbn-xsrf', 'xxxx');
.set('kbn-xsrf', 'xxxx')
.send({ force: true });
};
describe('Installing custom integrations', async () => {
@ -36,7 +37,7 @@ export default function (providerContext: FtrProviderContext) {
.type('application/json')
.send({
force: true,
integrationName: 'my_custom_nginx',
integrationName: INTEGRATION_NAME,
datasets: [
{ name: 'access', type: 'logs' },
{ name: 'error', type: 'metrics' },
@ -46,22 +47,22 @@ export default function (providerContext: FtrProviderContext) {
.expect(200);
const expectedIngestPipelines = [
'logs-my_custom_nginx.access-1.0.0',
'metrics-my_custom_nginx.error-1.0.0',
'logs-my_custom_nginx.warning-1.0.0',
`logs-${INTEGRATION_NAME}.access-1.0.0`,
`metrics-${INTEGRATION_NAME}.error-1.0.0`,
`logs-${INTEGRATION_NAME}.warning-1.0.0`,
];
const expectedIndexTemplates = [
'logs-my_custom_nginx.access',
'metrics-my_custom_nginx.error',
'logs-my_custom_nginx.warning',
`logs-${INTEGRATION_NAME}.access`,
`metrics-${INTEGRATION_NAME}.error`,
`logs-${INTEGRATION_NAME}.warning`,
];
const expectedComponentTemplates = [
'logs-my_custom_nginx.access@package',
'logs-my_custom_nginx.access@custom',
'metrics-my_custom_nginx.error@package',
'metrics-my_custom_nginx.error@custom',
'logs-my_custom_nginx.warning@package',
'logs-my_custom_nginx.warning@custom',
`logs-${INTEGRATION_NAME}.access@package`,
`logs-${INTEGRATION_NAME}.access@custom`,
`metrics-${INTEGRATION_NAME}.error@package`,
`metrics-${INTEGRATION_NAME}.error@custom`,
`logs-${INTEGRATION_NAME}.warning@package`,
`logs-${INTEGRATION_NAME}.warning@custom`,
];
expect(response.body._meta.install_source).to.be('custom');
@ -92,11 +93,65 @@ export default function (providerContext: FtrProviderContext) {
type: PACKAGES_SAVED_OBJECT_TYPE,
id: INTEGRATION_NAME,
});
expect(installation.attributes.name).to.be(INTEGRATION_NAME);
expect(installation.attributes.version).to.be(INTEGRATION_VERSION);
expect(installation.attributes.install_source).to.be('custom');
expect(installation.attributes.install_status).to.be('installed');
});
it('Throws an error when there is a naming collision with a current package installation', async () => {
await supertest
.post(`/api/fleet/epm/custom_integrations`)
.set('kbn-xsrf', 'xxxx')
.type('application/json')
.send({
force: true,
integrationName: INTEGRATION_NAME,
datasets: [
{ name: 'access', type: 'logs' },
{ name: 'error', type: 'metrics' },
{ name: 'warning', type: 'logs' },
],
})
.expect(200);
const response = await supertest
.post(`/api/fleet/epm/custom_integrations`)
.set('kbn-xsrf', 'xxxx')
.type('application/json')
.send({
force: true,
integrationName: INTEGRATION_NAME,
datasets: [
{ name: 'access', type: 'logs' },
{ name: 'error', type: 'metrics' },
{ name: 'warning', type: 'logs' },
],
})
.expect(409);
expect(response.body.message).to.be(
`Failed to create the integration as an installation with the name ${INTEGRATION_NAME} already exists.`
);
});
it('Throws an error when there is a naming collision with a registry package', async () => {
const pkgName = 'apache';
const response = await supertest
.post(`/api/fleet/epm/custom_integrations`)
.set('kbn-xsrf', 'xxxx')
.type('application/json')
.send({
force: true,
integrationName: pkgName,
datasets: [{ name: 'error', type: 'logs' }],
})
.expect(409);
expect(response.body.message).to.be(
`Failed to create the integration as an integration with the name ${pkgName} already exists in the package registry or as a bundled package.`
);
});
});
}

View file

@ -5884,6 +5884,10 @@
version "0.0.0"
uid ""
"@kbn/use-tracked-promise@link:packages/kbn-use-tracked-promise":
version "0.0.0"
uid ""
"@kbn/user-profile-components@link:packages/kbn-user-profile-components":
version "0.0.0"
uid ""