mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 09:48:58 -04:00
[Onboarding] AWS Firehose Flow (#187904)
Closes https://github.com/elastic/kibana/issues/187249 Closes https://github.com/elastic/kibana/issues/190792 Closes https://github.com/elastic/kibana/issues/190793 This change adds a new Firehose quickstart flow and replaces the old one in the onboarding recommendations. The new card is only visible on Cloud deployments, but you can still access the flow locally via `app/observabilityOnboarding/firehose` path if needed.  ### How to test You going to need an AWS account. You can use just a personal one or "Elastic Observability" account which you can access through Okta (type "AWS" in Okta's search and you should see "AWS - Elastic Observability"). In case you decide to use the shared "Elastic Observability" account, make sure it does not already has `Elastic-CloudwatchLogsAndMetricsToFirehose` CloudFormation stack left from the previous tester. Feel free to delete it if it's there. You also need to [install and setup AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). In case you use "Elastic Observability" account here is [the instruction to get CLI credentials.](https://github.com/elastic/observability-dev/blob/main/docs/how-we-work/aws-onboarding.md#programmatic-access-for-developers). 1. In AWS account, create a few entities that generate logs and put them into a CloudWatch log group (see instructions below for a few services). 2. Generate some logs by accessing the entities that you've created and make sure they appear in CloudWatch (mind that there is a ~1 minute delay). **If you don't see anything in CloudWatch, there is no point in proceeding further, make sure to fix your AWS setup before starting the flow in Kibana.** 3. Go to the [serverless Kibana instance deployed from this PR](https://issue-serverless-skhgw-pr187904-e0f30a.kb.eu-west-1.aws.qa.elastic.cloud) 4. Add Data → Collect and analyze logs → View AWS Collection → Firehose quickstart 5. Open the Firehose flow and copy the command snippet 6. Run the command in your terminal. It should succeed and output the new Stack ID. 7. Wait for the stack to finish creating. The quickstart flow screen in Kibana has a command to check current status or you can monitor it directly in AWS. 8. Generate some some logs by accessing the AWS services you've created. 9. Go back to the Kibana screen, after a minute or so incoming logs should be detected and corresponding AWS service will be highlighted. 10. Expand one of the detected services and navigate the the suggested dashboard, make sure you see some data there. ### Example AWS Services Configs **Before creating any resources, make sure you're in the same region (top right corner in AWS Console) you've used while configuring AWS CLI.** #### API Gateway 1. [Create an IAM role](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-logging.html#set-up-access-logging-permissions) to grant API Gateway permissions to write into a CloudWatch log groups 1. Copy the ARN of the created role 1. Open "CloudWatch" in AWS and select "Log groups" in the sidebar 1. Create a new log group with default parameters 1. Copy the ARN of the new group 1. Open **API Gateway** in AWS 1. Navigate to "Settings" in the sidebar 1. In the "Logging" section click "Edit" and paste the ARN of the IAM role you created in step 1. Hit "Save changes" 1. Now go back to "APIs" in the sidebar and click "Create API" 1. In "REST API" click "Build" 1. Select "Example API" and click "Create API" 1. Click on "Deploy API" 1. For "Stage" dropdown select "New stage", give it any name and click "Deploy" 1. You will now see "Invoke URL", you can use it later to access this API and generate logs 1. Scroll to "Logs and tracing" section and click "Edit" 1. In the dropdown select "Full request and response logs" 1. Toggle "Custom access logging" 1. Paste the ARN of the CloudWatch log group you've created in step 4. But make sure to not include ":*" symbols at the end. 1. In the log format input paste [this format from our docs](https://www.elastic.co/docs/current/integrations/aws/apigateway#data-streams) and click "Save" ``` {"requestId":"$context.requestId","ip":"$context.identity.sourceIp","caller":"$context.identity.caller","user":"$context.identity.user","requestTime":"$context.requestTime","httpMethod":"$context.httpMethod","resourcePath":"$context.resourcePath","status":"$context.status","protocol":"$context.protocol","responseLength":"$context.responseLength","apiId":"$context.apiId","domainName":"$context.domainName","stage":"$context.stage"} ``` 1. Now when you access this API, you should see logs coming into the CloudWatch group. #### WAF **This sets up WAF for an API Gateway, see above if you don't have one already.** 1. Open WAF in AWS 3. Click "Web ACLs" in the sidebar 4. Click "Create web ACL" 5. Select the region where you've created your API Gateway and give ACL any name 6. In the "Associated AWS resources" section click "Add AWS resources" 7. Select you API Gateway and click "Add" 8. Click "Next" 9. Create some basic rule, for example to block requests that have a specific parameter in the URL 10. Click through the other configuration step leaving everything as is and then finally click "Create web ACL" 11. Select the created ACL and click on the "Logging and metrics" tab 12. Click "Edit" in "Logging" section 13. Click "Create new" in the "Amazon CloudWatch Logs log group" section 14. Create a new log group. **The log group name should start with `aws-waf-logs-`**. 15. Select the new group in the dropdown and click "Save" 16. Now you should have logs generated and saved into the log group when you access your API gateway #### VPC 1. [Create an IAM role](https://docs.aws.amazon.com/vpc/latest/tgw/flow-logs-cwl.html#flow-logs-iam) to write flow logs to CloudWatch log groups. 3. Create and EC2 instance and configure there some HTTP server like Nginx. Using Docker would probably be the fastest way. 4. Create a CloudWatch log group with default parameters 5. Open "VPC" in AWS and select the VPC where you've created the EC2 instance. 6. Click the "Flow logs" tab and click "Create flow logs" 7. In "Maximum aggregation interval" select 1 minute to see logs faster 8. In "Destination log group" select the log group you've created in step 3 9. In "IAM role" select the role you've created in step 1 10. Click "Create flow log" 11. Now when you access your EC2 instance, you should see logs in the CloudWatch log group
This commit is contained in:
Mykola Harmash
GitHub
parent
24e1b67027
commit
8a429b5953
27 changed files with 1184 additions and 26 deletions
|
|
@ -48,6 +48,7 @@ export type LogDocument = Fields &
|
|||
'orchestrator.resource.id'?: string;
|
||||
'kubernetes.pod.uid'?: string;
|
||||
'aws.s3.bucket.name'?: string;
|
||||
'aws.kinesis.name'?: string;
|
||||
'orchestrator.namespace'?: string;
|
||||
'container.name'?: string;
|
||||
'cloud.provider'?: string;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,49 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
export const FIREHOSE_CLOUDFORMATION_STACK_NAME = 'Elastic-CloudwatchLogsAndMetricsToFirehose';
|
||||
export const FIREHOSE_LOGS_STREAM_NAME = 'Elastic-CloudwatchLogs';
|
||||
export const FIREHOSE_METRICS_STREAM_NAME = 'Elastic-CloudwatchMetrics';
|
||||
|
||||
export const FIREHOSE_CLOUDFORMATION_TEMPLATE_URL =
|
||||
'https://elastic-cloudformation-templates.s3.amazonaws.com/v0.1.0/firehose_default_start.yml';
|
||||
|
||||
export const AWS_INDEX_NAME_LIST = [
|
||||
// Logs
|
||||
'logs-aws.vpcflow',
|
||||
'logs-aws.apigateway_logs',
|
||||
'logs-aws.cloudtrail',
|
||||
'logs-aws.firewall_logs',
|
||||
'logs-aws.route53_public_logs',
|
||||
'logs-aws.route53_resolver_logs',
|
||||
'logs-aws.waf',
|
||||
'logs-awsfirehose',
|
||||
// Metrics
|
||||
'metrics-aws.apigateway_metrics',
|
||||
'metrics-aws.dynamodb',
|
||||
'metrics-aws.ebs',
|
||||
'metrics-aws.ec2_metrics',
|
||||
'metrics-aws.ecs_metrics',
|
||||
'metrics-aws.elb_metrics',
|
||||
'metrics-aws.emr_metrics',
|
||||
'metrics-aws.firewall_metrics',
|
||||
'metrics-aws.kafka_metrics',
|
||||
'metrics-aws.kinesis',
|
||||
'metrics-aws.lambda',
|
||||
'metrics-aws.natgateway',
|
||||
'metrics-aws.rds',
|
||||
'metrics-aws.s3_storage_lens',
|
||||
'metrics-aws.s3_daily_storage',
|
||||
'metrics-aws.s3_request',
|
||||
'metrics-aws.sns',
|
||||
'metrics-aws.sqs',
|
||||
'metrics-aws.transitgateway',
|
||||
'metrics-aws.usage',
|
||||
'metrics-aws.vpn',
|
||||
] as const;
|
||||
|
||||
export type AWSIndexName = (typeof AWS_INDEX_NAME_LIST)[number];
|
||||
|
|
@ -9,6 +9,7 @@ import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
|
|||
import React, { useEffect } from 'react';
|
||||
import { Route, Routes } from '@kbn/shared-ux-router';
|
||||
import { useLocation } from 'react-router-dom-v5-compat';
|
||||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import {
|
||||
AutoDetectPage,
|
||||
CustomLogsPage,
|
||||
|
|
@ -16,12 +17,19 @@ import {
|
|||
LandingPage,
|
||||
OtelLogsPage,
|
||||
SystemLogsPage,
|
||||
FirehosePage,
|
||||
} from './pages';
|
||||
import { ObservabilityOnboardingAppServices } from '..';
|
||||
|
||||
const queryClient = new QueryClient();
|
||||
|
||||
export function ObservabilityOnboardingFlow() {
|
||||
const { pathname } = useLocation();
|
||||
const {
|
||||
services: {
|
||||
context: { isDev, isCloud },
|
||||
},
|
||||
} = useKibana<ObservabilityOnboardingAppServices>();
|
||||
|
||||
useEffect(() => {
|
||||
window.scrollTo(0, 0);
|
||||
|
|
@ -45,6 +53,11 @@ export function ObservabilityOnboardingFlow() {
|
|||
<Route path="/otel-logs">
|
||||
<OtelLogsPage />
|
||||
</Route>
|
||||
{(isCloud || isDev) && (
|
||||
<Route path="/firehose">
|
||||
<FirehosePage />
|
||||
</Route>
|
||||
)}
|
||||
<Route>
|
||||
<LandingPage />
|
||||
</Route>
|
||||
|
|
|
|||
|
|
@ -24,12 +24,14 @@ import {
|
|||
} from '@elastic/eui';
|
||||
|
||||
import { useSearchParams } from 'react-router-dom-v5-compat';
|
||||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import { OnboardingFlowPackageList } from '../packages_list';
|
||||
import { useCustomMargin } from '../shared/use_custom_margin';
|
||||
import { Category } from './types';
|
||||
import { useCustomCardsForCategory } from './use_custom_cards_for_category';
|
||||
import { useVirtualSearchResults } from './use_virtual_search_results';
|
||||
import { LogoIcon, SupportedLogo } from '../shared/logo_icon';
|
||||
import { ObservabilityOnboardingAppServices } from '../..';
|
||||
|
||||
interface UseCaseOption {
|
||||
id: Category;
|
||||
|
|
@ -90,6 +92,11 @@ export const OnboardingFlowForm: FunctionComponent = () => {
|
|||
},
|
||||
];
|
||||
|
||||
const {
|
||||
services: {
|
||||
context: { isCloud },
|
||||
},
|
||||
} = useKibana<ObservabilityOnboardingAppServices>();
|
||||
const customMargin = useCustomMargin();
|
||||
const radioGroupId = useGeneratedHtmlId({ prefix: 'onboardingCategory' });
|
||||
const categorySelectorTitleId = useGeneratedHtmlId();
|
||||
|
|
@ -134,6 +141,12 @@ export const OnboardingFlowForm: FunctionComponent = () => {
|
|||
searchParams.get('category') as Category | null
|
||||
);
|
||||
const virtualSearchResults = useVirtualSearchResults();
|
||||
/**
|
||||
* Cloud deployments have the new Firehose quick start
|
||||
* flow enabled, so the ond card 'epr:awsfirehose' should
|
||||
* not show up in the search results.
|
||||
*/
|
||||
const searchExcludePackageIdList = isCloud ? ['epr:awsfirehose'] : [];
|
||||
|
||||
let isSelectingCategoryWithKeyboard: boolean = false;
|
||||
|
||||
|
|
@ -264,6 +277,7 @@ export const OnboardingFlowForm: FunctionComponent = () => {
|
|||
(card) => card.type === 'virtual' && !card.isCollectionCard
|
||||
)
|
||||
.concat(virtualSearchResults)}
|
||||
excludePackageIdList={searchExcludePackageIdList}
|
||||
joinCardLists
|
||||
/>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -6,14 +6,48 @@
|
|||
*/
|
||||
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import { useHistory } from 'react-router-dom';
|
||||
import { reactRouterNavigate, useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import { CustomCard } from '../packages_list/types';
|
||||
import { ObservabilityOnboardingAppServices } from '../..';
|
||||
|
||||
export function useVirtualSearchResults(): CustomCard[] {
|
||||
const {
|
||||
services: { application },
|
||||
} = useKibana();
|
||||
services: {
|
||||
application,
|
||||
context: { isCloud },
|
||||
},
|
||||
} = useKibana<ObservabilityOnboardingAppServices>();
|
||||
|
||||
const history = useHistory();
|
||||
const { href: firehoseUrl } = reactRouterNavigate(history, `/firehose/${location.search}`);
|
||||
const getUrlForApp = application?.getUrlForApp;
|
||||
const firehoseQuickstartCard: CustomCard = {
|
||||
id: 'firehose-quick-start',
|
||||
type: 'virtual',
|
||||
title: i18n.translate('xpack.observability_onboarding.packageList.uploadFileTitle', {
|
||||
defaultMessage: 'AWS Firehose',
|
||||
}),
|
||||
release: 'preview',
|
||||
description: i18n.translate(
|
||||
'xpack.observability_onboarding.packageList.uploadFileDescription',
|
||||
{
|
||||
defaultMessage: 'Collect logs and metrics from Amazon Web Services (AWS).',
|
||||
}
|
||||
),
|
||||
name: 'firehose-quick-start',
|
||||
categories: [],
|
||||
icons: [
|
||||
{
|
||||
type: 'svg',
|
||||
src: 'https://epr.elastic.co/package/awsfirehose/1.1.0/img/logo_firehose.svg',
|
||||
},
|
||||
],
|
||||
url: firehoseUrl,
|
||||
version: '',
|
||||
integration: '',
|
||||
isCollectionCard: false,
|
||||
};
|
||||
|
||||
return [
|
||||
{
|
||||
|
|
@ -42,5 +76,11 @@ export function useVirtualSearchResults(): CustomCard[] {
|
|||
integration: '',
|
||||
isCollectionCard: false,
|
||||
},
|
||||
/**
|
||||
* The new Firehose card should only be visible on Cloud
|
||||
* as Firehose integration requires additional proxy,
|
||||
* which is not available for on-prem customers.
|
||||
*/
|
||||
...(isCloud ? [firehoseQuickstartCard] : []),
|
||||
];
|
||||
}
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ interface Props {
|
|||
* When enabled, custom and integration cards are joined into a single list.
|
||||
*/
|
||||
joinCardLists?: boolean;
|
||||
excludePackageIdList?: string[];
|
||||
onLoaded?: () => void;
|
||||
}
|
||||
|
||||
|
|
@ -58,6 +59,7 @@ const PackageListGridWrapper = ({
|
|||
flowCategory,
|
||||
flowSearch,
|
||||
joinCardLists = false,
|
||||
excludePackageIdList = [],
|
||||
onLoaded,
|
||||
}: WrapperProps) => {
|
||||
const customMargin = useCustomMargin();
|
||||
|
|
@ -68,6 +70,7 @@ const PackageListGridWrapper = ({
|
|||
const list: IntegrationCardItem[] = useIntegrationCardList(
|
||||
filteredCards,
|
||||
selectedCategory,
|
||||
excludePackageIdList,
|
||||
customCards,
|
||||
flowCategory,
|
||||
flowSearch,
|
||||
|
|
|
|||
|
|
@ -80,10 +80,12 @@ function useFilteredCards(
|
|||
rewriteUrl: (card: IntegrationCardItem) => IntegrationCardItem,
|
||||
integrationsList: IntegrationCardItem[],
|
||||
selectedCategory: string[],
|
||||
excludePackageIdList: string[],
|
||||
customCards?: CustomCard[]
|
||||
) {
|
||||
return useMemo(() => {
|
||||
const integrationCards = integrationsList
|
||||
.filter((card) => !excludePackageIdList.includes(card.id))
|
||||
.filter((card) => card.categories.some((category) => selectedCategory.includes(category)))
|
||||
.map(rewriteUrl)
|
||||
.map(toCustomCard);
|
||||
|
|
@ -99,7 +101,7 @@ function useFilteredCards(
|
|||
),
|
||||
integrationCards,
|
||||
};
|
||||
}, [integrationsList, customCards, selectedCategory, rewriteUrl]);
|
||||
}, [integrationsList, rewriteUrl, customCards, excludePackageIdList, selectedCategory]);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -113,6 +115,7 @@ function useFilteredCards(
|
|||
export function useIntegrationCardList(
|
||||
integrationsList: IntegrationCardItem[],
|
||||
selectedCategory: string[],
|
||||
excludePackageIdList: string[],
|
||||
customCards?: CustomCard[],
|
||||
flowCategory?: string | null,
|
||||
flowSearch?: string,
|
||||
|
|
@ -123,6 +126,7 @@ export function useIntegrationCardList(
|
|||
rewriteUrl,
|
||||
integrationsList,
|
||||
selectedCategory,
|
||||
excludePackageIdList,
|
||||
customCards
|
||||
);
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,11 @@
|
|||
|
||||
import { IntegrationCardItem } from '@kbn/fleet-plugin/public';
|
||||
|
||||
export const QUICKSTART_FLOWS = ['auto-detect-logs-virtual', 'kubernetes-quick-start'];
|
||||
export const QUICKSTART_FLOWS = [
|
||||
'auto-detect-logs-virtual',
|
||||
'kubernetes-quick-start',
|
||||
'firehose-quick-start',
|
||||
];
|
||||
|
||||
export const toCustomCard = (card: IntegrationCardItem) => ({
|
||||
...card,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,37 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import React from 'react';
|
||||
import { FirehosePanel } from '../quickstart_flows/firehose';
|
||||
import { PageTemplate } from './template';
|
||||
import { CustomHeader } from '../header';
|
||||
|
||||
export const FirehosePage = () => (
|
||||
<PageTemplate
|
||||
customHeader={
|
||||
<CustomHeader
|
||||
logo="firehose"
|
||||
headlineCopy={i18n.translate(
|
||||
'xpack.observability_onboarding.experimentalOnboardingFlow.customHeader.firehose.text',
|
||||
{
|
||||
defaultMessage: 'Setting up Amazon Data Firehose',
|
||||
}
|
||||
)}
|
||||
captionCopy={i18n.translate(
|
||||
'xpack.observability_onboarding.experimentalOnboardingFlow.customHeader.firehose.caption.description',
|
||||
{
|
||||
defaultMessage:
|
||||
'This installation is tailored for setting up Firehose in your Observability project with minimal configuration.',
|
||||
}
|
||||
)}
|
||||
/>
|
||||
}
|
||||
>
|
||||
<FirehosePanel />
|
||||
</PageTemplate>
|
||||
);
|
||||
|
|
@ -11,3 +11,4 @@ export { KubernetesPage } from './kubernetes';
|
|||
export { LandingPage } from './landing';
|
||||
export { OtelLogsPage } from './otel_logs';
|
||||
export { SystemLogsPage } from './system_logs';
|
||||
export { FirehosePage } from './firehose';
|
||||
|
|
|
|||
|
|
@ -24,6 +24,8 @@ import {
|
|||
type SingleDatasetLocatorParams,
|
||||
SINGLE_DATASET_LOCATOR_ID,
|
||||
} from '@kbn/deeplinks-observability/locators';
|
||||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import { DASHBOARD_APP_LOCATOR } from '@kbn/deeplinks-analytics';
|
||||
import { getAutoDetectCommand } from './get_auto_detect_command';
|
||||
import { DASHBOARDS, useOnboardingFlow } from './use_onboarding_flow';
|
||||
import { ProgressIndicator } from '../shared/progress_indicator';
|
||||
|
|
@ -34,12 +36,16 @@ import { LocatorButtonEmpty } from '../shared/locator_button_empty';
|
|||
import { GetStartedPanel } from '../shared/get_started_panel';
|
||||
import { isSupportedLogo, LogoIcon } from '../../shared/logo_icon';
|
||||
import { FeedbackButtons } from '../shared/feedback_buttons';
|
||||
import { ObservabilityOnboardingContextValue } from '../../../plugin';
|
||||
import { useAutoDetectTelemetry } from './use_auto_detect_telemetry';
|
||||
|
||||
export const AutoDetectPanel: FunctionComponent = () => {
|
||||
const { status, data, error, refetch, installedIntegrations } = useOnboardingFlow();
|
||||
const command = data ? getAutoDetectCommand(data) : undefined;
|
||||
const accordionId = useGeneratedHtmlId({ prefix: 'accordion' });
|
||||
const {
|
||||
services: { share },
|
||||
} = useKibana<ObservabilityOnboardingContextValue>();
|
||||
|
||||
useAutoDetectTelemetry(
|
||||
status,
|
||||
|
|
@ -61,6 +67,7 @@ export const AutoDetectPanel: FunctionComponent = () => {
|
|||
const customIntegrations = installedIntegrations.filter(
|
||||
(integration) => integration.installSource === 'custom'
|
||||
);
|
||||
const dashboardLocator = share.url.locators.get(DASHBOARD_APP_LOCATOR);
|
||||
|
||||
return (
|
||||
<EuiPanel hasBorder paddingSize="xl">
|
||||
|
|
@ -173,10 +180,14 @@ export const AutoDetectPanel: FunctionComponent = () => {
|
|||
integration={integration.pkgName}
|
||||
newTab
|
||||
isLoading={status !== 'dataReceived'}
|
||||
dashboardLinks={integration.kibanaAssets
|
||||
actionLinks={integration.kibanaAssets
|
||||
.filter((asset) => asset.type === 'dashboard')
|
||||
.map((asset) => {
|
||||
const dashboard = DASHBOARDS[asset.id as keyof typeof DASHBOARDS];
|
||||
const href =
|
||||
dashboardLocator?.getRedirectUrl({
|
||||
dashboardId: asset.id,
|
||||
}) ?? '';
|
||||
|
||||
return {
|
||||
id: asset.id,
|
||||
|
|
@ -210,6 +221,7 @@ export const AutoDetectPanel: FunctionComponent = () => {
|
|||
defaultMessage: 'Explore logs data',
|
||||
}
|
||||
),
|
||||
href,
|
||||
};
|
||||
})}
|
||||
/>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,105 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import React from 'react';
|
||||
import {
|
||||
EuiAccordion,
|
||||
EuiCodeBlock,
|
||||
EuiLink,
|
||||
EuiSpacer,
|
||||
EuiText,
|
||||
useGeneratedHtmlId,
|
||||
} from '@elastic/eui';
|
||||
import { FormattedMessage } from '@kbn/i18n-react';
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import {
|
||||
FIREHOSE_CLOUDFORMATION_STACK_NAME,
|
||||
FIREHOSE_LOGS_STREAM_NAME,
|
||||
FIREHOSE_METRICS_STREAM_NAME,
|
||||
} from '../../../../common/aws_firehose';
|
||||
import { CopyToClipboardButton } from '../shared/copy_to_clipboard_button';
|
||||
import { buildCreateStackCommand, buildStackStatusCommand } from './utils';
|
||||
|
||||
interface Props {
|
||||
encodedApiKey: string;
|
||||
onboardingId: string;
|
||||
elasticsearchUrl: string;
|
||||
templateUrl: string;
|
||||
isCopyPrimaryAction: boolean;
|
||||
}
|
||||
|
||||
export function CreateStackCommandSnippet({
|
||||
encodedApiKey,
|
||||
elasticsearchUrl,
|
||||
templateUrl,
|
||||
isCopyPrimaryAction,
|
||||
}: Props) {
|
||||
const stackStatusAccordionId = useGeneratedHtmlId({ prefix: 'stackStatusAccordion' });
|
||||
const createStackCommand = buildCreateStackCommand({
|
||||
templateUrl,
|
||||
stackName: FIREHOSE_CLOUDFORMATION_STACK_NAME,
|
||||
logsStreamName: FIREHOSE_LOGS_STREAM_NAME,
|
||||
metricsStreamName: FIREHOSE_METRICS_STREAM_NAME,
|
||||
encodedApiKey,
|
||||
elasticsearchUrl,
|
||||
});
|
||||
const stackStatusCommand = buildStackStatusCommand({
|
||||
stackName: FIREHOSE_CLOUDFORMATION_STACK_NAME,
|
||||
});
|
||||
|
||||
return (
|
||||
<>
|
||||
<EuiText>
|
||||
<p>
|
||||
<FormattedMessage
|
||||
id="xpack.observability_onboarding.firehosePanel.createFirehoseStreamDescription"
|
||||
defaultMessage="Run the command bellow in your terminal where you have {awsCLIInstallGuideLink} configured. The command will create a CloudFormation stack that includes a Firehose delivery, backup S3 bucket, CloudWatch subscription filter and metrics stream along with required IAM roles."
|
||||
values={{
|
||||
awsCLIInstallGuideLink: (
|
||||
<EuiLink
|
||||
data-test-subj="observabilityOnboardingFirehosePanelAwsInstallGuideLink"
|
||||
href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html"
|
||||
external
|
||||
target="_blank"
|
||||
>
|
||||
{i18n.translate(
|
||||
'xpack.observability_onboarding.firehosePanel.awsCLIInstallGuideLinkLabel',
|
||||
{ defaultMessage: 'AWS CLI' }
|
||||
)}
|
||||
</EuiLink>
|
||||
),
|
||||
}}
|
||||
/>
|
||||
</p>
|
||||
</EuiText>
|
||||
|
||||
<EuiSpacer />
|
||||
|
||||
<EuiCodeBlock
|
||||
language="text"
|
||||
paddingSize="m"
|
||||
fontSize="m"
|
||||
data-test-subj="observabilityOnboardingFirehoseCreateStackCommand"
|
||||
>
|
||||
{createStackCommand}
|
||||
</EuiCodeBlock>
|
||||
|
||||
<EuiSpacer />
|
||||
|
||||
<CopyToClipboardButton textToCopy={createStackCommand} fill={isCopyPrimaryAction} />
|
||||
|
||||
<EuiSpacer />
|
||||
|
||||
<EuiAccordion id={stackStatusAccordionId} buttonContent="Check stack status">
|
||||
<EuiSpacer size="xs" />
|
||||
<EuiCodeBlock language="text" paddingSize="m" fontSize="m" isCopyable>
|
||||
{stackStatusCommand}
|
||||
</EuiCodeBlock>
|
||||
</EuiAccordion>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import React, { useState } from 'react';
|
||||
import {
|
||||
EuiPanel,
|
||||
EuiSkeletonRectangle,
|
||||
EuiSkeletonText,
|
||||
EuiSpacer,
|
||||
EuiSteps,
|
||||
EuiStepStatus,
|
||||
} from '@elastic/eui';
|
||||
import useEvent from 'react-use/lib/useEvent';
|
||||
import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher';
|
||||
import { EmptyPrompt } from '../shared/empty_prompt';
|
||||
import { CreateStackCommandSnippet } from './create_stack_command_snippet';
|
||||
import { VisualizeData } from './visualize_data';
|
||||
|
||||
export function FirehosePanel() {
|
||||
const [windowLostFocus, setWindowLostFocus] = useState(false);
|
||||
const { data, status, error, refetch } = useFetcher(
|
||||
(callApi) => {
|
||||
return callApi('POST /internal/observability_onboarding/firehose/flow');
|
||||
},
|
||||
[],
|
||||
{ showToastOnError: false }
|
||||
);
|
||||
|
||||
useEvent('blur', () => setWindowLostFocus(true), window);
|
||||
|
||||
if (error !== undefined) {
|
||||
return <EmptyPrompt error={error} onRetryClick={refetch} />;
|
||||
}
|
||||
|
||||
const isVisualizeStepActive =
|
||||
status === FETCH_STATUS.SUCCESS && data !== undefined && windowLostFocus;
|
||||
|
||||
const steps = [
|
||||
{
|
||||
title: 'Create a Firehose delivery stream and ingest CloudWatch logs',
|
||||
children: (
|
||||
<>
|
||||
{status !== FETCH_STATUS.SUCCESS && (
|
||||
<>
|
||||
<EuiSkeletonText lines={5} />
|
||||
<EuiSpacer />
|
||||
<EuiSkeletonRectangle width="170px" height="40px" />
|
||||
</>
|
||||
)}
|
||||
{status === FETCH_STATUS.SUCCESS && data !== undefined && (
|
||||
<CreateStackCommandSnippet
|
||||
templateUrl={data.templateUrl}
|
||||
encodedApiKey={data.apiKeyEncoded}
|
||||
onboardingId={data.onboardingId}
|
||||
elasticsearchUrl={data.elasticsearchUrl}
|
||||
isCopyPrimaryAction={!isVisualizeStepActive}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
),
|
||||
},
|
||||
{
|
||||
title: 'Visualize your data',
|
||||
status: (isVisualizeStepActive ? 'current' : 'incomplete') as EuiStepStatus,
|
||||
children: isVisualizeStepActive && <VisualizeData />,
|
||||
},
|
||||
];
|
||||
|
||||
return (
|
||||
<EuiPanel hasBorder paddingSize="xl">
|
||||
<EuiSteps steps={steps} />
|
||||
</EuiPanel>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,368 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { useCallback, useMemo } from 'react';
|
||||
import { DASHBOARD_APP_LOCATOR } from '@kbn/deeplinks-analytics';
|
||||
import { SINGLE_DATASET_LOCATOR_ID } from '@kbn/deeplinks-observability';
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import { DISCOVER_APP_LOCATOR } from '@kbn/discover-plugin/common';
|
||||
import { AWSIndexName } from '../../../../common/aws_firehose';
|
||||
import { ObservabilityOnboardingContextValue } from '../../../plugin';
|
||||
|
||||
interface AWSServiceGetStartedConfig {
|
||||
id: string;
|
||||
indexNameList: AWSIndexName[];
|
||||
title: string;
|
||||
logoURL: string;
|
||||
previewImage?: string;
|
||||
actionLinks: Array<{
|
||||
id: string;
|
||||
title: string;
|
||||
label: string;
|
||||
href: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
export function useAWSServiceGetStartedList(): AWSServiceGetStartedConfig[] {
|
||||
const {
|
||||
services: { share },
|
||||
} = useKibana<ObservabilityOnboardingContextValue>();
|
||||
const dashboardLocator = share.url.locators.get(DASHBOARD_APP_LOCATOR);
|
||||
const singleDatasetLocator = share.url.locators.get(SINGLE_DATASET_LOCATOR_ID);
|
||||
const discoverLocator = share.url.locators.get(DISCOVER_APP_LOCATOR);
|
||||
|
||||
const generateMetricsDashboardActionLink = useCallback(
|
||||
(dashboardId: string, name?: string) => ({
|
||||
id: `dashboard-${dashboardId}`,
|
||||
title: i18n.translate(
|
||||
'xpack.observability_onboarding.firehosePanel.exploreMetricsDataTitle',
|
||||
{
|
||||
defaultMessage: 'Overview{name} metrics data with this pre-made dashboard',
|
||||
values: { name: name ? ` ${name}` : '' },
|
||||
}
|
||||
),
|
||||
label: i18n.translate(
|
||||
'xpack.observability_onboarding.firehosePanel.exploreMetricsDataLabel',
|
||||
{
|
||||
defaultMessage: 'Explore metrics data',
|
||||
}
|
||||
),
|
||||
href:
|
||||
dashboardLocator?.getRedirectUrl({
|
||||
dashboardId,
|
||||
}) ?? '',
|
||||
}),
|
||||
[dashboardLocator]
|
||||
);
|
||||
|
||||
const generateLogsDashboardActionLink = useCallback(
|
||||
(dashboardId: string) => ({
|
||||
id: `dashboard-${dashboardId}`,
|
||||
title: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreLogsDataTitle', {
|
||||
defaultMessage: 'Overview your logs data with this pre-made dashboard',
|
||||
}),
|
||||
label: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreLogsDataLabel', {
|
||||
defaultMessage: 'Explore logs data',
|
||||
}),
|
||||
href:
|
||||
dashboardLocator?.getRedirectUrl({
|
||||
dashboardId,
|
||||
}) ?? '',
|
||||
}),
|
||||
[dashboardLocator]
|
||||
);
|
||||
|
||||
const generateLogsExplorerActionLink = useCallback(
|
||||
(dataset: string, name: string) => ({
|
||||
id: `logs-explorer-${dataset}`,
|
||||
title: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreDataTitle', {
|
||||
defaultMessage: 'See {name} data in Logs Explorer',
|
||||
values: { name },
|
||||
}),
|
||||
label: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreDataLabel', {
|
||||
defaultMessage: 'Explore',
|
||||
}),
|
||||
href:
|
||||
singleDatasetLocator?.getRedirectUrl({
|
||||
integration: 'AWS',
|
||||
dataset,
|
||||
}) ?? '',
|
||||
}),
|
||||
[singleDatasetLocator]
|
||||
);
|
||||
|
||||
const generateMetricsDiscoverActionLink = useCallback(
|
||||
(namespace: string, name: string) => ({
|
||||
id: `logs-explorer-${namespace}`,
|
||||
title: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreDataTitle', {
|
||||
defaultMessage: 'See {name} metrics data in Discover',
|
||||
values: { name },
|
||||
}),
|
||||
label: i18n.translate('xpack.observability_onboarding.firehosePanel.exploreDataLabel', {
|
||||
defaultMessage: 'Explore',
|
||||
}),
|
||||
href:
|
||||
discoverLocator?.getRedirectUrl({
|
||||
dataViewId: `metrics-*`,
|
||||
query: {
|
||||
query: `aws.cloudwatch.namespace: ${namespace}`,
|
||||
language: 'kuery',
|
||||
},
|
||||
}) ?? '',
|
||||
}),
|
||||
[discoverLocator]
|
||||
);
|
||||
|
||||
return useMemo(
|
||||
() => [
|
||||
{
|
||||
id: 'vpc-flow',
|
||||
indexNameList: ['logs-aws.vpcflow'],
|
||||
title: 'VPC',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.21.0/img/logo_vpcflow.svg',
|
||||
actionLinks: [generateLogsDashboardActionLink('aws-15503340-4488-11ea-ad63-791a5dc86f10')],
|
||||
},
|
||||
{
|
||||
id: 'api-gateway',
|
||||
indexNameList: ['logs-aws.apigateway_logs', 'metrics-aws.apigateway_metrics'],
|
||||
title: 'API Gateway',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.21.0/img/logo_apigateway.svg',
|
||||
actionLinks: [
|
||||
generateLogsDashboardActionLink('aws-5465f0f0-26e4-11ee-9051-011d57d86fe2'),
|
||||
generateMetricsDashboardActionLink('aws-bff88770-56d6-11ee-893f-c96e4c6c871e'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'cloudtrail',
|
||||
indexNameList: ['logs-aws.cloudtrail'],
|
||||
title: 'CloudTrail',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.21.0/img/logo_cloudtrail.svg',
|
||||
actionLinks: [generateLogsDashboardActionLink('aws-9c09cd20-7399-11ea-a345-f985c61fe654')],
|
||||
},
|
||||
{
|
||||
id: 'firewall',
|
||||
indexNameList: ['logs-aws.firewall_logs'],
|
||||
title: 'Network Firewall',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.21.0/img/logo_firewall.svg',
|
||||
actionLinks: [
|
||||
generateLogsDashboardActionLink('aws-2ba11b50-4b9d-11ec-8282-5342b8988acc'),
|
||||
generateMetricsDashboardActionLink('aws-3abffe60-4ba9-11ec-8282-5342b8988acc'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'route53',
|
||||
indexNameList: ['logs-aws.route53_public_logs', 'logs-aws.route53_resolver_logs'],
|
||||
title: 'Route53',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.21.0/img/logo_route53.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateLogsExplorerActionLink('route53_public_logs', 'Route53 public'),
|
||||
generateLogsExplorerActionLink('route53_resolver_logs', 'Route53 resolver'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'waf',
|
||||
indexNameList: ['logs-aws.waf'],
|
||||
title: 'WAF',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.21.0/img/logo_waf.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [generateLogsExplorerActionLink('waf', 'WAF')],
|
||||
},
|
||||
{
|
||||
id: 'dynamodb',
|
||||
indexNameList: ['metrics-aws.dynamodb'],
|
||||
title: 'DynamoDB',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_dynamodb.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-68ba7bd0-20b6-11ea-8f72-2f8d21e50b0c'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'ebs',
|
||||
indexNameList: ['metrics-aws.ebs'],
|
||||
title: 'EBS',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_ebs.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-44ce4680-b7ba-11e9-8349-f15f850c5cd0'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'ec2',
|
||||
indexNameList: ['metrics-aws.ec2_metrics'],
|
||||
title: 'EC2',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_ec2.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-c5846400-f7fb-11e8-af03-c999c9dea608'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'ecs',
|
||||
indexNameList: ['metrics-aws.ecs_metrics'],
|
||||
title: 'ECS',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_ecs.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [generateMetricsDiscoverActionLink('AWS/ECS', 'ECS')],
|
||||
},
|
||||
{
|
||||
id: 'elb',
|
||||
indexNameList: ['metrics-aws.elb_metrics'],
|
||||
title: 'ELB',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_elb.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-e74bf320-b3ce-11e9-87a4-078dbbae220d'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'emr',
|
||||
indexNameList: ['metrics-aws.emr_metrics'],
|
||||
title: 'EMR',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_emr.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-98f85120-0ea4-11ee-9c37-e55025c0278a'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'msk',
|
||||
indexNameList: ['metrics-aws.kafka_metrics'],
|
||||
title: 'MSK',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_msk.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-62d43b00-d10d-11ee-b93f-db5ae1f208de'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'kinesis',
|
||||
indexNameList: ['metrics-aws.kinesis'],
|
||||
title: 'Kinesis Data Stream',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_kinesis.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-07d67a60-d872-11eb-8220-c9141cc1b15c'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'lambda',
|
||||
indexNameList: ['metrics-aws.lambda'],
|
||||
title: 'Lambda',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_lambda.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-7ac8e1d0-28d2-11ea-ba6c-49a884eb104f'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'nat-gateway',
|
||||
indexNameList: ['metrics-aws.natgateway'],
|
||||
title: 'NAT Gateway',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_natgateway.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-c2b1cbc0-6891-11ea-b0ac-95d4ecb1fecd'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'rds',
|
||||
indexNameList: ['metrics-aws.rds'],
|
||||
title: 'RDS',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_rds.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-3367c170-921f-11e9-aa19-159bf182e06f'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 's3',
|
||||
indexNameList: [
|
||||
'metrics-aws.s3_storage_lens',
|
||||
'metrics-aws.s3_daily_storage',
|
||||
'metrics-aws.s3_request',
|
||||
],
|
||||
title: 'S3',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_s3.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink(
|
||||
'aws-80ed1380-41a6-11ec-a605-bff67d9b7872',
|
||||
'S3 Storage Lens'
|
||||
),
|
||||
generateMetricsDiscoverActionLink('AWS/S3', 'S3'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'sns',
|
||||
indexNameList: ['metrics-aws.sns'],
|
||||
title: 'SNS',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_sns.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-d17b1000-17a4-11ea-8e91-03c7047cbb9d'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'sqs',
|
||||
indexNameList: ['metrics-aws.sqs'],
|
||||
title: 'SQS',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_sqs.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-234aeda0-43b7-11e9-8697-530f39afc6eb'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'transitgateway',
|
||||
indexNameList: ['metrics-aws.transitgateway'],
|
||||
title: 'Transit Gateway',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_transitgateway.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-0eb5a6a0-694f-11ea-b0ac-95d4ecb1fecd'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'usage',
|
||||
indexNameList: ['metrics-aws.usage'],
|
||||
title: 'AWS Usage',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_aws.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-917a07b0-178e-11ea-8650-fb606deb5be4'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'vpn',
|
||||
indexNameList: ['metrics-aws.vpn'],
|
||||
title: 'VPN',
|
||||
logoURL: 'https://epr.elastic.co/package/aws/2.23.0/img/logo_vpn.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [
|
||||
generateMetricsDashboardActionLink('aws-67c9f900-693e-11ea-b0ac-95d4ecb1fecd'),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'firehose',
|
||||
indexNameList: ['logs-awsfirehose'],
|
||||
title: 'Uncategorized Firehose Logs',
|
||||
logoURL: 'https://epr.elastic.co/package/awsfirehose/1.1.0/img/logo_firehose.svg',
|
||||
previewImage: 'waterfall_screen.svg',
|
||||
actionLinks: [generateLogsExplorerActionLink('awsfirehose', 'Firehose')],
|
||||
},
|
||||
],
|
||||
[
|
||||
generateLogsDashboardActionLink,
|
||||
generateLogsExplorerActionLink,
|
||||
generateMetricsDashboardActionLink,
|
||||
generateMetricsDiscoverActionLink,
|
||||
]
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
export function buildCreateStackCommand({
|
||||
templateUrl,
|
||||
stackName,
|
||||
logsStreamName,
|
||||
metricsStreamName,
|
||||
encodedApiKey,
|
||||
elasticsearchUrl,
|
||||
}: {
|
||||
templateUrl: string;
|
||||
stackName: string;
|
||||
logsStreamName: string;
|
||||
metricsStreamName: string;
|
||||
encodedApiKey: string;
|
||||
elasticsearchUrl: string;
|
||||
}) {
|
||||
const escapedElasticsearchUrl = elasticsearchUrl.replace(/\//g, '\\/');
|
||||
const escapedTemplateUrl = templateUrl.replace(/\//g, '\\/');
|
||||
|
||||
return `
|
||||
aws cloudformation create-stack
|
||||
--stack-name ${stackName}
|
||||
--template-url ${escapedTemplateUrl}
|
||||
--parameters ParameterKey=FirehoseStreamNameForLogs,ParameterValue=${logsStreamName}
|
||||
ParameterKey=FirehoseStreamNameForMetrics,ParameterValue=${metricsStreamName}
|
||||
ParameterKey=ElasticEndpointURL,ParameterValue=${escapedElasticsearchUrl}
|
||||
ParameterKey=ElasticAPIKey,ParameterValue=${encodedApiKey}
|
||||
--capabilities CAPABILITY_IAM
|
||||
`
|
||||
.trim()
|
||||
.replace(/\n/g, ' ')
|
||||
.replace(/\s\s+/g, ' ');
|
||||
}
|
||||
|
||||
export function buildStackStatusCommand({ stackName }: { stackName: string }) {
|
||||
return `
|
||||
aws cloudformation describe-stacks
|
||||
--stack-name ${stackName}
|
||||
--query "Stacks[0].StackStatus"
|
||||
`
|
||||
.trim()
|
||||
.replace(/\n/g, ' ')
|
||||
.replace(/\s\s+/g, ' ');
|
||||
}
|
||||
|
|
@ -0,0 +1,114 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { EuiIcon, EuiSpacer, useEuiTheme, useGeneratedHtmlId } from '@elastic/eui';
|
||||
import { css } from '@emotion/react';
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import React from 'react';
|
||||
import useInterval from 'react-use/lib/useInterval';
|
||||
import {
|
||||
FIREHOSE_CLOUDFORMATION_STACK_NAME,
|
||||
FIREHOSE_LOGS_STREAM_NAME,
|
||||
} from '../../../../common/aws_firehose';
|
||||
import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher';
|
||||
import { AccordionWithIcon } from '../shared/accordion_with_icon';
|
||||
import { GetStartedPanel } from '../shared/get_started_panel';
|
||||
import { ProgressIndicator } from '../shared/progress_indicator';
|
||||
import { useAWSServiceGetStartedList } from './use_aws_service_get_started_list';
|
||||
|
||||
const FETCH_INTERVAL = 2000;
|
||||
const REQUEST_PENDING_STATUS_LIST = [FETCH_STATUS.LOADING, FETCH_STATUS.NOT_INITIATED];
|
||||
|
||||
export function VisualizeData() {
|
||||
const accordionId = useGeneratedHtmlId({ prefix: 'accordion' });
|
||||
const { euiTheme } = useEuiTheme();
|
||||
const {
|
||||
data: populatedAWSLogsIndexList,
|
||||
status,
|
||||
refetch,
|
||||
} = useFetcher((callApi) => {
|
||||
return callApi('GET /internal/observability_onboarding/firehose/has-data', {
|
||||
params: {
|
||||
query: {
|
||||
logsStreamName: FIREHOSE_LOGS_STREAM_NAME,
|
||||
stackName: FIREHOSE_CLOUDFORMATION_STACK_NAME,
|
||||
},
|
||||
},
|
||||
});
|
||||
}, []);
|
||||
const awsServiceGetStartedConfigList = useAWSServiceGetStartedList();
|
||||
|
||||
useInterval(() => {
|
||||
if (REQUEST_PENDING_STATUS_LIST.includes(status)) {
|
||||
return;
|
||||
}
|
||||
|
||||
refetch();
|
||||
}, FETCH_INTERVAL);
|
||||
|
||||
if (populatedAWSLogsIndexList === undefined) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<ProgressIndicator
|
||||
title={i18n.translate('xpack.observability_onboarding.firehosePanel.waitingForDataTitle', {
|
||||
defaultMessage: 'Waiting for data from the Firehose stream',
|
||||
})}
|
||||
isLoading={true}
|
||||
css={css`
|
||||
max-width: 40%;
|
||||
`}
|
||||
/>
|
||||
|
||||
<EuiSpacer size="xl" />
|
||||
|
||||
<div data-test-subj="observabilityOnboardingAWSServiceList">
|
||||
{awsServiceGetStartedConfigList.map(
|
||||
({ id, indexNameList, actionLinks, title, logoURL, previewImage }) => {
|
||||
const isEnabled = indexNameList.some((indexName) =>
|
||||
populatedAWSLogsIndexList.includes(indexName)
|
||||
);
|
||||
|
||||
return (
|
||||
<AccordionWithIcon
|
||||
data-test-subj={`observabilityOnboardingAWSService-${id}`}
|
||||
key={id}
|
||||
id={`${accordionId}_${id}`}
|
||||
icon={<EuiIcon type={logoURL} size="l" />}
|
||||
title={i18n.translate(
|
||||
'xpack.observability_onboarding.firehosePanel.awsServiceDataFoundTitle',
|
||||
{
|
||||
defaultMessage: '{title}',
|
||||
values: { title },
|
||||
}
|
||||
)}
|
||||
extraAction={
|
||||
isEnabled ? <EuiIcon type="checkInCircleFilled" color="success" /> : null
|
||||
}
|
||||
isDisabled={!isEnabled}
|
||||
css={{
|
||||
paddingRight: euiTheme.size.s,
|
||||
filter: `grayscale(${isEnabled ? 0 : 1})`,
|
||||
}}
|
||||
>
|
||||
<GetStartedPanel
|
||||
integration="aws"
|
||||
newTab
|
||||
isLoading={false}
|
||||
actionLinks={actionLinks}
|
||||
previewImage={previewImage}
|
||||
/>
|
||||
</AccordionWithIcon>
|
||||
);
|
||||
}
|
||||
)}
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
|
@ -10,6 +10,8 @@ import { EuiLink, EuiSpacer, EuiText } from '@elastic/eui';
|
|||
import { css } from '@emotion/react';
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import { FormattedMessage } from '@kbn/i18n-react';
|
||||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import { DASHBOARD_APP_LOCATOR } from '@kbn/deeplinks-analytics';
|
||||
import {
|
||||
StepsProgress,
|
||||
useFlowProgressTelemetry,
|
||||
|
|
@ -17,6 +19,7 @@ import {
|
|||
import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher';
|
||||
import { ProgressIndicator } from '../shared/progress_indicator';
|
||||
import { GetStartedPanel } from '../shared/get_started_panel';
|
||||
import { ObservabilityOnboardingContextValue } from '../../../plugin';
|
||||
|
||||
interface Props {
|
||||
onboardingId: string;
|
||||
|
|
@ -29,6 +32,10 @@ const CLUSTER_OVERVIEW_DASHBOARD_ID = 'kubernetes-f4dc26db-1b53-4ea2-a78b-1bfab8
|
|||
export function DataIngestStatus({ onboardingId }: Props) {
|
||||
const [progress, setProgress] = useState<StepsProgress | undefined>(undefined);
|
||||
const [checkDataStartTime] = useState(Date.now());
|
||||
const {
|
||||
services: { share },
|
||||
} = useKibana<ObservabilityOnboardingContextValue>();
|
||||
const dashboardLocator = share.url.locators.get(DASHBOARD_APP_LOCATOR);
|
||||
|
||||
const { data, status, refetch } = useFetcher(
|
||||
(callApi) => {
|
||||
|
|
@ -112,7 +119,7 @@ export function DataIngestStatus({ onboardingId }: Props) {
|
|||
integration="kubernetes"
|
||||
newTab={false}
|
||||
isLoading={false}
|
||||
dashboardLinks={[
|
||||
actionLinks={[
|
||||
{
|
||||
id: CLUSTER_OVERVIEW_DASHBOARD_ID,
|
||||
label: i18n.translate(
|
||||
|
|
@ -127,6 +134,10 @@ export function DataIngestStatus({ onboardingId }: Props) {
|
|||
defaultMessage: 'Overview your Kubernetes cluster with this pre-made dashboard',
|
||||
}
|
||||
),
|
||||
href:
|
||||
dashboardLocator?.getRedirectUrl({
|
||||
dashboardId: CLUSTER_OVERVIEW_DASHBOARD_ID,
|
||||
}) ?? '',
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
|
|
|||
|
|
@ -24,9 +24,13 @@ import { FeedbackButtons } from '../shared/feedback_buttons';
|
|||
|
||||
export const KubernetesPanel: React.FC = () => {
|
||||
const [windowLostFocus, setWindowLostFocus] = useState(false);
|
||||
const { data, status, error, refetch } = useFetcher((callApi) => {
|
||||
return callApi('POST /internal/observability_onboarding/kubernetes/flow');
|
||||
}, []);
|
||||
const { data, status, error, refetch } = useFetcher(
|
||||
(callApi) => {
|
||||
return callApi('POST /internal/observability_onboarding/kubernetes/flow');
|
||||
},
|
||||
[],
|
||||
{ showToastOnError: false }
|
||||
);
|
||||
|
||||
useEvent('blur', () => setWindowLostFocus(true), window);
|
||||
|
||||
|
|
|
|||
|
|
@ -17,31 +17,31 @@ import {
|
|||
import { useKibana } from '@kbn/kibana-react-plugin/public';
|
||||
import React from 'react';
|
||||
import { i18n } from '@kbn/i18n';
|
||||
import { DASHBOARD_APP_LOCATOR } from '@kbn/deeplinks-analytics';
|
||||
import { FormattedMessage } from '@kbn/i18n-react';
|
||||
import { ObservabilityOnboardingContextValue } from '../../../plugin';
|
||||
|
||||
export function GetStartedPanel({
|
||||
integration,
|
||||
dashboardLinks,
|
||||
actionLinks,
|
||||
previewImage = 'charts_screen.svg',
|
||||
newTab,
|
||||
isLoading,
|
||||
}: {
|
||||
integration: string;
|
||||
newTab: boolean;
|
||||
dashboardLinks: Array<{
|
||||
actionLinks: Array<{
|
||||
id: string;
|
||||
label: string;
|
||||
title: string;
|
||||
label: string;
|
||||
href: string;
|
||||
}>;
|
||||
previewImage?: string;
|
||||
isLoading: boolean;
|
||||
}) {
|
||||
const {
|
||||
services: { http, share },
|
||||
services: { http },
|
||||
} = useKibana<ObservabilityOnboardingContextValue>();
|
||||
|
||||
const dashboardLocator = share.url.locators.get(DASHBOARD_APP_LOCATOR);
|
||||
|
||||
return (
|
||||
<>
|
||||
<EuiFlexGroup alignItems="center" responsive={false}>
|
||||
|
|
@ -50,7 +50,7 @@ export function GetStartedPanel({
|
|||
<EuiSkeletonRectangle width={162} height={117} />
|
||||
) : (
|
||||
<EuiImage
|
||||
src={http.staticAssets.getPluginAssetHref('charts_screen.svg')}
|
||||
src={http.staticAssets.getPluginAssetHref(previewImage)}
|
||||
width={162}
|
||||
height={117}
|
||||
alt=""
|
||||
|
|
@ -61,17 +61,15 @@ export function GetStartedPanel({
|
|||
|
||||
<EuiFlexItem>
|
||||
<EuiFlexGroup direction="column" gutterSize="s">
|
||||
{dashboardLinks.map(({ id, label, title }) => (
|
||||
{actionLinks.map(({ id, title, label, href }) => (
|
||||
<EuiFlexItem key={id}>
|
||||
<EuiFlexGroup direction="column" gutterSize="xs" alignItems="flexStart">
|
||||
<EuiText key={id} size="s">
|
||||
<p>{title}</p>
|
||||
</EuiText>
|
||||
<EuiLink
|
||||
data-test-subj={`observabilityOnboardingDataIngestStatusViewDashboardLink-${id}`}
|
||||
href={dashboardLocator?.getRedirectUrl({
|
||||
dashboardId: id,
|
||||
})}
|
||||
data-test-subj={`observabilityOnboardingDataIngestStatusActionLink-${id}`}
|
||||
href={href}
|
||||
target={newTab ? '_blank' : '_self'}
|
||||
>
|
||||
{label}
|
||||
|
|
|
|||
|
|
@ -23,7 +23,8 @@ export type SupportedLogo =
|
|||
| 'nginx'
|
||||
| 'apache'
|
||||
| 'system'
|
||||
| 'opentelemetry';
|
||||
| 'opentelemetry'
|
||||
| 'firehose';
|
||||
|
||||
export function isSupportedLogo(logo: string): logo is SupportedLogo {
|
||||
return [
|
||||
|
|
|
|||
|
|
@ -0,0 +1,12 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg width="24px" height="24px" viewBox="0 0 24 24" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
|
||||
<title>Icon-Architecture/16/Arch_Amazon-Kinesis-Data-Firehose_16</title>
|
||||
<g id="Icon-Architecture/16/Arch_Amazon-Kinesis-Data-Firehose_16" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
|
||||
<g id="Icon-Architecture-BG/16/Analytics" fill="#8C4FFF">
|
||||
<rect id="Rectangle" x="0" y="0" width="24" height="24"></rect>
|
||||
</g>
|
||||
<g id="Icon-Service/16/Amazon-Kinesis-Data-Firehose" transform="translate(5.000000, 6.000000)" fill="#FFFFFF">
|
||||
<path d="M14.5836386,7.241 L13.9526386,8.017 C13.1876386,7.396 12.0286386,6.994 11.0006386,6.994 L11.0006386,6.056 L11.0006386,5.994 L11.0006386,5.056 C12.1386386,5.056 13.2796386,4.507 13.9696386,3.992 L14.5666386,4.794 C14.2016386,5.067 12.9546386,5.9 11.3996386,6.03 C12.5466386,6.117 13.7346386,6.552 14.5836386,7.241 L14.5836386,7.241 Z M9.00063863,6.62 L7.00063863,7.12 L7.00063863,4.902 L9.00063863,5.402 L9.00063863,6.62 Z M6.00063863,8 L5.00063863,8 L5.00063863,4 L6.00063863,4 L6.00063863,4.261 L6.00063863,7.761 L6.00063863,8 Z M9.62163863,4.526 L7.00063863,3.871 L7.00063863,3.5 C7.00063863,3.223 6.77663863,3 6.50063863,3 L4.50063863,3 C4.22463863,3 4.00063863,3.223 4.00063863,3.5 L4.00063863,4.409 C1.46663863,4.086 1.00063863,3.16 1.00063863,1.42 L1.00063863,1.77635684e-15 L0.000638629816,1.77635684e-15 L0.000638629816,1.42 C0.000638629816,3.918 1.14663863,5.056 4.00063863,5.403 L4.00063863,5.5 L0.000638629816,5.5 L0.000638629816,6.5 L4.00063863,6.5 L4.00063863,6.604 C1.21763863,6.983 0.00163862982,8.274 0.00163862982,10.882 L0.00163862982,10.99 C-0.000361370184,11.279 -0.00136137018,11.995 0.00363862982,12.048 L1.00063863,11.97 C0.99663863,11.874 0.99963863,11.252 1.00163863,10.994 L1.00163863,10.882 C1.00163863,9.148 1.54263863,7.991 4.00063863,7.615 L4.00063863,8.5 C4.00063863,8.776 4.22463863,9 4.50063863,9 L6.50063863,9 C6.77663863,9 7.00063863,8.776 7.00063863,8.5 L7.00063863,8.151 L9.62163863,7.496 C9.84463863,7.44 10.0006386,7.241 10.0006386,7.011 L10.0006386,5.011 C10.0006386,4.782 9.84463863,4.582 9.62163863,4.526 L9.62163863,4.526 Z" id="Amazon-Kinesis-Data-Firehose_Icon_16_Squid"></path>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.3 KiB |
|
|
@ -30,6 +30,8 @@ export interface ConfigSchema {
|
|||
}
|
||||
|
||||
export interface AppContext {
|
||||
isDev: boolean;
|
||||
isCloud: boolean;
|
||||
isServerless: boolean;
|
||||
stackVersion: string;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -81,7 +81,8 @@ export class ObservabilityOnboardingPlugin
|
|||
const {
|
||||
ui: { enabled: isObservabilityOnboardingUiEnabled },
|
||||
} = config;
|
||||
|
||||
const isServerlessBuild = this.ctx.env.packageInfo.buildFlavor === 'serverless';
|
||||
const isDevEnvironment = this.ctx.env.mode.dev;
|
||||
const pluginSetupDeps = plugins;
|
||||
|
||||
// set xpack.observability_onboarding.ui.enabled: true
|
||||
|
|
@ -112,7 +113,10 @@ export class ObservabilityOnboardingPlugin
|
|||
corePlugins: corePlugins as ObservabilityOnboardingPluginStartDeps,
|
||||
config,
|
||||
context: {
|
||||
isServerless: Boolean(pluginSetupDeps.cloud?.isServerlessEnabled),
|
||||
isDev: isDevEnvironment,
|
||||
isCloud: Boolean(pluginSetupDeps.cloud?.isCloudEnabled),
|
||||
isServerless:
|
||||
Boolean(pluginSetupDeps.cloud?.isServerlessEnabled) || isServerlessBuild,
|
||||
stackVersion,
|
||||
},
|
||||
});
|
||||
|
|
|
|||
|
|
@ -0,0 +1,141 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { v4 as uuidv4 } from 'uuid';
|
||||
import Boom from '@hapi/boom';
|
||||
import * as t from 'io-ts';
|
||||
import { termQuery, wildcardQuery } from '@kbn/observability-plugin/server';
|
||||
import type { estypes } from '@elastic/elasticsearch';
|
||||
import {
|
||||
AWSIndexName,
|
||||
AWS_INDEX_NAME_LIST,
|
||||
FIREHOSE_CLOUDFORMATION_TEMPLATE_URL,
|
||||
} from '../../../common/aws_firehose';
|
||||
import { getFallbackESUrl } from '../../lib/get_fallback_urls';
|
||||
import { createObservabilityOnboardingServerRoute } from '../create_observability_onboarding_server_route';
|
||||
import { hasLogMonitoringPrivileges } from '../../lib/api_key/has_log_monitoring_privileges';
|
||||
import { createShipperApiKey } from '../../lib/api_key/create_shipper_api_key';
|
||||
|
||||
export interface CreateFirehoseOnboardingFlowRouteResponse {
|
||||
apiKeyEncoded: string;
|
||||
onboardingId: string;
|
||||
elasticsearchUrl: string;
|
||||
templateUrl: string;
|
||||
}
|
||||
|
||||
export type HasFirehoseDataRouteResponse = AWSIndexName[];
|
||||
|
||||
interface DocumentCountPerIndexBucket {
|
||||
key: string;
|
||||
doc_count: number;
|
||||
}
|
||||
|
||||
const createFirehoseOnboardingFlowRoute = createObservabilityOnboardingServerRoute({
|
||||
endpoint: 'POST /internal/observability_onboarding/firehose/flow',
|
||||
options: { tags: [] },
|
||||
async handler({
|
||||
context,
|
||||
request,
|
||||
plugins,
|
||||
services,
|
||||
}): Promise<CreateFirehoseOnboardingFlowRouteResponse> {
|
||||
const {
|
||||
elasticsearch: { client },
|
||||
} = await context.core;
|
||||
|
||||
const hasPrivileges = await hasLogMonitoringPrivileges(client.asCurrentUser);
|
||||
|
||||
if (!hasPrivileges) {
|
||||
throw Boom.forbidden(
|
||||
"You don't have enough privileges to start a new onboarding flow. Contact your system administrator to grant you the required privileges."
|
||||
);
|
||||
}
|
||||
|
||||
const fleetPluginStart = await plugins.fleet.start();
|
||||
const packageClient = fleetPluginStart.packageService.asScoped(request);
|
||||
|
||||
const [{ encoded: apiKeyEncoded }] = await Promise.all([
|
||||
createShipperApiKey(client.asCurrentUser, 'firehose_onboarding'),
|
||||
packageClient.ensureInstalledPackage({ pkgName: 'awsfirehose' }),
|
||||
packageClient.ensureInstalledPackage({ pkgName: 'aws' }),
|
||||
]);
|
||||
|
||||
const elasticsearchUrlList = plugins.cloud?.setup?.elasticsearchUrl
|
||||
? [plugins.cloud?.setup?.elasticsearchUrl]
|
||||
: await getFallbackESUrl(services.esLegacyConfigService);
|
||||
|
||||
return {
|
||||
onboardingId: uuidv4(),
|
||||
apiKeyEncoded,
|
||||
elasticsearchUrl: elasticsearchUrlList.length > 0 ? elasticsearchUrlList[0] : '',
|
||||
templateUrl: FIREHOSE_CLOUDFORMATION_TEMPLATE_URL,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
const hasFirehoseDataRoute = createObservabilityOnboardingServerRoute({
|
||||
endpoint: 'GET /internal/observability_onboarding/firehose/has-data',
|
||||
params: t.type({
|
||||
query: t.type({
|
||||
logsStreamName: t.string,
|
||||
stackName: t.string,
|
||||
}),
|
||||
}),
|
||||
options: { tags: [] },
|
||||
async handler(resources): Promise<HasFirehoseDataRouteResponse> {
|
||||
const { logsStreamName, stackName } = resources.params.query;
|
||||
const { elasticsearch } = await resources.context.core;
|
||||
const indexPatternList = AWS_INDEX_NAME_LIST.map((index) => `${index}-*`);
|
||||
|
||||
try {
|
||||
const result = await elasticsearch.client.asCurrentUser.search<
|
||||
unknown,
|
||||
Record<
|
||||
'documents_per_index',
|
||||
estypes.AggregationsMultiBucketAggregateBase<DocumentCountPerIndexBucket>
|
||||
>
|
||||
>({
|
||||
index: indexPatternList,
|
||||
ignore_unavailable: true,
|
||||
size: 0,
|
||||
query: {
|
||||
bool: {
|
||||
should: [
|
||||
...termQuery('aws.kinesis.name', logsStreamName),
|
||||
...wildcardQuery('aws.exporter.arn', stackName),
|
||||
],
|
||||
},
|
||||
},
|
||||
aggs: {
|
||||
documents_per_index: {
|
||||
terms: {
|
||||
field: '_index',
|
||||
size: indexPatternList.length,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const buckets = result.aggregations?.documents_per_index.buckets;
|
||||
|
||||
if (!Array.isArray(buckets)) {
|
||||
return [];
|
||||
}
|
||||
|
||||
return AWS_INDEX_NAME_LIST.filter((indexName) => {
|
||||
return buckets.some((bucket) => bucket.key.includes(indexName) && bucket.doc_count > 0);
|
||||
});
|
||||
} catch (error) {
|
||||
throw Boom.internal(`Elasticsearch responded with an error. ${error.message}`);
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
export const firehoseOnboardingRouteRepository = {
|
||||
...createFirehoseOnboardingFlowRoute,
|
||||
...hasFirehoseDataRoute,
|
||||
};
|
||||
|
|
@ -9,6 +9,7 @@ import { elasticAgentRouteRepository } from './elastic_agent/route';
|
|||
import { flowRouteRepository } from './flow/route';
|
||||
import { kubernetesOnboardingRouteRepository } from './kubernetes/route';
|
||||
import { logsOnboardingRouteRepository } from './logs/route';
|
||||
import { firehoseOnboardingRouteRepository } from './firehose/route';
|
||||
|
||||
function getTypedObservabilityOnboardingServerRouteRepository() {
|
||||
const repository = {
|
||||
|
|
@ -16,6 +17,7 @@ function getTypedObservabilityOnboardingServerRouteRepository() {
|
|||
...logsOnboardingRouteRepository,
|
||||
...elasticAgentRouteRepository,
|
||||
...kubernetesOnboardingRouteRepository,
|
||||
...firehoseOnboardingRouteRepository,
|
||||
};
|
||||
|
||||
return repository;
|
||||
|
|
|
|||
|
|
@ -0,0 +1,89 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import expect from 'expect';
|
||||
import { log, timerange } from '@kbn/apm-synthtrace-client';
|
||||
import moment from 'moment';
|
||||
import { FtrProviderContext } from '../../../ftr_provider_context';
|
||||
|
||||
const CF_COMMAND_REGEXP =
|
||||
/aws cloudformation create-stack --stack-name (\S+) --template-url \S+ --parameters ParameterKey=FirehoseStreamNameForLogs,ParameterValue=(\S+) .+? --capabilities CAPABILITY_IAM/;
|
||||
|
||||
export default function ({ getPageObjects, getService }: FtrProviderContext) {
|
||||
const PageObjects = getPageObjects(['common', 'svlCommonPage']);
|
||||
|
||||
const browser = getService('browser');
|
||||
const testSubjects = getService('testSubjects');
|
||||
const synthtrace = getService('svlLogsSynthtraceClient');
|
||||
|
||||
describe('Onboarding Firehose Quickstart Flow', () => {
|
||||
before(async () => {
|
||||
await PageObjects.svlCommonPage.loginAsAdmin(); // Onboarding requires admin role
|
||||
await PageObjects.common.navigateToUrlWithBrowserHistory(
|
||||
'observabilityOnboarding',
|
||||
'/firehose',
|
||||
undefined,
|
||||
{
|
||||
ensureCurrentUrl: false, // the check sometimes is too slow for the page so it misses the point in time before the app rewrites the URL
|
||||
}
|
||||
);
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
await synthtrace.clean();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await testSubjects.existOrFail('observabilityOnboardingFirehoseCreateStackCommand');
|
||||
});
|
||||
|
||||
it('shows the correct CloudFormation command snippet', async () => {
|
||||
await testSubjects.clickWhenNotDisabled('observabilityOnboardingCopyToClipboardButton');
|
||||
const copiedCommand = await browser.getClipboardValue();
|
||||
|
||||
expect(copiedCommand).toMatch(CF_COMMAND_REGEXP);
|
||||
});
|
||||
|
||||
it('starts to monitor for incoming data after user leaves the page', async () => {
|
||||
await browser.execute(`window.dispatchEvent(new Event("blur"))`);
|
||||
|
||||
await testSubjects.isDisplayed('observabilityOnboardingAWSServiceList');
|
||||
});
|
||||
|
||||
it('highlights an AWS service when data is detected', async () => {
|
||||
const DATASET = 'aws.vpcflow';
|
||||
const AWS_SERVICE_ID = 'vpc-flow';
|
||||
await testSubjects.clickWhenNotDisabled('observabilityOnboardingCopyToClipboardButton');
|
||||
const copiedCommand = await browser.getClipboardValue();
|
||||
const [, _stackName, logsStreamName] = copiedCommand.match(CF_COMMAND_REGEXP) ?? [];
|
||||
|
||||
expect(logsStreamName).toBeDefined();
|
||||
|
||||
await browser.execute(`window.dispatchEvent(new Event("blur"))`);
|
||||
|
||||
// Simulate Firehose stream ingesting log files
|
||||
const to = new Date().toISOString();
|
||||
const count = 1;
|
||||
await synthtrace.index(
|
||||
timerange(moment(to).subtract(count, 'minute'), moment(to))
|
||||
.interval('1m')
|
||||
.rate(1)
|
||||
.generator((timestamp) => {
|
||||
return log.create().dataset(DATASET).timestamp(timestamp).defaults({
|
||||
'aws.kinesis.name': logsStreamName,
|
||||
});
|
||||
})
|
||||
);
|
||||
|
||||
// Checking that an AWS service item is enabled after data is detected
|
||||
await testSubjects
|
||||
.find(`observabilityOnboardingAWSService-${AWS_SERVICE_ID}`)
|
||||
.then((el) => el.findByTagName('button'))
|
||||
.then((el) => el.isEnabled());
|
||||
});
|
||||
});
|
||||
}
|
||||
|
|
@ -10,5 +10,6 @@ import { FtrProviderContext } from '../../../ftr_provider_context';
|
|||
export default function ({ loadTestFile }: FtrProviderContext) {
|
||||
describe('Onboarding', function () {
|
||||
loadTestFile(require.resolve('./auto_detect'));
|
||||
loadTestFile(require.resolve('./firehose'));
|
||||
});
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue