github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 01:38:56 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Security Solutions] Adds bsearch service to FTR e2e tests to reduce flake, boilerplate, and technique choices (#116211) (#116514)

Browse source 
## Summary

Fixes flake tests of:
https://github.com/elastic/kibana/issues/115918
https://github.com/elastic/kibana/issues/103273
https://github.com/elastic/kibana/issues/108640
https://github.com/elastic/kibana/issues/109447
https://github.com/elastic/kibana/issues/100630
https://github.com/elastic/kibana/issues/94535
https://github.com/elastic/kibana/issues/104260

Security solution has been using `bsearch` and has encountered flake in various forms. Different developers have been fixing the flake in a few odd ways (myself included) which aren't 100%. This PR introduces a once-in-for-all REST API retry service called `bsearch` which will query `bsearch` and if `bsearch` is not completed because of async occurring due to slower CI runtimes it will continuously call into the `bsearch` with the correct API to ensure it gets a complete response before returning.


## Usage

Anyone can use this service like so:
```ts
const bsearch = getService('bsearch');
const response = await bsearch.send<MyType>({
 supertest,
 options: {
   defaultIndex: ['large_volume_dns_data'],
}
  strategy: 'securitySolutionSearchStrategy',
});
```

If you're using a custom auth then you can set that beforehand like so:
```ts
const bsearch = getService('bsearch');
const supertestWithoutAuth = getService('supertestWithoutAuth');
const supertest supertestWithoutAuth.auth(username, password);
const response = await bsearch.send<MyType>({
 supertest,
 options: {
   defaultIndex: ['large_volume_dns_data'],
  }
  strategy: 'securitySolutionSearchStrategy',
});
```

## Misconceptions in the tests leading to flake
* Can you just call the bsearch REST API and it will always return data first time? Not always true, as when CI slows down or data increases `bsearch` will give you back an async reference and then your test will blow up.
* Can we wrap the REST API in `retry` to fix the flake? Not always but mostly true, as when CI slows down or data increases `bsearch` could return the async version continuously which could then fail your test. It's also tedious to tell everyone in code reviews to wrap everything in `retry` instead of just fixing it with a service as well as inform new people why we are constantly wrapping these tests in `retry`.
* Can we manually parse the `bsearch` if it has `async` for each test? This is true but is error prone and I did this for one test and it's ugly and I had issues as I have to wrap 2 things in `retry` and test several conditions. Also it's harder for people to read the tests rather than just reading there is a service call. Also people in code reviews missed where I had bugs with it. Also lots of boiler plate.
* Can we just increase the timeout with `wait_for_completion_timeout` and the tests will pass for sure then? Not true today but maybe true later, as this hasn't been added as plumbing yet. See this [open ticket](https://github.com/elastic/kibana/issues/107241). Even if it is and we increase the timeout to a very large number bsearch might return with an `async` or you might want to test the `async` path. Either way, if/when we add the ability we can increase it within 1 spot which is this service for everyone rather than going to each individual test to add it. If/when it's added if people don't use the bsearch service we can remove it later if we find this is deterministic enough and no one wants to test bsearch features with their strategies down the road.

## Manual test of bsearch service
If you want to manually watch the bsearch operate as if the CI system is running slow or to cause an `async` manually you manually modify this setting here:
https://github.com/elastic/kibana/blob/master/src/plugins/data/server/search/strategies/ese_search/request_utils.ts#L61

To be of a lower number such as `1ms` and then you will see it enter the `async` code within `bsearch` consistently

## Reference PRs
We cannot set the wait_for_complete just yet
https://github.com/elastic/kibana/issues/107241 so we decided this was the best way to reduce flake for testing for now. 

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Frank Hassanabad 2021-10-28 12:09:42 -06:00 • committed by GitHub
parent 649f467aa3
commit 2add12fa65
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
20 changed files with 824 additions and 723 deletions
Download patch file Download diff file Expand all files Collapse all files

122
test/common/services/bsearch.ts Normal file
View file

@ -0,0 +1,122 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import expect from '@kbn/expect';
import request from 'superagent';
import type SuperTest from 'supertest';
import { IEsSearchResponse } from 'src/plugins/data/common';
import { FtrProviderContext } from '../ftr_provider_context';
import { RetryService } from './retry/retry';
/**
* Function copied from here:
* test/api_integration/apis/search/bsearch.ts without the compress
*
* Splits the JSON lines from bsearch
*/
const parseBfetchResponse = (resp: request.Response): Array<Record<string, any>> => {
return resp.text
.trim()
.split('\n')
.map((item) => JSON.parse(item));
};
/**
* Function copied from here:
* x-pack/test/rule_registry/common/lib/authentication/spaces.ts
* @param spaceId The space id we want to utilize
*/
const getSpaceUrlPrefix = (spaceId?: string): string => {
return spaceId && spaceId !== 'default' ? `/s/${spaceId}` : ``;
};
/**
* Options for the send method
*/
interface SendOptions {
supertest: SuperTest.SuperTest<SuperTest.Test>;
options: object;
strategy: string;
space?: string;
}
/**
* Bsearch factory which will return a new bsearch capable service that can reduce flake
* on the CI systems when they are under pressure and bsearch returns an async search
* response or a sync response.
*
* @example
* const supertest = getService('supertest');
* const bsearch = getService('bsearch');
* const response = await bsearch.send<MyType>({
* supertest,
* options: {
* defaultIndex: ['large_volume_dns_data'],
* }
* strategy: 'securitySolutionSearchStrategy',
* });
* expect(response).eql({ ... your value ... });
*/
export const BSearchFactory = (retry: RetryService) => ({
/** Send method to send in your supertest, url, options, and strategy name */
send: async <T extends IEsSearchResponse>({
supertest,
options,
strategy,
space,
}: SendOptions): Promise<T> => {
const spaceUrl = getSpaceUrlPrefix(space);
const { body } = await retry.try(async () => {
return supertest
.post(`${spaceUrl}/internal/search/${strategy}`)
.set('kbn-xsrf', 'true')
.send(options)
.expect(200);
});
if (body.isRunning) {
const result = await retry.try(async () => {
const resp = await supertest
.post(`${spaceUrl}/internal/bsearch`)
.set('kbn-xsrf', 'true')
.send({
batch: [
{
request: {
id: body.id,
...options,
},
options: {
strategy,
},
},
],
})
.expect(200);
const [parsedResponse] = parseBfetchResponse(resp);
expect(parsedResponse.result.isRunning).equal(false);
return parsedResponse.result;
});
return result;
} else {
return body;
}
},
});
/**
* Bsearch provider which will return a new bsearch capable service that can reduce flake
* on the CI systems when they are under pressure and bsearch returns an async search response
* or a sync response.
*/
export function BSearchProvider({
getService,
}: FtrProviderContext): ReturnType<typeof BSearchFactory> {
const retry = getService('retry');
return BSearchFactory(retry);
}

2
test/common/services/index.ts
View file

@ -16,6 +16,7 @@ import { SecurityServiceProvider } from './security';
import { EsDeleteAllIndicesProvider } from './es_delete_all_indices';
import { SavedObjectInfoService } from './saved_object_info';
import { IndexPatternsService } from './index_patterns';
import { BSearchProvider } from './bsearch';
export const services = {
deployment: DeploymentService,
@ -28,4 +29,5 @@ export const services = {
esDeleteAllIndices: EsDeleteAllIndicesProvider,
savedObjectInfo: SavedObjectInfoService,
indexPatterns: IndexPatternsService,
bsearch: BSearchProvider,
};

43
x-pack/test/api_integration/apis/security_solution/authentications.ts
View file

@ -6,7 +6,10 @@
*/
import expect from '@kbn/expect';
import { HostsQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
HostAuthenticationsStrategyResponse,
HostsQueries,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -22,16 +25,19 @@ const EDGE_LENGTH = 1;
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('authentications', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/hosts'));
before(async () => await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts'));
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/hosts')
);
it('Make sure that we get Authentication data', async () => {
const { body: authentications } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const authentications = await bsearch.send<HostAuthenticationsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.authentications,
timerange: {
interval: '12h',
@ -47,9 +53,9 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(authentications.edges.length).to.be(EDGE_LENGTH);
expect(authentications.totalCount).to.be(TOTAL_COUNT);
@ -57,10 +63,9 @@ export default function ({ getService }: FtrProviderContext) {
});
it('Make sure that pagination is working in Authentications query', async () => {
const { body: authentications } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const authentications = await bsearch.send<HostAuthenticationsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.authentications,
timerange: {
interval: '12h',
@ -76,16 +81,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(authentications.edges.length).to.be(EDGE_LENGTH);
expect(authentications.totalCount).to.be(TOTAL_COUNT);
expect(authentications.edges[0]!.node.lastSuccess!.source!.ip).to.eql([
expect(authentications.edges[0].node.lastSuccess?.source?.ip).to.eql([
LAST_SUCCESS_SOURCE_IP,
]);
expect(authentications.edges[0]!.node.lastSuccess!.host!.name).to.eql([HOST_NAME]);
expect(authentications.edges[0].node.lastSuccess?.host?.name).to.eql([HOST_NAME]);
});
});
}

33
x-pack/test/api_integration/apis/security_solution/events.ts
View file

@ -11,13 +11,13 @@ import { JsonObject } from '@kbn/utility-types';
import {
Direction,
TimelineEventsQueries,
TimelineEventsAllStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
import { getDocValueFields, getFieldsToRequest, getFilterValue } from './utils';
const TO = '3000-01-01T00:00:00.000Z';
const FROM = '2000-01-01T00:00:00.000Z';
const TEST_URL = '/internal/search/timelineSearchStrategy/';
// typical values that have to change after an update from "scripts/es_archiver"
const DATA_COUNT = 7;
const HOST_NAME = 'suricata-sensor-amsterdam';
@ -30,6 +30,8 @@ const LIMITED_PAGE_SIZE = 2;
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
const getPostBody = (): JsonObject => ({
defaultIndex: ['auditbeat-*'],
docValueFields: getDocValueFields(),
@ -66,14 +68,14 @@ export default function ({ getService }: FtrProviderContext) {
});
it('returns Timeline data', async () => {
const resp = await supertest
.post(TEST_URL)
.set('kbn-xsrf', 'true')
.set('Content-Type', 'application/json')
.send(getPostBody())
.expect(200);
const timeline = await bsearch.send<TimelineEventsAllStrategyResponse>({
supertest,
options: {
...getPostBody(),
},
strategy: 'timelineSearchStrategy',
});
const timeline = resp.body;
expect(timeline.edges.length).to.be(EDGE_LENGTH);
expect(timeline.edges[0].node.data.length).to.be(DATA_COUNT);
expect(timeline.totalCount).to.be(TOTAL_COUNT);
@ -82,20 +84,17 @@ export default function ({ getService }: FtrProviderContext) {
});
it('returns paginated Timeline query', async () => {
const resp = await supertest
.post(TEST_URL)
.set('kbn-xsrf', 'true')
.set('Content-Type', 'application/json')
.send({
const timeline = await bsearch.send<TimelineEventsAllStrategyResponse>({
supertest,
options: {
...getPostBody(),
pagination: {
activePage: 0,
querySize: LIMITED_PAGE_SIZE,
},
})
.expect(200);
const timeline = resp.body;
},
strategy: 'timelineSearchStrategy',
});
expect(timeline.edges.length).to.be(LIMITED_PAGE_SIZE);
expect(timeline.edges[0].node.data.length).to.be(DATA_COUNT);
expect(timeline.totalCount).to.be(TOTAL_COUNT);

29
x-pack/test/api_integration/apis/security_solution/host_details.ts
View file

@ -7,16 +7,24 @@
import expect from '@kbn/expect';
import { FtrProviderContext } from '../../ftr_provider_context';
import { HostsQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
HostDetailsStrategyResponse,
HostsQueries,
} from '../../../../plugins/security_solution/common/search_strategy';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Host Details', () => {
describe('With filebeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -213,12 +221,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get HostDetails data', async () => {
const {
body: { hostDetails },
} = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const { hostDetails } = await bsearch.send<HostDetailsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.details,
timerange: {
interval: '12h',
@ -229,9 +234,9 @@ export default function ({ getService }: FtrProviderContext) {
docValueFields: [],
hostName: 'raspberrypi',
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(hostDetails).to.eql(expectedResult.hostDetails);
});
});

135
x-pack/test/api_integration/apis/security_solution/hosts.ts
View file

@ -10,6 +10,9 @@ import {
HostsQueries,
Direction,
HostsFields,
HostsStrategyResponse,
HostDetailsStrategyResponse,
HostFirstLastSeenStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -26,17 +29,19 @@ const CURSOR_ID = '2ab45fc1c41e4c84bbd02202a7e5761f';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
// Failing: See https://github.com/elastic/kibana/issues/104260
describe.skip('hosts', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/hosts'));
describe('hosts', () => {
before(async () => await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts'));
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/hosts')
);
it('Make sure that we get Hosts Table data', async () => {
const { body: hosts } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const hosts = await bsearch.send<HostsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.hosts,
timerange: {
interval: '12h',
@ -56,19 +61,18 @@ export default function ({ getService }: FtrProviderContext) {
querySize: 1,
},
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(hosts.edges.length).to.be(EDGE_LENGTH);
expect(hosts.totalCount).to.be(TOTAL_COUNT);
expect(hosts.pageInfo.fakeTotalCount).to.equal(3);
});
it('Make sure that pagination is working in Hosts Table query', async () => {
const { body: hosts } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const hosts = await bsearch.send<HostsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.hosts,
timerange: {
interval: '12h',
@ -88,16 +92,32 @@ export default function ({ getService }: FtrProviderContext) {
querySize: 2,
},
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(hosts.edges.length).to.be(EDGE_LENGTH);
expect(hosts.totalCount).to.be(TOTAL_COUNT);
expect(hosts.edges[0]!.node.host!.os!.name).to.eql([HOST_NAME]);
expect(hosts.edges[0].node.host?.os?.name).to.eql([HOST_NAME]);
});
it('Make sure that we get Host details data', async () => {
const expectedHostDetails = {
const { hostDetails } = await bsearch.send<HostDetailsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.details,
hostName: 'zeek-sensor-san-francisco',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(hostDetails).to.eql({
_id: 'zeek-sensor-san-francisco',
host: {
architecture: ['x86_64'],
@ -122,91 +142,66 @@ export default function ({ getService }: FtrProviderContext) {
provider: ['digitalocean'],
region: ['sfo2'],
},
};
const {
body: { hostDetails },
} = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsQueries.details,
hostName: 'zeek-sensor-san-francisco',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(hostDetails).to.eql(expectedHostDetails);
});
});
it('Make sure that we get First Seen for a Host without docValueFields', async () => {
const { body: firstLastSeenHost } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const firstLastSeenHost = await bsearch.send<HostFirstLastSeenStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.firstOrLastSeen,
defaultIndex: ['auditbeat-*'],
docValueFields: [],
hostName: 'zeek-sensor-san-francisco',
order: 'asc',
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(firstLastSeenHost.firstSeen).to.eql('2019-02-19T19:36:23.561Z');
});
it('Make sure that we get Last Seen for a Host without docValueFields', async () => {
const { body: firstLastSeenHost } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const firstLastSeenHost = await bsearch.send<HostFirstLastSeenStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.firstOrLastSeen,
defaultIndex: ['auditbeat-*'],
docValueFields: [],
hostName: 'zeek-sensor-san-francisco',
order: 'desc',
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(firstLastSeenHost.lastSeen).to.eql('2019-02-19T20:42:33.561Z');
});
it('Make sure that we get First Seen for a Host with docValueFields', async () => {
const { body: firstLastSeenHost } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const firstLastSeenHost = await bsearch.send<HostFirstLastSeenStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.firstOrLastSeen,
defaultIndex: ['auditbeat-*', 'filebeat-*', 'packetbeat-*', 'winlogbeat-*'],
docValueFields: [{ field: '@timestamp', format: 'epoch_millis' }],
hostName: 'zeek-sensor-san-francisco',
order: 'asc',
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(firstLastSeenHost.firstSeen).to.eql(new Date('2019-02-19T19:36:23.561Z').valueOf());
});
it('Make sure that we get Last Seen for a Host with docValueFields', async () => {
const { body: firstLastSeenHost } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const firstLastSeenHost = await bsearch.send<HostFirstLastSeenStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.firstOrLastSeen,
defaultIndex: ['auditbeat-*', 'filebeat-*', 'packetbeat-*', 'winlogbeat-*'],
docValueFields: [{ field: '@timestamp', format: 'epoch_millis' }],
hostName: 'zeek-sensor-san-francisco',
order: 'desc',
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(firstLastSeenHost.lastSeen).to.eql(new Date('2019-02-19T20:42:33.561Z').valueOf());
});
});

265
x-pack/test/api_integration/apis/security_solution/kpi_hosts.ts
View file

@ -6,18 +6,27 @@
*/
import expect from '@kbn/expect';
import { HostsKpiQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
HostsKpiAuthenticationsStrategyResponse,
HostsKpiHostsStrategyResponse,
HostsKpiQueries,
HostsKpiUniqueIpsStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
export default function ({ getService }: FtrProviderContext) {
const retry = getService('retry');
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Kpi Hosts', () => {
describe('With filebeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/kpi_hosts'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/kpi_hosts'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/kpi_hosts')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/kpi_hosts')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -50,88 +59,80 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get KpiHosts data', async () => {
await retry.try(async () => {
const { body: kpiHosts } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsKpiQueries.kpiHosts,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(kpiHosts.hostsHistogram!).to.eql(expectedResult.hostsHistogram);
expect(kpiHosts.hosts!).to.eql(expectedResult.hosts);
const kpiHosts = await bsearch.send<HostsKpiHostsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsKpiQueries.kpiHosts,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiHosts.hostsHistogram).to.eql(expectedResult.hostsHistogram);
expect(kpiHosts.hosts).to.eql(expectedResult.hosts);
});
it('Make sure that we get KpiAuthentications data', async () => {
await retry.try(async () => {
const { body } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsKpiQueries.kpiAuthentications,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
/* We need a very long timeout to avoid returning just partial data.
** https://github.com/elastic/kibana/blob/master/x-pack/test/api_integration/apis/search/search.ts#L18
*/
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(body.authenticationsSuccess!).to.eql(expectedResult.authSuccess);
expect(body.authenticationsSuccessHistogram!).to.eql(expectedResult.authSuccessHistogram);
expect(body.authenticationsFailure!).to.eql(expectedResult.authFailure);
expect(body.authenticationsFailureHistogram!).to.eql(expectedResult.authFailureHistogram);
const body = await bsearch.send<HostsKpiAuthenticationsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsKpiQueries.kpiAuthentications,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(body.authenticationsSuccess).to.eql(expectedResult.authSuccess);
expect(body.authenticationsSuccessHistogram).to.eql(expectedResult.authSuccessHistogram);
expect(body.authenticationsFailure).to.eql(expectedResult.authFailure);
expect(body.authenticationsFailureHistogram).to.eql(expectedResult.authFailureHistogram);
});
it('Make sure that we get KpiUniqueIps data', async () => {
await retry.try(async () => {
const { body } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsKpiQueries.kpiUniqueIps,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(body.uniqueDestinationIps!).to.eql(expectedResult.uniqueDestinationIps);
expect(body.uniqueDestinationIpsHistogram!).to.eql(
expectedResult.uniqueDestinationIpsHistogram
);
expect(body.uniqueSourceIps!).to.eql(expectedResult.uniqueSourceIps);
expect(body.uniqueSourceIpsHistogram!).to.eql(expectedResult.uniqueSourceIpsHistogram);
const body = await bsearch.send<HostsKpiUniqueIpsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsKpiQueries.kpiUniqueIps,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(body.uniqueDestinationIps).to.eql(expectedResult.uniqueDestinationIps);
expect(body.uniqueDestinationIpsHistogram).to.eql(
expectedResult.uniqueDestinationIpsHistogram
);
expect(body.uniqueSourceIps).to.eql(expectedResult.uniqueSourceIps);
expect(body.uniqueSourceIpsHistogram).to.eql(expectedResult.uniqueSourceIpsHistogram);
});
});
describe('With auditbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/auditbeat/kpi_hosts'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/kpi_hosts'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/kpi_hosts')
);
after(
async () =>
await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/kpi_hosts')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -188,79 +189,69 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get KpiHosts data', async () => {
await retry.try(async () => {
const { body: kpiHosts } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsKpiQueries.kpiHosts,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(kpiHosts.hostsHistogram!).to.eql(expectedResult.hostsHistogram);
expect(kpiHosts.hosts!).to.eql(expectedResult.hosts);
const kpiHosts = await bsearch.send<HostsKpiHostsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsKpiQueries.kpiHosts,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiHosts.hostsHistogram).to.eql(expectedResult.hostsHistogram);
expect(kpiHosts.hosts).to.eql(expectedResult.hosts);
});
it('Make sure that we get KpiAuthentications data', async () => {
await retry.try(async () => {
const { body } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsKpiQueries.kpiAuthentications,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(body.authenticationsSuccess!).to.eql(expectedResult.authSuccess);
expect(body.authenticationsSuccessHistogram!).to.eql(expectedResult.authSuccessHistogram);
expect(body.authenticationsFailure!).to.eql(expectedResult.authFailure);
expect(body.authenticationsFailureHistogram!).to.eql(expectedResult.authFailureHistogram);
const body = await bsearch.send<HostsKpiAuthenticationsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsKpiQueries.kpiAuthentications,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(body.authenticationsSuccess).to.eql(expectedResult.authSuccess);
expect(body.authenticationsSuccessHistogram).to.eql(expectedResult.authSuccessHistogram);
expect(body.authenticationsFailure).to.eql(expectedResult.authFailure);
expect(body.authenticationsFailureHistogram).to.eql(expectedResult.authFailureHistogram);
});
it('Make sure that we get KpiUniqueIps data', async () => {
await retry.try(async () => {
const { body } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsKpiQueries.kpiUniqueIps,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(body.uniqueDestinationIps!).to.eql(expectedResult.uniqueDestinationIps);
expect(body.uniqueDestinationIpsHistogram!).to.eql(
expectedResult.uniqueDestinationIpsHistogram
);
expect(body.uniqueSourceIps!).to.eql(expectedResult.uniqueSourceIps);
expect(body.uniqueSourceIpsHistogram!).to.eql(expectedResult.uniqueSourceIpsHistogram);
const body = await bsearch.send<HostsKpiUniqueIpsStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsKpiQueries.kpiUniqueIps,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-*'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(body.uniqueDestinationIps!).to.eql(expectedResult.uniqueDestinationIps);
expect(body.uniqueDestinationIpsHistogram!).to.eql(
expectedResult.uniqueDestinationIpsHistogram
);
expect(body.uniqueSourceIps!).to.eql(expectedResult.uniqueSourceIps);
expect(body.uniqueSourceIpsHistogram!).to.eql(expectedResult.uniqueSourceIpsHistogram);
});
});
});

176
x-pack/test/api_integration/apis/security_solution/kpi_network.ts
View file

@ -7,16 +7,28 @@
import expect from '@kbn/expect';
import { FtrProviderContext } from '../../ftr_provider_context';
import { NetworkKpiQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
NetworkKpiDnsStrategyResponse,
NetworkKpiNetworkEventsStrategyResponse,
NetworkKpiQueries,
NetworkKpiTlsHandshakesStrategyResponse,
NetworkKpiUniqueFlowsStrategyResponse,
NetworkKpiUniquePrivateIpsStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Kpi Network', () => {
describe('With filebeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -66,10 +78,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get KpiNetwork uniqueFlows data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiUniqueFlowsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.uniqueFlows,
timerange: {
interval: '12h',
@ -79,18 +90,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.uniqueFlowId).to.eql(expectedResult.uniqueFlowId);
});
it('Make sure that we get KpiNetwork networkEvents data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiNetworkEventsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.networkEvents,
timerange: {
interval: '12h',
@ -100,18 +109,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.networkEvents).to.eql(expectedResult.networkEvents);
});
it('Make sure that we get KpiNetwork DNS data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiDnsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.dns,
timerange: {
interval: '12h',
@ -121,18 +128,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.dnsQueries).to.eql(expectedResult.dnsQueries);
});
it('Make sure that we get KpiNetwork networkEvents data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiNetworkEventsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.networkEvents,
timerange: {
interval: '12h',
@ -142,18 +147,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.networkEvents).to.eql(expectedResult.networkEvents);
});
it('Make sure that we get KpiNetwork tlsHandshakes data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiTlsHandshakesStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.tlsHandshakes,
timerange: {
interval: '12h',
@ -163,18 +166,17 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.tlsHandshakes).to.eql(expectedResult.tlsHandshakes);
});
it('Make sure that we get KpiNetwork uniquePrivateIps data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiUniquePrivateIpsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.uniquePrivateIps,
timerange: {
interval: '12h',
@ -184,9 +186,9 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['filebeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.uniqueDestinationPrivateIps).to.eql(
expectedResult.uniqueDestinationPrivateIps
@ -202,8 +204,12 @@ export default function ({ getService }: FtrProviderContext) {
});
describe('With packetbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/packetbeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/packetbeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/default')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -219,10 +225,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get KpiNetwork uniqueFlows data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiUniqueFlowsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.uniqueFlows,
timerange: {
interval: '12h',
@ -232,18 +237,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.uniqueFlowId).to.eql(expectedResult.uniqueFlowId);
});
it('Make sure that we get KpiNetwork DNS data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiDnsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.dns,
timerange: {
interval: '12h',
@ -253,18 +256,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.dnsQueries).to.eql(expectedResult.dnsQueries);
});
it('Make sure that we get KpiNetwork networkEvents data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiNetworkEventsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.networkEvents,
timerange: {
interval: '12h',
@ -274,18 +275,17 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.networkEvents).to.eql(expectedResult.networkEvents);
});
it('Make sure that we get KpiNetwork tlsHandshakes data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiTlsHandshakesStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.tlsHandshakes,
timerange: {
interval: '12h',
@ -295,18 +295,16 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.tlsHandshakes).to.eql(expectedResult.tlsHandshakes);
});
it('Make sure that we get KpiNetwork uniquePrivateIps data', async () => {
const { body: kpiNetwork } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const kpiNetwork = await bsearch.send<NetworkKpiUniquePrivateIpsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkKpiQueries.uniquePrivateIps,
timerange: {
interval: '12h',
@ -316,9 +314,9 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(kpiNetwork.uniqueDestinationPrivateIps).to.eql(
expectedResult.uniqueDestinationPrivateIps

95
x-pack/test/api_integration/apis/security_solution/matrix_dns_histogram.ts
View file

@ -6,54 +6,43 @@
*/
import expect from '@kbn/expect';
import request from 'superagent';
import {
MatrixHistogramQuery,
MatrixHistogramType,
NetworkDnsStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
/**
* Function copied from here:
* test/api_integration/apis/search/bsearch.ts
*
* Splits the JSON lines from bsearch
*/
export const parseBfetchResponse = (resp: request.Response): Array<Record<string, any>> => {
return resp.text
.trim()
.split('\n')
.map((item) => JSON.parse(item));
};
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const retry = getService('retry');
const bsearch = getService('bsearch');
describe('Matrix DNS Histogram', () => {
describe('Large data set', () => {
before(() =>
esArchiver.load(
'x-pack/test/functional/es_archives/security_solution/matrix_dns_histogram/large_dns_query'
)
before(
async () =>
await esArchiver.load(
'x-pack/test/functional/es_archives/security_solution/matrix_dns_histogram/large_dns_query'
)
);
after(() =>
esArchiver.unload(
'x-pack/test/functional/es_archives/security_solution/matrix_dns_histogram/large_dns_query'
)
after(
async () =>
await esArchiver.unload(
'x-pack/test/functional/es_archives/security_solution/matrix_dns_histogram/large_dns_query'
)
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
it('Make sure that we get dns data without getting bucket errors when querying large volume of data', async () => {
const { body: networkDns } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkDns = await bsearch.send<NetworkDnsStrategyResponse>({
supertest,
options: {
defaultIndex: ['large_volume_dns_data'],
docValueFields: [],
factoryQueryType: MatrixHistogramQuery,
@ -62,56 +51,14 @@ export default function ({ getService }: FtrProviderContext) {
'{"bool":{"must":[],"filter":[{"match_all":{}}],"should":[],"must_not":[]}}',
isPtrIncluded: false,
timerange: {
interval: '12h',
interval: '12',
to: TO,
from: FROM,
},
})
.expect(200);
if (networkDns.isRunning === true) {
await retry.waitForWithTimeout('bsearch to give us results', 5000, async () => {
const resp = await supertest
.post('/internal/bsearch')
.set('kbn-xsrf', 'true')
.send({
batch: [
{
request: {
id: networkDns.id,
defaultIndex: ['large_volume_dns_data'],
docValueFields: [],
factoryQueryType: MatrixHistogramQuery,
histogramType: MatrixHistogramType.dns,
filterQuery:
'{"bool":{"must":[],"filter":[{"match_all":{}}],"should":[],"must_not":[]}}',
isPtrIncluded: false,
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
},
options: {
strategy: 'securitySolutionSearchStrategy',
},
},
],
});
const parsedResponse = parseBfetchResponse(resp);
// NOTE: I would like this test to be ".to.equal(6604)" but that is flakey as sometimes the query
// does not give me that exact value. It gives me failures as seen here with notes: https://github.com/elastic/kibana/issues/97365
// I don't think this is a bug with the query but possibly a consistency view issue with interacting with the archive
// so we instead loosen this test up a bit to avoid flake.
expect(parsedResponse[0].result.rawResponse.aggregations.dns_count.value).to.be.above(
0
);
return true;
});
} else {
expect(networkDns.isRunning).to.equal(false);
expect(networkDns.rawResponse.aggregations.dns_count.value).to.equal(6604);
}
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkDns.rawResponse.aggregations?.dns_count).to.eql({ value: 6604 });
});
});
});

61
x-pack/test/api_integration/apis/security_solution/network_details.ts
View file

@ -6,59 +6,70 @@
*/
import expect from '@kbn/expect';
import { NetworkQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
NetworkDetailsStrategyResponse,
NetworkQueries,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Network details', () => {
describe('With filebeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default')
);
it('Make sure that we get Network details data', async () => {
const { body } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const body = await bsearch.send<NetworkDetailsStrategyResponse>({
supertest,
options: {
ip: '151.205.0.17',
defaultIndex: ['filebeat-*'],
factoryQueryType: NetworkQueries.details,
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(body.networkDetails!.source!.geo!.continent_name).to.be('North America');
expect(body.networkDetails!.source!.geo!.location!.lat!).to.be(37.751);
expect(body.networkDetails!.host.os!.platform!).to.eql(['raspbian']);
expect(body.networkDetails.source?.geo.continent_name).to.be('North America');
expect(body.networkDetails.source?.geo.location?.lat!).to.be(37.751);
expect(body.networkDetails.host?.os?.platform).to.eql(['raspbian']);
});
});
describe('With packetbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/packetbeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/packetbeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/default')
);
it('Make sure that we get Network details data', async () => {
const { body } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const body = await bsearch.send<NetworkDetailsStrategyResponse>({
supertest,
options: {
ip: '185.53.91.88',
defaultIndex: ['packetbeat-*'],
factoryQueryType: NetworkQueries.details,
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(body.networkDetails!.host.id!).to.eql(['2ce8b1e7d69e4a1d9c6bcddc473da9d9']);
expect(body.networkDetails!.host.name!).to.eql(['zeek-sensor-amsterdam']);
expect(body.networkDetails!.host.os!.platform!).to.eql(['ubuntu']);
expect(body.networkDetails.host?.id).to.eql(['2ce8b1e7d69e4a1d9c6bcddc473da9d9']);
expect(body.networkDetails.host?.name).to.eql(['zeek-sensor-amsterdam']);
expect(body.networkDetails.host?.os?.platform!).to.eql(['ubuntu']);
});
});
});

36
x-pack/test/api_integration/apis/security_solution/network_dns.ts
View file

@ -11,6 +11,7 @@ import {
NetworkDnsEdges,
Direction,
NetworkDnsFields,
NetworkDnsStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -18,20 +19,24 @@ import { FtrProviderContext } from '../../ftr_provider_context';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Network DNS', () => {
describe('With packetbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/packetbeat/dns'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/dns'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/packetbeat/dns')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/dns')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
it('Make sure that we get Dns data and sorting by uniqueDomains ascending', async () => {
const { body: networkDns } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkDns = await bsearch.send<NetworkDnsStrategyResponse>({
supertest,
options: {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
factoryQueryType: NetworkQueries.dns,
@ -45,9 +50,9 @@ export default function ({ getService }: FtrProviderContext) {
to: TO,
from: FROM,
},
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkDns.edges.length).to.be(10);
expect(networkDns.totalCount).to.be(44);
@ -58,10 +63,9 @@ export default function ({ getService }: FtrProviderContext) {
});
it('Make sure that we get Dns data and sorting by uniqueDomains descending', async () => {
const { body: networkDns } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkDns = await bsearch.send<NetworkDnsStrategyResponse>({
supertest,
options: {
ip: '151.205.0.17',
defaultIndex: ['packetbeat-*'],
factoryQueryType: NetworkQueries.dns,
@ -80,9 +84,9 @@ export default function ({ getService }: FtrProviderContext) {
to: TO,
from: FROM,
},
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkDns.edges.length).to.be(10);
expect(networkDns.totalCount).to.be(44);

63
x-pack/test/api_integration/apis/security_solution/network_top_n_flow.ts
View file

@ -12,6 +12,7 @@ import {
Direction,
FlowTargetSourceDest,
NetworkTopTablesFields,
NetworkTopNFlowStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -21,20 +22,24 @@ const EDGE_LENGTH = 10;
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Network Top N Flow', () => {
describe('With filebeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default')
);
const FROM = '2019-02-09T01:57:24.870Z';
const TO = '2019-02-12T01:57:24.870Z';
it('Make sure that we get Source NetworkTopNFlow data with bytes_in descending sort', async () => {
const { body: networkTopNFlow } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkTopNFlow = await bsearch.send<NetworkTopNFlowStrategyResponse>({
supertest,
options: {
defaultIndex: ['filebeat-*'],
factoryQueryType: NetworkQueries.topNFlow,
flowTarget: FlowTargetSourceDest.source,
@ -52,9 +57,9 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkTopNFlow.edges.length).to.be(EDGE_LENGTH);
expect(networkTopNFlow.totalCount).to.be(121);
@ -70,10 +75,9 @@ export default function ({ getService }: FtrProviderContext) {
});
it('Make sure that we get Source NetworkTopNFlow data with bytes_in ascending sort ', async () => {
const { body: networkTopNFlow } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkTopNFlow = await bsearch.send<NetworkTopNFlowStrategyResponse>({
supertest,
options: {
defaultIndex: ['filebeat-*'],
factoryQueryType: 'topNFlow',
filterQuery:
@ -93,9 +97,9 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkTopNFlow.edges.length).to.be(EDGE_LENGTH);
expect(networkTopNFlow.totalCount).to.be(121);
@ -111,10 +115,9 @@ export default function ({ getService }: FtrProviderContext) {
});
it('Make sure that we get Destination NetworkTopNFlow data', async () => {
const { body: networkTopNFlow } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkTopNFlow = await bsearch.send<NetworkTopNFlowStrategyResponse>({
supertest,
options: {
defaultIndex: ['filebeat-*'],
factoryQueryType: 'topNFlow',
filterQuery:
@ -134,9 +137,10 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkTopNFlow.edges.length).to.be(EDGE_LENGTH);
expect(networkTopNFlow.totalCount).to.be(154);
expect(networkTopNFlow.edges[0].node.destination!.flows).to.be(19);
@ -146,10 +150,9 @@ export default function ({ getService }: FtrProviderContext) {
});
it('Make sure that pagination is working in NetworkTopNFlow query', async () => {
const { body: networkTopNFlow } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const networkTopNFlow = await bsearch.send<NetworkTopNFlowStrategyResponse>({
supertest,
options: {
defaultIndex: ['filebeat-*'],
factoryQueryType: 'topNFlow',
filterQuery:
@ -169,9 +172,9 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(networkTopNFlow.edges.length).to.be(EDGE_LENGTH);
expect(networkTopNFlow.totalCount).to.be(121);

29
x-pack/test/api_integration/apis/security_solution/overview_host.ts
View file

@ -8,16 +8,24 @@
import expect from '@kbn/expect';
import { FtrProviderContext } from '../../ftr_provider_context';
import { HostsQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
HostsQueries,
HostsOverviewStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Overview Host', () => {
describe('With auditbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/auditbeat/overview'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/overview'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/overview')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/overview')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -41,12 +49,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get OverviewHost data', async () => {
const {
body: { overviewHost },
} = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const { overviewHost } = await bsearch.send<HostsOverviewStrategyResponse>({
supertest,
options: {
defaultIndex: ['auditbeat-*'],
factoryQueryType: HostsQueries.overview,
timerange: {
@ -56,9 +61,9 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(overviewHost).to.eql(expectedResult);
});
});

77
x-pack/test/api_integration/apis/security_solution/overview_network.ts
View file

@ -7,16 +7,24 @@
import expect from '@kbn/expect';
import { FtrProviderContext } from '../../ftr_provider_context';
import { NetworkQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
NetworkOverviewStrategyResponse,
NetworkQueries,
} from '../../../../plugins/security_solution/common/search_strategy';
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Overview Network', () => {
describe('With filebeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -34,12 +42,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get OverviewNetwork data', async () => {
const {
body: { overviewNetwork },
} = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const { overviewNetwork } = await bsearch.send<NetworkOverviewStrategyResponse>({
supertest,
options: {
defaultIndex: ['filebeat-*'],
factoryQueryType: NetworkQueries.overview,
timerange: {
@ -49,16 +54,21 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(overviewNetwork).to.eql(expectedResult);
});
});
describe('With packetbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/packetbeat/overview'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/overview'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/packetbeat/overview')
);
after(
async () =>
await esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/overview')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -75,12 +85,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get OverviewNetwork data', async () => {
const {
body: { overviewNetwork },
} = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const { overviewNetwork } = await bsearch.send<NetworkOverviewStrategyResponse>({
supertest,
options: {
defaultIndex: ['packetbeat-*'],
factoryQueryType: NetworkQueries.overview,
timerange: {
@ -90,17 +97,20 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(overviewNetwork).to.eql(expectedResult);
});
});
describe('With auditbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/auditbeat/overview'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/overview'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/overview')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/overview')
);
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -117,12 +127,9 @@ export default function ({ getService }: FtrProviderContext) {
};
it('Make sure that we get OverviewNetwork data', async () => {
const {
body: { overviewNetwork },
} = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const { overviewNetwork } = await bsearch.send<NetworkOverviewStrategyResponse>({
supertest,
options: {
defaultIndex: ['auditbeat-*'],
factoryQueryType: NetworkQueries.overview,
timerange: {
@ -132,9 +139,9 @@ export default function ({ getService }: FtrProviderContext) {
},
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(overviewNetwork).to.eql(expectedResult);
});
});

67
x-pack/test/api_integration/apis/security_solution/timeline_details.ts
View file

@ -7,7 +7,11 @@
import expect from '@kbn/expect';
import { sortBy } from 'lodash';
import { TimelineEventsQueries } from '../../../../plugins/security_solution/common/search_strategy';
import {
TimelineEventsQueries,
TimelineEventsDetailsStrategyResponse,
TimelineKpiStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -668,54 +672,49 @@ const EXPECTED_KPI_COUNTS = {
};
export default function ({ getService }: FtrProviderContext) {
const retry = getService('retry');
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Timeline Details', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/filebeat/default'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/filebeat/default')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/filebeat/default')
);
it('Make sure that we get Event Details data', async () => {
await retry.try(async () => {
const {
body: { data: detailsData },
} = await supertest
.post('/internal/search/timelineSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: TimelineEventsQueries.details,
docValueFields: [],
indexName: INDEX_NAME,
inspect: false,
eventId: ID,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(sortBy(detailsData, 'field')).to.eql(sortBy(EXPECTED_DATA, 'field'));
const { data: detailsData } = await bsearch.send<TimelineEventsDetailsStrategyResponse>({
supertest,
options: {
factoryQueryType: TimelineEventsQueries.details,
docValueFields: [],
indexName: INDEX_NAME,
inspect: false,
eventId: ID,
},
strategy: 'timelineSearchStrategy',
});
expect(sortBy(detailsData, 'field')).to.eql(sortBy(EXPECTED_DATA, 'field'));
});
it('Make sure that we get kpi data', async () => {
await retry.try(async () => {
const {
body: { destinationIpCount, hostCount, processCount, sourceIpCount, userCount },
} = await supertest
.post('/internal/search/timelineSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const { destinationIpCount, hostCount, processCount, sourceIpCount, userCount } =
await bsearch.send<TimelineKpiStrategyResponse>({
supertest,
options: {
factoryQueryType: TimelineEventsQueries.kpi,
docValueFields: [],
indexName: INDEX_NAME,
inspect: false,
eventId: ID,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect({ destinationIpCount, hostCount, processCount, sourceIpCount, userCount }).to.eql(
EXPECTED_KPI_COUNTS
);
});
},
strategy: 'timelineSearchStrategy',
});
expect({ destinationIpCount, hostCount, processCount, sourceIpCount, userCount }).to.eql(
EXPECTED_KPI_COUNTS
);
});
});
}

72
x-pack/test/api_integration/apis/security_solution/tls.ts
View file

@ -11,6 +11,7 @@ import {
Direction,
NetworkTlsFields,
FlowTarget,
NetworkTlsStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -83,17 +84,21 @@ const expectedOverviewSourceResult = {
export default function ({ getService }: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Tls Test with Packetbeat', () => {
describe('Tls Test', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/packetbeat/tls'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/tls'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/packetbeat/tls')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/tls')
);
it('Ensure data is returned for FlowTarget.Source', async () => {
const { body: tls } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const tls = await bsearch.send<NetworkTlsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkQueries.tls,
timerange: {
interval: '12h',
@ -112,19 +117,18 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(tls.edges.length).to.be(1);
expect(tls.totalCount).to.be(1);
expect(tls.edges[0].node).to.eql(expectedResult);
});
it('Ensure data is returned for FlowTarget.Destination', async () => {
const { body: tls } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const tls = await bsearch.send<NetworkTlsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkQueries.tls,
timerange: {
interval: '12h',
@ -143,9 +147,9 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(tls.edges.length).to.be(1);
expect(tls.totalCount).to.be(1);
expect(tls.edges[0].node).to.eql(expectedResult);
@ -153,14 +157,17 @@ export default function ({ getService }: FtrProviderContext) {
});
describe('Tls Overview Test', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/packetbeat/tls'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/tls'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/packetbeat/tls')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/packetbeat/tls')
);
it('Ensure data is returned for FlowTarget.Source', async () => {
const { body: tls } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const tls = await bsearch.send<NetworkTlsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkQueries.tls,
timerange: {
interval: '12h',
@ -179,18 +186,18 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(tls.pageInfo).to.eql(expectedOverviewSourceResult.pageInfo);
expect(tls.edges[0]).to.eql(expectedOverviewSourceResult.edges[0]);
});
it('Ensure data is returned for FlowTarget.Destination', async () => {
const { body: tls } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const tls = await bsearch.send<NetworkTlsStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkQueries.tls,
timerange: {
interval: '12h',
@ -209,9 +216,10 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['packetbeat-*'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
expect(tls.pageInfo).to.eql(expectedOverviewDestinationResult.pageInfo);
expect(tls.edges[0]).to.eql(expectedOverviewDestinationResult.edges[0]);
});

152
x-pack/test/api_integration/apis/security_solution/uncommon_processes.ts
View file

@ -13,10 +13,6 @@ import {
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
interface UncommonProcessesResponse {
body: HostsUncommonProcessesStrategyResponse;
}
const FROM = '2000-01-01T00:00:00.000Z';
const TO = '3000-01-01T00:00:00.000Z';
@ -24,24 +20,80 @@ const TO = '3000-01-01T00:00:00.000Z';
const TOTAL_COUNT = 3;
export default function ({ getService }: FtrProviderContext) {
const retry = getService('retry');
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('uncommon_processes', () => {
before(() =>
esArchiver.load('x-pack/test/functional/es_archives/auditbeat/uncommon_processes')
before(
async () =>
await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/uncommon_processes')
);
after(() =>
esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/uncommon_processes')
after(
async () =>
await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/uncommon_processes')
);
it('should return an edge of length 1 when given a pagination of length 1', async () => {
await retry.try(async () => {
const response = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
const response = await bsearch.send<HostsUncommonProcessesStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.uncommonProcesses,
sourceId: 'default',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
pagination: {
activePage: 0,
cursorStart: 0,
fakePossibleCount: 3,
querySize: 1,
},
defaultIndex: ['auditbeat-uncommon-processes'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(response.edges.length).to.be(1);
});
describe('when given a pagination of length 2', () => {
it('should return an edge of length 2 ', async () => {
const response = await bsearch.send<HostsUncommonProcessesStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.uncommonProcesses,
sourceId: 'default',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
pagination: {
activePage: 0,
cursorStart: 0,
fakePossibleCount: 3,
querySize: 2,
},
defaultIndex: ['auditbeat-uncommon-processes'],
docValueFields: [],
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(response.edges.length).to.be(2);
});
});
describe('when given a pagination of length 1', () => {
let response: HostsUncommonProcessesStrategyResponse | null = null;
before(async () => {
response = await bsearch.send<HostsUncommonProcessesStrategyResponse>({
supertest,
options: {
factoryQueryType: HostsQueries.uncommonProcesses,
sourceId: 'default',
timerange: {
@ -58,79 +110,17 @@ export default function ({ getService }: FtrProviderContext) {
defaultIndex: ['auditbeat-uncommon-processes'],
docValueFields: [],
inspect: false,
})
.expect(200);
expect(response!.body.edges.length).to.be(1);
});
});
describe('when given a pagination of length 2', () => {
it('should return an edge of length 2 ', async () => {
await retry.try(async () => {
const response = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsQueries.uncommonProcesses,
sourceId: 'default',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
pagination: {
activePage: 0,
cursorStart: 0,
fakePossibleCount: 3,
querySize: 2,
},
defaultIndex: ['auditbeat-uncommon-processes'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(response!.body.edges.length).to.be(2);
});
});
});
describe('when given a pagination of length 1', () => {
let response: null | UncommonProcessesResponse = null;
before(async () => {
await retry.try(async () => {
response = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: HostsQueries.uncommonProcesses,
sourceId: 'default',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
pagination: {
activePage: 0,
cursorStart: 0,
fakePossibleCount: 3,
querySize: 1,
},
defaultIndex: ['auditbeat-uncommon-processes'],
docValueFields: [],
inspect: false,
wait_for_completion_timeout: '10s',
})
.expect(200);
},
strategy: 'securitySolutionSearchStrategy',
});
});
it('should return an edge of length 1 ', () => {
expect(response!.body.edges.length).to.be(1);
expect(response?.edges.length).to.be(1);
});
it('should return a total count of elements', () => {
expect(response!.body.totalCount).to.be(TOTAL_COUNT);
expect(response?.totalCount).to.be(TOTAL_COUNT);
});
it('should return a single data set with pagination of 1', () => {
@ -152,7 +142,7 @@ export default function ({ getService }: FtrProviderContext) {
},
],
};
expect(response!.body.edges[0].node).to.eql(expected);
expect(response?.edges[0].node).to.eql(expected);
});
});
});

80
x-pack/test/api_integration/apis/security_solution/users.ts
View file

@ -11,6 +11,7 @@ import {
Direction,
NetworkUsersFields,
FlowTarget,
NetworkUsersStrategyResponse,
} from '../../../../plugins/security_solution/common/search_strategy';
import { FtrProviderContext } from '../../ftr_provider_context';
@ -20,53 +21,52 @@ const TO = '3000-01-01T00:00:00.000Z';
const IP = '0.0.0.0';
export default function ({ getService }: FtrProviderContext) {
const retry = getService('retry');
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const bsearch = getService('bsearch');
describe('Users', () => {
describe('With auditbeat', () => {
before(() => esArchiver.load('x-pack/test/functional/es_archives/auditbeat/users'));
after(() => esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/users'));
before(
async () => await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/users')
);
after(
async () => await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/users')
);
it('Ensure data is returned from auditbeat', async () => {
await retry.try(async () => {
const { body: users } = await supertest
.post('/internal/search/securitySolutionSearchStrategy/')
.set('kbn-xsrf', 'true')
.send({
factoryQueryType: NetworkQueries.users,
sourceId: 'default',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-users'],
docValueFields: [],
ip: IP,
flowTarget: FlowTarget.destination,
sort: { field: NetworkUsersFields.name, direction: Direction.asc },
pagination: {
activePage: 0,
cursorStart: 0,
fakePossibleCount: 30,
querySize: 10,
},
inspect: false,
/* We need a very long timeout to avoid returning just partial data.
** https://github.com/elastic/kibana/blob/master/x-pack/test/api_integration/apis/search/search.ts#L18
*/
wait_for_completion_timeout: '10s',
})
.expect(200);
expect(users.edges.length).to.be(1);
expect(users.totalCount).to.be(1);
expect(users.edges[0].node.user!.id).to.eql(['0']);
expect(users.edges[0].node.user!.name).to.be('root');
expect(users.edges[0].node.user!.groupId).to.eql(['0']);
expect(users.edges[0].node.user!.groupName).to.eql(['root']);
expect(users.edges[0].node.user!.count).to.be(1);
const users = await bsearch.send<NetworkUsersStrategyResponse>({
supertest,
options: {
factoryQueryType: NetworkQueries.users,
sourceId: 'default',
timerange: {
interval: '12h',
to: TO,
from: FROM,
},
defaultIndex: ['auditbeat-users'],
docValueFields: [],
ip: IP,
flowTarget: FlowTarget.destination,
sort: { field: NetworkUsersFields.name, direction: Direction.asc },
pagination: {
activePage: 0,
cursorStart: 0,
fakePossibleCount: 30,
querySize: 10,
},
inspect: false,
},
strategy: 'securitySolutionSearchStrategy',
});
expect(users.edges.length).to.be(1);
expect(users.totalCount).to.be(1);
expect(users.edges[0].node.user?.id).to.eql(['0']);
expect(users.edges[0].node.user?.name).to.be('root');
expect(users.edges[0].node.user?.groupId).to.eql(['0']);
expect(users.edges[0].node.user?.groupName).to.eql(['root']);
expect(users.edges[0].node.user?.count).to.be(1);
});
});
});

6
x-pack/test/timeline/security_and_spaces/tests/basic/events.ts
View file

@ -135,6 +135,8 @@ export default ({ getService }: FtrProviderContext) => {
it(`${username} should be able to view alerts from "${featureIds.join(',')}" ${
space != null ? `in space ${space}` : 'when no space specified'
}`, async () => {
// This will be flake until it uses the bsearch service, but these tests aren't operational. Once you do make this operational
// use const bsearch = getService('bsearch');
const resp = await supertestWithoutAuth
.post(`${getSpaceUrlPrefix(space)}${TEST_URL}`)
.auth(username, password)
@ -164,6 +166,8 @@ export default ({ getService }: FtrProviderContext) => {
it(`${username} should NOT be able to view alerts from "${featureIds.join(',')}" ${
space != null ? `in space ${space}` : 'when no space specified'
}`, async () => {
// This will be flake until it uses the bsearch service, but these tests aren't operational. Once you do make this operational
// use const bsearch = getService('bsearch');
const resp = await supertestWithoutAuth
.post(`${getSpaceUrlPrefix(space)}${TEST_URL}`)
.auth(username, password)
@ -183,6 +187,8 @@ export default ({ getService }: FtrProviderContext) => {
it(`${username} should NOT be able to access "${featureIds.join(',')}" ${
space != null ? `in space ${space}` : 'when no space specified'
}`, async () => {
// This will be flake until it uses the bsearch service, but these tests aren't operational. Once you do make this operational
// use const bsearch = getService('bsearch');
await supertestWithoutAuth
.post(`${getSpaceUrlPrefix(space)}${TEST_URL}`)
.auth(username, password)

4
x-pack/test/timeline/security_and_spaces/tests/trial/events.ts
View file

@ -117,6 +117,8 @@ export default ({ getService }: FtrProviderContext) => {
it(`${username} should be able to view alerts from "${featureIds.join(',')}" ${
space != null ? `in space ${space}` : 'when no space specified'
}`, async () => {
// This will be flake until it uses the bsearch service, but these tests aren't operational. Once you do make this operational
// use const bsearch = getService('bsearch');
const resp = await supertestWithoutAuth
.post(`${getSpaceUrlPrefix(space)}${TEST_URL}`)
.auth(username, password)
@ -145,6 +147,8 @@ export default ({ getService }: FtrProviderContext) => {
it(`${username} should NOT be able to access "${featureIds.join(',')}" ${
space != null ? `in space ${space}` : 'when no space specified'
}`, async () => {
// This will be flake until it uses the bsearch service, but these tests aren't operational. Once you do make this operational
// use const bsearch = getService('bsearch');
await supertestWithoutAuth
.post(`${getSpaceUrlPrefix(space)}${TEST_URL}`)
.auth(username, password)