[MKI][EDR Workflows] Enable MKI on EDR Workflows Cypress tests (#181080)

This PR sets up everything required for running Cypress tests for EDR
Workflows on the MKI QA environment.

MKI pipeline triggered with these changes -
https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/20

---------

Co-authored-by: dkirchan <diamantis.kirchantzoglou@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Paul Tavares <paul.tavares@elastic.co>
Co-authored-by: dkirchan <55240027+dkirchan@users.noreply.github.com>
This commit is contained in:
Konrad Szwarc 2024-04-26 14:10:36 +02:00 committed by GitHub
parent 8623c91cc4
commit 96bf7b1f06
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
40 changed files with 673 additions and 385 deletions

View file

@ -14,6 +14,20 @@ steps:
- exit_status: "*"
limit: 1
- command: "echo 'Running the defend worklows tests in this step"
- command: .buildkite/scripts/pipelines/security_solution_quality_gate/edr_workflows/mki_security_solution_defend_workflows.sh cypress:dw:qa:serverless:run
label: 'Serverless MKI QA Defend Workflows Cypress Tests on Serverless'
key: test_defend_workflows
label: "Serverless MKI QA Defend Workflows - Security Solution Cypress Tests"
agents:
image: family/kibana-ubuntu-2004
imageProject: elastic-images-qa
provider: gcp
enableNestedVirtualization: true
localSsds: 1
localSsdInterface: nvme
machineType: n2-standard-4
timeout_in_minutes: 300
parallelism: 6
retry:
automatic:
- exit_status: '*'
limit: 1

View file

@ -0,0 +1,28 @@
#!/bin/bash
set -euo pipefail
if [ -z "$1" ]
then
echo "No target script from the package.json file, is supplied"
exit 1
fi
source .buildkite/scripts/common/util.sh
source .buildkite/scripts/steps/functional/common_cypress.sh
.buildkite/scripts/bootstrap.sh
export JOB=kibana-defend-workflows-serverless-cypress
buildkite-agent meta-data set "${BUILDKITE_JOB_ID}_is_test_execution_step" "true"
source .buildkite/scripts/pipelines/security_solution_quality_gate/prepare_vault_entries.sh
cd x-pack/plugins/security_solution
set +e
export BK_ANALYTICS_API_KEY=$(vault_get security-solution-quality-gate serverless-cypress-defend-workflows)
echo "--- Running the tests for target $1"
BK_ANALYTICS_API_KEY=$BK_ANALYTICS_API_KEY yarn $1; status=$?; yarn junit:merge || :; exit $status

View file

@ -16,6 +16,8 @@
"cypress:dw:serverless": "NODE_OPTIONS=--openssl-legacy-provider node ./scripts/start_cypress_parallel --config-file ./public/management/cypress/cypress_serverless.config.ts --ftr-config-file ../../test/defend_workflows_cypress/serverless_config",
"cypress:dw:serverless:open": "yarn cypress:dw:serverless open",
"cypress:dw:serverless:run": "yarn cypress:dw:serverless run",
"cypress:dw:qa:serverless": "NODE_OPTIONS=--openssl-legacy-provider node ./scripts/start_cypress_parallel_serverless --config-file ./public/management/cypress/cypress_serverless_qa.config.ts",
"cypress:dw:qa:serverless:run": "yarn cypress:dw:qa:serverless run",
"cypress:dw:serverless:changed-specs-only": "yarn cypress:dw:serverless run --changed-specs-only --env burn=2",
"cypress:dw:endpoint": "echo '\n** WARNING **: Run script `cypress:dw:endpoint` no longer valid! Use `cypress:dw` instead\n'",
"cypress:dw:endpoint:run": "echo '\n** WARNING **: Run script `cypress:dw:endpoint:run` no longer valid! Use `cypress:dw:run` instead\n'",

View file

@ -34,9 +34,11 @@ for more information.
Similarly to Security Solution cypress tests, we use tags in order to select which tests we want to execute on which environment:
- `@serverless` includes a test in the Serverless test suite. You need to explicitly add this tag to any test you want to run against a Serverless environment.
- `@serverlessQA` includes a test in the Serverless test suite for the Kibana release process of serverless. You need to explicitly add this tag to any test you want you run in CI for the second quality gate. These tests should be stable, otherwise they will be blocking the release pipeline. They should be also critical enough, so that when they fail, there's a high chance of an SDH or blocker issue to be reported.
- `@ess` includes a test in the normal, non-Serverless test suite. You need to explicitly add this tag to any test you want to run against a non-Serverless environment.
- `@brokenInServerless` excludes a test from the Serverless test suite (even if it's tagged as `@serverless`). Indicates that a test should run in Serverless, but currently is broken.
- `@skipInServerless` excludes a test from the Serverless test suite (even if it's tagged as `@serverless`). Indicates that we don't want to run the given test in Serverless.
- `@skipInServerlessMKI` excludes a test from any MKI environment, but it will continue being executed as part of the PR process if the `@serverless` tag is present.
Important: if you don't provide any tag, your test won't be executed.

View file

@ -197,6 +197,12 @@ declare global {
options?: Partial<Loggable & Timeoutable>
): Chainable<string>;
task(
name: 'getSessionCookie',
arg: string,
options?: Partial<Loggable & Timeoutable>
): Chainable<{ cookie: string; username: string; password: string }>;
task(
name: 'loadUserAndRole',
arg: LoadUserAndRoleCyTaskOptions,

View file

@ -8,6 +8,7 @@
// @ts-expect-error
import registerDataSession from 'cypress-data-session/src/plugin';
import { merge } from 'lodash';
import { samlAuthentication } from './support/saml_authentication';
import { getVideosForFailedSpecs } from './support/filter_videos';
import { setupToolingLogLevel } from './support/setup_tooling_log_level';
import { createToolingLogger } from '../../../common/endpoint/data_loaders/utils';
@ -81,6 +82,7 @@ export const getCypressBaseConfig = (
registerDataSession(on, config);
// IMPORTANT: setting the log level should happen before any tooling is called
setupToolingLogLevel(config);
samlAuthentication(on, config);
dataLoaders(on, config);

View file

@ -0,0 +1,23 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { defineCypressConfig } from '@kbn/cypress-config';
import { getCypressBaseConfig } from './cypress_base.config';
// eslint-disable-next-line import/no-default-export
export default defineCypressConfig(
getCypressBaseConfig({
e2e: {
experimentalCspAllowList: ['default-src', 'script-src', 'script-src-elem'],
},
env: {
// Uncomment to enable logging
// TOOLING_LOG_LEVEL: 'verbose',
grepTags: '@serverless --@skipInServerless --@brokenInServerless --@skipInServerlessMKI',
},
})
);

View file

@ -63,149 +63,171 @@ const clickArtifactTab = (tabId: string) => {
cy.get(`#${tabId}`).click();
};
describe('Artifact tabs in Policy Details page', { tags: ['@ess', '@serverless'] }, () => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts> | undefined;
describe(
'Artifact tabs in Policy Details page',
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
() => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts> | undefined;
before(() => {
indexEndpointHosts().then((indexEndpoints) => {
endpointData = indexEndpoints;
});
});
after(() => {
removeAllArtifacts();
endpointData?.cleanup();
endpointData = undefined;
});
for (const testData of getArtifactsListTestsData()) {
describe(`${testData.title} tab`, () => {
beforeEach(() => {
login();
removeExceptionsList(testData.createRequestBody.list_id);
before(() => {
indexEndpointHosts().then((indexEndpoints) => {
endpointData = indexEndpoints;
});
});
it(
`[NONE] User cannot see the tab for ${testData.title}`,
// there is no such role in Serverless environment that can read policy but cannot read artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeNone(testData.privilegePrefix);
visitPolicyDetailsPage();
after(() => {
removeAllArtifacts();
cy.get(`#${testData.tabId}`).should('not.exist');
}
);
endpointData?.cleanup();
endpointData = undefined;
});
for (const testData of getArtifactsListTestsData()) {
describe(`${testData.title} tab`, () => {
beforeEach(() => {
login();
removeExceptionsList(testData.createRequestBody.list_id);
});
context(`Given there are no ${testData.title} entries`, () => {
it(
`[READ] User CANNOT add ${testData.title} artifact`,
// there is no such role in Serverless environment that only reads artifacts
`[NONE] User cannot see the tab for ${testData.title}`,
// there is no such role in Serverless environment that can read policy but cannot read artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeRead(testData.privilegePrefix);
loginWithPrivilegeNone(testData.privilegePrefix);
visitPolicyDetailsPage();
cy.get(`#${testData.tabId}`).should('not.exist');
}
);
context(`Given there are no ${testData.title} entries`, () => {
it(
`[READ] User CANNOT add ${testData.title} artifact`,
// there is no such role in Serverless environment that only reads artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeRead(testData.privilegePrefix);
visitArtifactTab(testData.tabId);
cy.getByTestSubj('policy-artifacts-empty-unexisting').should('exist');
cy.getByTestSubj('unexisting-manage-artifacts-button').should('not.exist');
}
);
it(`[ALL] User can add ${testData.title} artifact`, () => {
loginWithPrivilegeAll();
visitArtifactTab(testData.tabId);
cy.getByTestSubj('policy-artifacts-empty-unexisting').should('exist');
cy.getByTestSubj('unexisting-manage-artifacts-button').should('not.exist');
}
);
cy.getByTestSubj('unexisting-manage-artifacts-button').should('exist').click();
it(`[ALL] User can add ${testData.title} artifact`, () => {
loginWithPrivilegeAll();
visitArtifactTab(testData.tabId);
const { formActions, checkResults } = testData.create;
cy.getByTestSubj('policy-artifacts-empty-unexisting').should('exist');
performUserActions(formActions);
cy.getByTestSubj('unexisting-manage-artifacts-button').should('exist').click();
// Add a per policy artifact - but not assign it to any policy
cy.get('[data-test-subj$="-perPolicy"]').click(); // test-subjects are generated in different formats, but all ends with -perPolicy
cy.getByTestSubj(`${testData.pagePrefix}-flyout-submitButton`).click();
const { formActions, checkResults } = testData.create;
// Check new artifact is in the list
for (const checkResult of checkResults) {
cy.getByTestSubj(checkResult.selector).should('have.text', checkResult.value);
}
performUserActions(formActions);
cy.getByTestSubj('policyDetailsPage').should('not.exist');
cy.getByTestSubj('backToOrigin').contains(/^Back to .+ policy$/);
// Add a per policy artifact - but not assign it to any policy
cy.get('[data-test-subj$="-perPolicy"]').click(); // test-subjects are generated in different formats, but all ends with -perPolicy
cy.getByTestSubj(`${testData.pagePrefix}-flyout-submitButton`).click();
// Check new artifact is in the list
for (const checkResult of checkResults) {
cy.getByTestSubj(checkResult.selector).should('have.text', checkResult.value);
}
cy.getByTestSubj('policyDetailsPage').should('not.exist');
cy.getByTestSubj('backToOrigin').contains(/^Back to .+ policy$/);
cy.getByTestSubj('backToOrigin').click();
cy.getByTestSubj('policyDetailsPage').should('exist');
clickArtifactTab(testData.nextTabId); // Make sure the next tab is accessible and backLink doesn't throw errors
cy.getByTestSubj('policyDetailsPage');
});
});
context(`Given there are no assigned ${testData.title} entries`, () => {
beforeEach(() => {
login();
createArtifactList(testData.createRequestBody.list_id);
createPerPolicyArtifact(testData.artifactName, testData.createRequestBody);
cy.getByTestSubj('backToOrigin').click();
cy.getByTestSubj('policyDetailsPage').should('exist');
clickArtifactTab(testData.nextTabId); // Make sure the next tab is accessible and backLink doesn't throw errors
cy.getByTestSubj('policyDetailsPage');
});
});
it(
`[READ] User CANNOT Manage or Assign ${testData.title} artifacts`,
// there is no such role in Serverless environment that only reads artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeRead(testData.privilegePrefix);
context(`Given there are no assigned ${testData.title} entries`, () => {
beforeEach(() => {
login();
createArtifactList(testData.createRequestBody.list_id);
createPerPolicyArtifact(testData.artifactName, testData.createRequestBody);
});
it(
`[READ] User CANNOT Manage or Assign ${testData.title} artifacts`,
// there is no such role in Serverless environment that only reads artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeRead(testData.privilegePrefix);
visitArtifactTab(testData.tabId);
cy.getByTestSubj('policy-artifacts-empty-unassigned').should('exist');
cy.getByTestSubj('unassigned-manage-artifacts-button').should('not.exist');
cy.getByTestSubj('unassigned-assign-artifacts-button').should('not.exist');
}
);
it(`[ALL] User can Manage and Assign ${testData.title} artifacts`, () => {
loginWithPrivilegeAll();
visitArtifactTab(testData.tabId);
cy.getByTestSubj('policy-artifacts-empty-unassigned').should('exist');
cy.getByTestSubj('unassigned-manage-artifacts-button').should('not.exist');
cy.getByTestSubj('unassigned-assign-artifacts-button').should('not.exist');
}
);
// Manage artifacts
cy.getByTestSubj('unassigned-manage-artifacts-button').should('exist').click();
cy.location('pathname').should(
'equal',
`/app/security/administration/${testData.urlPath}`
);
cy.getByTestSubj('backToOrigin').click();
it(`[ALL] User can Manage and Assign ${testData.title} artifacts`, () => {
loginWithPrivilegeAll();
visitArtifactTab(testData.tabId);
// Assign artifacts
cy.getByTestSubj('unassigned-assign-artifacts-button').should('exist').click();
cy.getByTestSubj('policy-artifacts-empty-unassigned').should('exist');
cy.getByTestSubj('artifacts-assign-flyout').should('exist');
cy.getByTestSubj('artifacts-assign-confirm-button').should('be.disabled');
// Manage artifacts
cy.getByTestSubj('unassigned-manage-artifacts-button').should('exist').click();
cy.location('pathname').should(
'equal',
`/app/security/administration/${testData.urlPath}`
);
cy.getByTestSubj('backToOrigin').click();
// Assign artifacts
cy.getByTestSubj('unassigned-assign-artifacts-button').should('exist').click();
cy.getByTestSubj('artifacts-assign-flyout').should('exist');
cy.getByTestSubj('artifacts-assign-confirm-button').should('be.disabled');
cy.getByTestSubj(`${testData.artifactName}_checkbox`).click();
cy.getByTestSubj('artifacts-assign-confirm-button').click();
});
});
context(`Given there are assigned ${testData.title} entries`, () => {
beforeEach(() => {
login();
createArtifactList(testData.createRequestBody.list_id);
yieldFirstPolicyID().then((policyID) => {
createPerPolicyArtifact(testData.artifactName, testData.createRequestBody, policyID);
cy.getByTestSubj(`${testData.artifactName}_checkbox`).click();
cy.getByTestSubj('artifacts-assign-confirm-button').click();
});
});
it(
`[READ] User can see ${testData.title} artifacts but CANNOT assign or remove from policy`,
// there is no such role in Serverless environment that only reads artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeRead(testData.privilegePrefix);
context(`Given there are assigned ${testData.title} entries`, () => {
beforeEach(() => {
login();
createArtifactList(testData.createRequestBody.list_id);
yieldFirstPolicyID().then((policyID) => {
createPerPolicyArtifact(testData.artifactName, testData.createRequestBody, policyID);
});
});
it(
`[READ] User can see ${testData.title} artifacts but CANNOT assign or remove from policy`,
// there is no such role in Serverless environment that only reads artifacts
{ tags: ['@skipInServerless'] },
() => {
loginWithPrivilegeRead(testData.privilegePrefix);
visitArtifactTab(testData.tabId);
// List of artifacts
cy.getByTestSubj('artifacts-collapsed-list-card').should('have.length', 1);
cy.getByTestSubj('artifacts-collapsed-list-card-header-titleHolder').contains(
testData.artifactName
);
// Cannot assign artifacts
cy.getByTestSubj('artifacts-assign-button').should('not.exist');
// Cannot remove from policy
cy.getByTestSubj('artifacts-collapsed-list-card-header-actions-button').click();
cy.getByTestSubj('remove-from-policy-action').should('not.exist');
}
);
it(`[ALL] User can see ${testData.title} artifacts and can assign or remove artifacts from policy`, () => {
loginWithPrivilegeAll();
visitArtifactTab(testData.tabId);
// List of artifacts
@ -214,38 +236,20 @@ describe('Artifact tabs in Policy Details page', { tags: ['@ess', '@serverless']
testData.artifactName
);
// Cannot assign artifacts
cy.getByTestSubj('artifacts-assign-button').should('not.exist');
// Assign artifacts
cy.getByTestSubj('artifacts-assign-button').should('exist').click();
cy.getByTestSubj('artifacts-assign-flyout').should('exist');
cy.getByTestSubj('artifacts-assign-cancel-button').click();
// Cannot remove from policy
// Remove from policy
cy.getByTestSubj('artifacts-collapsed-list-card-header-actions-button').click();
cy.getByTestSubj('remove-from-policy-action').should('not.exist');
}
);
cy.getByTestSubj('remove-from-policy-action').click();
cy.getByTestSubj('confirmModalConfirmButton').click();
it(`[ALL] User can see ${testData.title} artifacts and can assign or remove artifacts from policy`, () => {
loginWithPrivilegeAll();
visitArtifactTab(testData.tabId);
// List of artifacts
cy.getByTestSubj('artifacts-collapsed-list-card').should('have.length', 1);
cy.getByTestSubj('artifacts-collapsed-list-card-header-titleHolder').contains(
testData.artifactName
);
// Assign artifacts
cy.getByTestSubj('artifacts-assign-button').should('exist').click();
cy.getByTestSubj('artifacts-assign-flyout').should('exist');
cy.getByTestSubj('artifacts-assign-cancel-button').click();
// Remove from policy
cy.getByTestSubj('artifacts-collapsed-list-card-header-actions-button').click();
cy.getByTestSubj('remove-from-policy-action').click();
cy.getByTestSubj('confirmModalConfirmButton').click();
cy.contains('Successfully removed');
cy.contains('Successfully removed');
});
});
});
});
}
}
});
);

View file

@ -31,7 +31,7 @@ const loginWithoutAccess = (url: string) => {
loadPage(url);
};
describe('Artifacts pages', { tags: ['@ess', '@serverless'] }, () => {
describe('Artifacts pages', { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] }, () => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts> | undefined;
before(() => {

View file

@ -14,7 +14,7 @@ import { indexEndpointRuleAlerts } from '../../tasks/index_endpoint_rule_alerts'
import { login, ROLE } from '../../tasks/login';
describe('Results', { tags: ['@ess', '@serverless'] }, () => {
describe('Results', { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] }, () => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts> | undefined;
let alertData: ReturnTypeFromChainable<typeof indexEndpointRuleAlerts> | undefined;
const [endpointAgentId, endpointHostname] = generateRandomStringName(2);

View file

@ -21,97 +21,101 @@ import { indexNewCase } from '../../tasks/index_new_case';
import { indexEndpointHosts } from '../../tasks/index_endpoint_hosts';
import { indexEndpointRuleAlerts } from '../../tasks/index_endpoint_rule_alerts';
describe('When accessing Endpoint Response Console', { tags: ['@ess', '@serverless'] }, () => {
const performResponderSanityChecks = () => {
openResponderActionLogFlyout();
// Ensure the popover in the action log date quick select picker is accessible
// (this is especially important for when Responder is displayed from a Timeline)
setResponderActionLogDateRange();
closeResponderActionLogFlyout();
describe(
'When accessing Endpoint Response Console',
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
() => {
const performResponderSanityChecks = () => {
openResponderActionLogFlyout();
// Ensure the popover in the action log date quick select picker is accessible
// (this is especially important for when Responder is displayed from a Timeline)
setResponderActionLogDateRange();
closeResponderActionLogFlyout();
// Global kibana nav bar should remain accessible
// (the login user button seems to be common in both ESS and serverless)
cy.getByTestSubj('userMenuButton').should('be.visible');
// Global kibana nav bar should remain accessible
// (the login user button seems to be common in both ESS and serverless)
cy.getByTestSubj('userMenuButton').should('be.visible');
closeResponder();
};
beforeEach(() => {
login();
});
describe('from Cases', () => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts>;
let caseData: ReturnTypeFromChainable<typeof indexNewCase>;
let alertData: ReturnTypeFromChainable<typeof indexEndpointRuleAlerts>;
let caseAlertActions: ReturnType<typeof addAlertsToCase>;
let alertId: string;
let caseUrlPath: string;
const openCaseAlertDetails = () => {
cy.getByTestSubj(`comment-action-show-alert-${caseAlertActions.comments[alertId]}`).click();
return cy.getByTestSubj('take-action-dropdown-btn').click();
closeResponder();
};
before(() => {
indexNewCase().then((indexCase) => {
caseData = indexCase;
caseUrlPath = `${APP_CASES_PATH}/${indexCase.data.id}`;
beforeEach(() => {
login();
});
describe('from Cases', () => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts>;
let caseData: ReturnTypeFromChainable<typeof indexNewCase>;
let alertData: ReturnTypeFromChainable<typeof indexEndpointRuleAlerts>;
let caseAlertActions: ReturnType<typeof addAlertsToCase>;
let alertId: string;
let caseUrlPath: string;
const openCaseAlertDetails = () => {
cy.getByTestSubj(`comment-action-show-alert-${caseAlertActions.comments[alertId]}`).click();
return cy.getByTestSubj('take-action-dropdown-btn').click();
};
before(() => {
indexNewCase().then((indexCase) => {
caseData = indexCase;
caseUrlPath = `${APP_CASES_PATH}/${indexCase.data.id}`;
});
indexEndpointHosts()
.then((indexEndpoints) => {
endpointData = indexEndpoints;
})
.then(() => {
return indexEndpointRuleAlerts({
endpointAgentId: endpointData.data.hosts[0].agent.id,
}).then((indexedAlert) => {
alertData = indexedAlert;
alertId = alertData.alerts[0]._id;
});
})
.then(() => {
caseAlertActions = addAlertsToCase({
caseId: caseData.data.id,
alertIds: [alertId],
});
});
});
indexEndpointHosts()
.then((indexEndpoints) => {
endpointData = indexEndpoints;
})
.then(() => {
return indexEndpointRuleAlerts({
endpointAgentId: endpointData.data.hosts[0].agent.id,
}).then((indexedAlert) => {
alertData = indexedAlert;
alertId = alertData.alerts[0]._id;
});
})
.then(() => {
caseAlertActions = addAlertsToCase({
caseId: caseData.data.id,
alertIds: [alertId],
});
});
after(() => {
if (caseData) {
caseData.cleanup();
// @ts-expect-error ignore setting to undefined
caseData = undefined;
}
if (endpointData) {
endpointData.cleanup();
// @ts-expect-error ignore setting to undefined
endpointData = undefined;
}
if (alertData) {
alertData.cleanup();
// @ts-expect-error ignore setting to undefined
alertData = undefined;
}
});
it('should display responder option in take action menu', () => {
loadPage(caseUrlPath);
closeAllToasts();
openCaseAlertDetails();
cy.getByTestSubj('endpointResponseActions-action-item').should('be.enabled');
});
it('should display Responder response action interface', () => {
loadPage(caseUrlPath);
closeAllToasts();
openCaseAlertDetails();
cy.getByTestSubj('endpointResponseActions-action-item').click();
performResponderSanityChecks();
});
});
after(() => {
if (caseData) {
caseData.cleanup();
// @ts-expect-error ignore setting to undefined
caseData = undefined;
}
if (endpointData) {
endpointData.cleanup();
// @ts-expect-error ignore setting to undefined
endpointData = undefined;
}
if (alertData) {
alertData.cleanup();
// @ts-expect-error ignore setting to undefined
alertData = undefined;
}
});
it('should display responder option in take action menu', () => {
loadPage(caseUrlPath);
closeAllToasts();
openCaseAlertDetails();
cy.getByTestSubj('endpointResponseActions-action-item').should('be.enabled');
});
it('should display Responder response action interface', () => {
loadPage(caseUrlPath);
closeAllToasts();
openCaseAlertDetails();
cy.getByTestSubj('endpointResponseActions-action-item').click();
performResponderSanityChecks();
});
});
});
}
);

View file

@ -10,64 +10,68 @@ import { indexEndpointHosts } from '../../tasks/index_endpoint_hosts';
import { login } from '../../tasks/login';
import { loadPage } from '../../tasks/common';
describe('Response actions history page', { tags: ['@ess', '@serverless'] }, () => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts>;
describe(
'Response actions history page',
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
() => {
let endpointData: ReturnTypeFromChainable<typeof indexEndpointHosts>;
before(() => {
indexEndpointHosts({ numResponseActions: 11 }).then((indexEndpoints) => {
endpointData = indexEndpoints;
before(() => {
indexEndpointHosts({ numResponseActions: 11 }).then((indexEndpoints) => {
endpointData = indexEndpoints;
});
});
});
beforeEach(() => {
login();
});
beforeEach(() => {
login();
});
after(() => {
if (endpointData) {
endpointData.cleanup();
// @ts-expect-error ignore setting to undefined
endpointData = undefined;
}
});
after(() => {
if (endpointData) {
endpointData.cleanup();
// @ts-expect-error ignore setting to undefined
endpointData = undefined;
}
});
it('retains expanded action details on page reload', () => {
loadPage(`/app/security/administration/response_actions_history`);
cy.getByTestSubj('response-actions-list-expand-button').eq(3).click(); // 4th row on 1st page
cy.getByTestSubj('response-actions-list-details-tray').should('exist');
cy.url().should('include', 'withOutputs');
it('retains expanded action details on page reload', () => {
loadPage(`/app/security/administration/response_actions_history`);
cy.getByTestSubj('response-actions-list-expand-button').eq(3).click(); // 4th row on 1st page
cy.getByTestSubj('response-actions-list-details-tray').should('exist');
cy.url().should('include', 'withOutputs');
// navigate to page 2
cy.getByTestSubj('pagination-button-1').click();
cy.getByTestSubj('response-actions-list-details-tray').should('not.exist');
// reload with URL params on page 2 with existing URL
cy.reload();
cy.getByTestSubj('response-actions-list-details-tray').should('not.exist');
// navigate to page 1
cy.getByTestSubj('pagination-button-0').click();
cy.getByTestSubj('response-actions-list-details-tray').should('exist');
});
it('collapses expanded tray with a single click', () => {
loadPage(`/app/security/administration/response_actions_history`);
// 2nd row on 1st page
cy.getByTestSubj('response-actions-list-expand-button').eq(1).as('2nd-row');
// expand the row
cy.get('@2nd-row').click();
cy.getByTestSubj('response-actions-list-details-tray').should('exist');
cy.url().should('include', 'withOutputs');
// collapse the row
cy.intercept('GET', '/api/endpoint/action*').as('getResponses');
cy.get('@2nd-row').click();
// wait for the API response to come back
// and then see if the tray is actually closed
cy.wait('@getResponses', { timeout: 500 }).then(() => {
// navigate to page 2
cy.getByTestSubj('pagination-button-1').click();
cy.getByTestSubj('response-actions-list-details-tray').should('not.exist');
cy.url().should('not.include', 'withOutputs');
// reload with URL params on page 2 with existing URL
cy.reload();
cy.getByTestSubj('response-actions-list-details-tray').should('not.exist');
// navigate to page 1
cy.getByTestSubj('pagination-button-0').click();
cy.getByTestSubj('response-actions-list-details-tray').should('exist');
});
});
});
it('collapses expanded tray with a single click', () => {
loadPage(`/app/security/administration/response_actions_history`);
// 2nd row on 1st page
cy.getByTestSubj('response-actions-list-expand-button').eq(1).as('2nd-row');
// expand the row
cy.get('@2nd-row').click();
cy.getByTestSubj('response-actions-list-details-tray').should('exist');
cy.url().should('include', 'withOutputs');
// collapse the row
cy.intercept('GET', '/api/endpoint/action*').as('getResponses');
cy.get('@2nd-row').click();
// wait for the API response to come back
// and then see if the tray is actually closed
cy.wait('@getResponses', { timeout: 500 }).then(() => {
cy.getByTestSubj('response-actions-list-details-tray').should('not.exist');
cy.url().should('not.include', 'withOutputs');
});
});
}
);

View file

@ -4,6 +4,7 @@
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { login } from '../../../tasks/login';
import type { PolicyData } from '../../../../../../common/endpoint/types';
import type { CreateAndEnrollEndpointHostResponse } from '../../../../../../scripts/endpoint/common/endpoint_host_services';
import {
@ -16,7 +17,6 @@ import {
import type { IndexedFleetEndpointPolicyResponse } from '../../../../../../common/endpoint/data_loaders/index_fleet_endpoint_policy';
import { createAgentPolicyTask, getEndpointIntegrationVersion } from '../../../tasks/fleet';
import { login } from '../../../tasks/login';
import { enableAllPolicyProtections } from '../../../tasks/endpoint_policy';
import { createEndpointHost } from '../../../tasks/create_endpoint_host';
import { deleteAllLoadedEndpointData } from '../../../tasks/delete_all_endpoint_data';

View file

@ -18,7 +18,7 @@ import {
describe(
'When on the Endpoint List in Security Essentials PLI',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [{ product_line: 'security', product_tier: 'essentials' }],

View file

@ -18,7 +18,7 @@ import { login } from '../../../../tasks/login';
describe(
'Agent policy settings API operations on Essentials',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [

View file

@ -14,7 +14,7 @@ import { getEndpointManagementPageList } from '../../../screens';
describe(
'App Features for Security Complete PLI',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: { productTypes: [{ product_line: 'security', product_tier: 'complete' }] },
},

View file

@ -17,7 +17,7 @@ import {
describe(
'App Features for Security Complete PLI with Endpoint Complete Addon',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [

View file

@ -13,7 +13,7 @@ import { APP_POLICIES_PATH } from '../../../../../../../common/constants';
describe(
'When displaying the Policy Details in Endpoint Essentials PLI',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [

View file

@ -14,7 +14,7 @@ import { getEndpointManagementPageList } from '../../../screens';
describe(
'App Features for Security Essential PLI',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [{ product_line: 'security', product_tier: 'essentials' }],

View file

@ -17,7 +17,7 @@ import {
describe(
'App Features for Security Essentials PLI with Endpoint Essentials Addon',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [

View file

@ -14,7 +14,7 @@ import { APP_POLICIES_PATH } from '../../../../../common/constants';
describe(
'When displaying the Policy Details in Security Essentials PLI',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [{ product_line: 'security', product_tier: 'essentials' }],

View file

@ -27,7 +27,7 @@ import {
describe.skip(
'Roles for Security Essential PLI with Endpoint Essentials addon',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipInServerlessMKI'],
env: {
ftrConfig: {
productTypes: [

View file

@ -25,7 +25,7 @@ export const setupStackServicesUsingCypressConfig = async (config: Cypress.Plugi
password: config.env.KIBANA_PASSWORD,
esUsername: config.env.ELASTICSEARCH_USERNAME,
esPassword: config.env.ELASTICSEARCH_PASSWORD,
asSuperuser: true,
asSuperuser: !config.env.CLOUD_SERVERLESS,
}).then(({ log, ...others }) => {
return {
...others,

View file

@ -5,17 +5,18 @@
* 2.0.
*/
import { kibanaPackageJson } from '@kbn/repo-info';
import type { Client } from '@elastic/elasticsearch';
import type { ToolingLog } from '@kbn/tooling-log';
import type { KbnClient } from '@kbn/test/src/kbn_client';
import { kibanaPackageJson } from '@kbn/repo-info';
import { isFleetServerRunning } from '../../../../scripts/endpoint/common/fleet_server/fleet_server_services';
import type { HostVm } from '../../../../scripts/endpoint/common/types';
import type { BaseVmCreateOptions } from '../../../../scripts/endpoint/common/vm_services';
import { createVm } from '../../../../scripts/endpoint/common/vm_services';
import {
fetchAgentPolicyEnrollmentKey,
fetchFleetAvailableVersions,
fetchFleetServerUrl,
getAgentDownloadUrl,
getAgentFileName,
@ -41,6 +42,8 @@ export interface CreateAndEnrollEndpointHostCIOptions
hostname?: string;
/** If `version` should be exact, or if this is `true`, then the closest version will be used. Defaults to `false` */
useClosestVersionMatch?: boolean;
/** If the environment is MKI */
isMkiEnvironment?: boolean;
}
export interface CreateAndEnrollEndpointHostCIResponse {
@ -63,10 +66,17 @@ export const createAndEnrollEndpointHostCI = async ({
hostname,
version = kibanaPackageJson.version,
useClosestVersionMatch = true,
isMkiEnvironment = false,
}: CreateAndEnrollEndpointHostCIOptions): Promise<CreateAndEnrollEndpointHostCIResponse> => {
let agentVersion = version;
const vmName = hostname ?? `test-host-${Math.random().toString().substring(2, 6)}`;
const fileNameNoExtension = getAgentFileName(version);
if (isMkiEnvironment) {
// MKI env provides own fleet server. We must be sure that currently deployed FS is compatible with agent version we want to deploy.
agentVersion = await fetchFleetAvailableVersions(kbnClient);
}
const fileNameNoExtension = getAgentFileName(agentVersion);
const agentFileName = `${fileNameNoExtension}.tar.gz`;
let agentDownload: DownloadedAgentInfo | undefined;
@ -78,7 +88,7 @@ export const createAndEnrollEndpointHostCI = async ({
log.warning(
`There is no agent installer for ${agentFileName} present on disk, trying to download it now.`
);
const { url: agentUrl } = await getAgentDownloadUrl(version, useClosestVersionMatch, log);
const { url: agentUrl } = await getAgentDownloadUrl(agentVersion, useClosestVersionMatch, log);
agentDownload = await downloadAndStoreAgent(agentUrl, agentFileName);
}

View file

@ -144,6 +144,7 @@ export const dataLoaders = (
): void => {
// Env. variable is set by `cypress_serverless.config.ts`
const isServerless = config.env.IS_SERVERLESS;
const isCloudServerless = Boolean(config.env.CLOUD_SERVERLESS);
const stackServicesPromise = setupStackServicesUsingCypressConfig(config);
const roleAndUserLoaderPromise: Promise<TestRoleAndUserLoader> = stackServicesPromise.then(
({ kbnClient, log }) => {
@ -277,8 +278,8 @@ export const dataLoaders = (
}: {
endpointAgentIds: string[];
}): Promise<DeleteAllEndpointDataResponse> => {
const { esClient } = await stackServicesPromise;
return deleteAllEndpointData(esClient, endpointAgentIds);
const { esClient, log } = await stackServicesPromise;
return deleteAllEndpointData(esClient, log, endpointAgentIds, !isCloudServerless);
},
/**
@ -305,6 +306,8 @@ export const dataLoadersForRealEndpoints = (
config: Cypress.PluginConfigOptions
): void => {
const stackServicesPromise = setupStackServicesUsingCypressConfig(config);
const isServerless = Boolean(config.env.IS_SERVERLESS);
const isCloudServerless = Boolean(config.env.CLOUD_SERVERLESS);
on('task', {
createSentinelOneHost: async () => {
@ -392,7 +395,7 @@ ${s1Info.status}
options: Omit<CreateAndEnrollEndpointHostCIOptions, 'log' | 'kbnClient'>
): Promise<CreateAndEnrollEndpointHostCIResponse> => {
const { kbnClient, log, esClient } = await stackServicesPromise;
const isMkiEnvironment = isServerless && isCloudServerless;
let retryAttempt = 0;
const attemptCreateEndpointHost =
async (): Promise<CreateAndEnrollEndpointHostCIResponse> => {
@ -401,6 +404,7 @@ ${s1Info.status}
const newHost = process.env.CI
? await createAndEnrollEndpointHostCI({
useClosestVersionMatch: true,
isMkiEnvironment,
...options,
log,
kbnClient,

View file

@ -102,7 +102,7 @@ Cypress.Commands.add(
Cypress.on('uncaught:exception', () => false);
// Login as a SOC_MANAGER to properly initialize Security Solution App
// Before any tests runs, Login and visit the Alerts page so that it properly initializes the Security Solution App
before(() => {
login(ROLE.soc_manager);
loadPage('/app/security/alerts');

View file

@ -10,6 +10,7 @@ import type { KbnClient } from '@kbn/test';
import pRetry from 'p-retry';
import { kibanaPackageJson } from '@kbn/repo-info';
import type { ToolingLog } from '@kbn/tooling-log';
import { dump } from '../../../../../scripts/endpoint/common/utils';
import { STARTED_TRANSFORM_STATES } from '../../../../../common/constants';
import {
ENDPOINT_ALERTS_INDEX,
@ -81,10 +82,10 @@ export const cyLoadEndpointDataHandler = async (
if (waitUntilTransformed) {
// need this before indexing docs so that the united transform doesn't
// create a checkpoint with a timestamp after the doc timestamps
await stopTransform(esClient, metadataTransformPrefix);
await stopTransform(esClient, METADATA_CURRENT_TRANSFORM_V2);
await stopTransform(esClient, METADATA_UNITED_TRANSFORM);
await stopTransform(esClient, METADATA_UNITED_TRANSFORM_V2);
await stopTransform(esClient, log, metadataTransformPrefix);
await stopTransform(esClient, log, METADATA_CURRENT_TRANSFORM_V2);
await stopTransform(esClient, log, METADATA_UNITED_TRANSFORM);
await stopTransform(esClient, log, METADATA_UNITED_TRANSFORM_V2);
}
// load data into the system
@ -127,13 +128,23 @@ export const cyLoadEndpointDataHandler = async (
return indexedData;
};
const stopTransform = async (esClient: Client, transformId: string): Promise<void> => {
await esClient.transform.stopTransform({
transform_id: `${transformId}*`,
force: true,
wait_for_completion: true,
allow_no_match: true,
});
const stopTransform = async (
esClient: Client,
log: ToolingLog,
transformId: string
): Promise<void> => {
await esClient.transform
.stopTransform({
transform_id: `${transformId}*`,
force: true,
wait_for_completion: true,
allow_no_match: true,
})
.catch((e) => {
Error.captureStackTrace(e);
log.verbose(dump(e, 8));
throw e;
});
};
const startTransform = async (esClient: Client, transformId: string): Promise<void> => {

View file

@ -0,0 +1,55 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { ToolingLog } from '@kbn/tooling-log';
import type { HostOptions } from '@kbn/test';
import { SamlSessionManager } from '@kbn/test';
import type { SecurityRoleName } from '../../../../common/test';
export const samlAuthentication = async (
on: Cypress.PluginEvents,
config: Cypress.PluginConfigOptions
): Promise<void> => {
const log = new ToolingLog({ level: 'verbose', writeTo: process.stdout });
const kbnHost = config.env.KIBANA_URL || config.env.BASE_URL;
const kbnUrl = new URL(kbnHost);
const hostOptions: HostOptions = {
protocol: kbnUrl.protocol as 'http' | 'https',
hostname: kbnUrl.hostname,
port: parseInt(kbnUrl.port, 10),
username: config.env.ELASTICSEARCH_USERNAME,
password: config.env.ELASTICSEARCH_PASSWORD,
};
on('task', {
getSessionCookie: async (
role: string | SecurityRoleName
): Promise<{ cookie: string; username: string; password: string }> => {
// If config.env.PROXY_ORG is set, it means that proxy service is used to create projects. Define the proxy org filename to override the roles.
const rolesFilename = config.env.PROXY_ORG ? `${config.env.PROXY_ORG}.json` : undefined;
const sessionManager = new SamlSessionManager(
{
hostOptions,
log,
isCloud: config.env.CLOUD_SERVERLESS,
},
rolesFilename
);
return sessionManager.getSessionCookieForRole(role).then((cookie) => {
return {
cookie,
username: hostOptions.username,
password: hostOptions.password,
};
});
},
});
};

View file

@ -48,8 +48,28 @@ export const login: CyLoginTask = (
): ReturnType<typeof sendApiLoginRequest> => {
let username = Cypress.env('KIBANA_USERNAME');
let password = Cypress.env('KIBANA_PASSWORD');
const isServerless = Cypress.env('IS_SERVERLESS');
const isCloudServerless = Cypress.env('CLOUD_SERVERLESS');
if (user) {
if (isServerless && isCloudServerless) {
// MKI QA Cloud Serverless
return cy
.task('getSessionCookie', user)
.then((result) => {
username = result.username;
password = result.password;
// Set cookie asynchronously
return cy.setCookie('sid', result.cookie as string);
})
.then(() => {
// Visit URL after setting cookie
return cy.visit('/');
})
.then(() => {
// Return username and password
return { username, password };
});
} else if (user) {
return cy.task('loadUserAndRole', { name: user }).then((loadedUser) => {
username = loadedUser.username;
password = loadedUser.password;

View file

@ -7,7 +7,9 @@
import type { Client, estypes } from '@elastic/elasticsearch';
import assert from 'assert';
import type { ToolingLog } from '@kbn/tooling-log';
import { createEsClient, isServerlessKibanaFlavor } from './stack_services';
import type { CreatedSecuritySuperuser } from './security_user_services';
import { createSecuritySuperuser } from './security_user_services';
export interface DeleteAllEndpointDataResponse {
@ -22,24 +24,49 @@ export interface DeleteAllEndpointDataResponse {
* **NOTE:** This utility will create a new role and user that has elevated privileges and access to system indexes.
*
* @param esClient
* @param log
* @param endpointAgentIds
* @param asSuperuser
*/
export const deleteAllEndpointData = async (
esClient: Client,
endpointAgentIds: string[]
log: ToolingLog,
endpointAgentIds: string[],
/** If true, then a new user will be created that has full privileges to indexes (especially system indexes) */
asSuperuser: boolean = true
): Promise<DeleteAllEndpointDataResponse> => {
assert(endpointAgentIds.length > 0, 'At least one endpoint agent id must be defined');
const isServerless = await isServerlessKibanaFlavor(esClient);
const unrestrictedUser = isServerless
? { password: 'changeme', username: 'system_indices_superuser', created: false }
: await createSecuritySuperuser(esClient, 'super_superuser');
const esUrl = getEsUrlFromClient(esClient);
const esClientUnrestricted = createEsClient({
url: esUrl,
username: unrestrictedUser.username,
password: unrestrictedUser.password,
});
let esClientUnrestricted = esClient;
if (asSuperuser) {
log.debug(`Looking to use a superuser type of account`);
const isServerless = await isServerlessKibanaFlavor(esClient);
let unrestrictedUser: CreatedSecuritySuperuser | undefined;
if (isServerless) {
log.debug(`In serverless mode. Creating new ES Client using 'system_indices_superuser'`);
unrestrictedUser = {
password: 'changeme',
username: 'system_indices_superuser',
created: false,
};
} else {
log.debug(`Creating new superuser account [super_superuser]`);
unrestrictedUser = await createSecuritySuperuser(esClient, 'super_superuser');
}
if (unrestrictedUser) {
const esUrl = getEsUrlFromClient(esClient);
esClientUnrestricted = createEsClient({
url: esUrl,
username: unrestrictedUser.username,
password: unrestrictedUser.password,
});
}
}
const queryString = endpointAgentIds.map((id) => `(${id})`).join(' OR ');
@ -56,6 +83,8 @@ export const deleteAllEndpointData = async (
conflicts: 'proceed',
});
log.verbose(`All deleted documents:\n`, deleteResponse);
return {
count: deleteResponse.deleted ?? 0,
query: queryString,

View file

@ -42,7 +42,7 @@ import {
import { maybeCreateDockerNetwork, SERVERLESS_NODES, verifyDockerInstalled } from '@kbn/es';
import { resolve } from 'path';
import type * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { captureCallingStack, prefixedOutputLogger } from '../utils';
import { captureCallingStack, dump, prefixedOutputLogger } from '../utils';
import {
createToolingLogger,
RETRYABLE_TRANSIENT_ERRORS,
@ -62,7 +62,6 @@ import {
getFleetElasticsearchOutputHost,
waitForHostToEnroll,
} from '../fleet_services';
import { dump } from '../../endpoint_agent_runner/utils';
import { getLocalhostRealIp } from '../network_services';
import { isLocalhost } from '../is_localhost';

View file

@ -395,11 +395,22 @@ export const fetchIntegrationPolicyList = async (
* Returns the Agent Version that matches the current stack version. Will use `SNAPSHOT` if
* appropriate too.
* @param kbnClient
* @param log
*/
export const getAgentVersionMatchingCurrentStack = async (
kbnClient: KbnClient
kbnClient: KbnClient,
log: ToolingLog = createToolingLogger()
): Promise<string> => {
const kbnStatus = await fetchKibanaStatus(kbnClient);
log.debug(`Kibana status:\n`, kbnStatus);
if (!kbnStatus.version) {
throw new Error(
`Kibana status api response did not include 'version' information - possibly due to invalid credentials`
);
}
const agentVersions = await axios
.get('https://artifacts-api.elastic.co/v1/versions')
.then((response) =>
@ -506,6 +517,24 @@ export const getAgentDownloadUrl = async (
};
};
/**
* Fetches the latest version of the Elastic Agent available for download
* @param kbnClient
*/
export const fetchFleetAvailableVersions = async (kbnClient: KbnClient): Promise<string> => {
return kbnClient
.request<{ items: string[] }>({
method: 'GET',
path: AGENT_API_ROUTES.AVAILABLE_VERSIONS_PATTERN,
headers: {
'elastic-api-version': '2023-10-31',
},
})
.then((response) => response.data.items[0])
.catch(catchAxiosErrorFormatAndThrow);
};
/**
* Given a stack version number, function will return the closest Agent download version available
* for download. THis could be the actual version passed in or lower.

View file

@ -8,11 +8,17 @@
import type { Client } from '@elastic/elasticsearch';
import { userInfo } from 'os';
export interface CreatedSecuritySuperuser {
username: string;
password: string;
created: boolean;
}
export const createSecuritySuperuser = async (
esClient: Client,
username: string = userInfo().username,
password: string = 'changeme'
): Promise<{ username: string; password: string; created: boolean }> => {
): Promise<CreatedSecuritySuperuser> => {
if (!username || !password) {
throw new Error(`username and password require values.`);
}

View file

@ -16,6 +16,7 @@ import { type AxiosResponse } from 'axios';
import type { ClientOptions } from '@elastic/elasticsearch/lib/client';
import fs from 'fs';
import { CA_CERT_PATH } from '@kbn/dev-utils';
import { omit } from 'lodash';
import { createToolingLogger } from '../../../common/endpoint/data_loaders/utils';
import { catchAxiosErrorFormatAndThrow } from '../../../common/endpoint/format_axios_error';
import { isLocalhost } from './is_localhost';
@ -107,15 +108,16 @@ export const createRuntimeServices = async ({
username: _username,
password: _password,
apiKey,
esUsername,
esPassword,
log: _log,
esUsername: _esUsername,
esPassword: _esPassword,
log = createToolingLogger(),
asSuperuser = false,
noCertForSsl,
}: CreateRuntimeServicesOptions): Promise<RuntimeServices> => {
const log = _log ?? createToolingLogger();
let username = _username;
let password = _password;
let esUsername = _esUsername;
let esPassword = _esPassword;
if (asSuperuser) {
const tmpKbnClient = createKbnClient({
@ -131,12 +133,15 @@ export const createRuntimeServices = async ({
if (isServerlessEs) {
log?.warning(
'Creating Security Superuser is not supported in current environment. ES is running in serverless mode. ' +
'Creating Security Superuser is not supported in current environment.\nES is running in serverless mode. ' +
'Will use username [system_indices_superuser] instead.'
);
username = 'system_indices_superuser';
password = 'changeme';
esUsername = 'system_indices_superuser';
esPassword = 'changeme';
} else {
const superuserResponse = await createSecuritySuperuser(
createEsClient({
@ -243,7 +248,12 @@ export const createEsClient = ({
}
if (log) {
log.verbose(`Creating Elasticsearch client options: ${JSON.stringify(clientOptions)}`);
log.verbose(
`Creating Elasticsearch client options: ${JSON.stringify({
...omit(clientOptions, 'tls'),
...(clientOptions.tls ? { tls: { ca: [typeof clientOptions.tls.ca] } } : {}),
})}`
);
}
return new Client(clientOptions);

View file

@ -9,6 +9,7 @@
import type { ToolingLog } from '@kbn/tooling-log';
import chalk from 'chalk';
import { inspect } from 'util';
/**
* Capture and return the calling stack for the context that called this utility.
@ -61,3 +62,12 @@ export const prefixedOutputLogger = (prefix: string, log: ToolingLog): ToolingLo
return proxy;
};
/**
* Safely traverse some content (object, array, etc) and stringify it
* @param content
* @param depth
*/
export const dump = (content: any, depth: number = 5): string => {
return inspect(content, { depth });
};

View file

@ -5,6 +5,7 @@
* 2.0.
*/
import { dump } from '../common/utils';
import { generateVmName } from '../common/vm_services';
import { createAndEnrollEndpointHost } from '../common/endpoint_host_services';
import {
@ -12,7 +13,6 @@ import {
getOrCreateDefaultAgentPolicy,
} from '../common/fleet_services';
import { getRuntimeServices } from './runtime';
import { dump } from './utils';
export const enrollEndpointHost = async (): Promise<string | undefined> => {
let vmName;

View file

@ -1,17 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { inspect } from 'util';
/**
* Safely traverse some content (object, array, etc) and stringify it
* @param content
*/
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export const dump = (content: any): string => {
return inspect(content, { depth: 5 });
};

View file

@ -10,8 +10,8 @@ import type { AxiosRequestConfig } from 'axios';
import axios from 'axios';
import type { KbnClient } from '@kbn/test';
import { SENTINELONE_CONNECTOR_ID } from '@kbn/stack-connectors-plugin/common/sentinelone/constants';
import { dump } from '../common/utils';
import { type RuleResponse } from '../../../common/api/detection_engine';
import { dump } from '../endpoint_agent_runner/utils';
import { createToolingLogger } from '../../../common/endpoint/data_loaders/utils';
import type {
S1SitesListApiResponse,

View file

@ -16,6 +16,7 @@ import cypress from 'cypress';
import grep from '@cypress/grep/src/plugin';
import crypto from 'crypto';
import fs from 'fs';
import { exec } from 'child_process';
import { createFailError } from '@kbn/dev-cli-errors';
import axios, { AxiosError } from 'axios';
import path from 'path';
@ -24,9 +25,11 @@ import pRetry from 'p-retry';
import { ELASTIC_HTTP_VERSION_HEADER } from '@kbn/core-http-common';
import { INITIAL_REST_VERSION } from '@kbn/data-views-plugin/server/constants';
import { exec } from 'child_process';
import { catchAxiosErrorFormatAndThrow } from '../../common/endpoint/format_axios_error';
import { createToolingLogger } from '../../common/endpoint/data_loaders/utils';
import { renderSummaryTable } from './print_run';
import { parseTestFileConfig, retrieveIntegrations } from './utils';
import { prefixedOutputLogger } from '../endpoint/common/utils';
import type { ProductType, Credentials, ProjectHandler } from './project_handler/project_handler';
import { CloudHandler } from './project_handler/cloud_project_handler';
@ -90,11 +93,13 @@ function waitForEsStatusGreen(esUrl: string, auth: string, runnerId: string): Pr
const fetchHealthStatusAttempt = async (attemptNum: number) => {
log.info(`Retry number ${attemptNum} to check if Elasticsearch is green.`);
const response = await axios.get(`${esUrl}/_cluster/health?wait_for_status=green&timeout=50s`, {
headers: {
Authorization: `Basic ${auth}`,
},
});
const response = await axios
.get(`${esUrl}/_cluster/health?wait_for_status=green&timeout=50s`, {
headers: {
Authorization: `Basic ${auth}`,
},
})
.catch(catchAxiosErrorFormatAndThrow);
log.info(`${runnerId}: Elasticsearch is ready with status ${response.data.status}.`);
};
@ -118,11 +123,13 @@ function waitForEsStatusGreen(esUrl: string, auth: string, runnerId: string): Pr
function waitForKibanaAvailable(kbUrl: string, auth: string, runnerId: string): Promise<void> {
const fetchKibanaStatusAttempt = async (attemptNum: number) => {
log.info(`Retry number ${attemptNum} to check if kibana is available.`);
const response = await axios.get(`${kbUrl}/api/status`, {
headers: {
Authorization: `Basic ${auth}`,
},
});
const response = await axios
.get(`${kbUrl}/api/status`, {
headers: {
Authorization: `Basic ${auth}`,
},
})
.catch(catchAxiosErrorFormatAndThrow);
if (response.data.status.overall.level !== 'available') {
throw new Error(`${runnerId}: Kibana is not available. A retry will be triggered soon...`);
} else {
@ -151,11 +158,13 @@ function waitForEsAccess(esUrl: string, auth: string, runnerId: string): Promise
const fetchEsAccessAttempt = async (attemptNum: number) => {
log.info(`Retry number ${attemptNum} to check if can be accessed.`);
await axios.get(`${esUrl}`, {
headers: {
Authorization: `Basic ${auth}`,
},
});
await axios
.get(`${esUrl}`, {
headers: {
Authorization: `Basic ${auth}`,
},
})
.catch(catchAxiosErrorFormatAndThrow);
};
const retryOptions = {
onFailedAttempt: (error: Error | AxiosError) => {
@ -183,9 +192,11 @@ function waitForKibanaLogin(kbUrl: string, credentials: Credentials): Promise<vo
const fetchLoginStatusAttempt = async (attemptNum: number) => {
log.info(`Retry number ${attemptNum} to check if login can be performed.`);
axios.post(`${kbUrl}/internal/security/login`, body, {
headers: API_HEADERS,
});
axios
.post(`${kbUrl}/internal/security/login`, body, {
headers: API_HEADERS,
})
.catch(catchAxiosErrorFormatAndThrow);
};
const retryOptions = {
onFailedAttempt: (error: Error | AxiosError) => {
@ -234,6 +245,7 @@ export const cli = () => {
});
// Checking if API key is either provided via env variable or in ~/.elastic.cloud.json
// This works for either local executions or fallback in case proxy service is unavailable.
if (!process.env.CLOUD_QA_API_KEY && !getApiKeyFromElasticCloudJsonFile()) {
log.error('The API key for the environment needs to be provided with the env var API_KEY.');
log.error(
@ -251,10 +263,17 @@ export const cli = () => {
? process.env.CLOUD_QA_API_KEY
: getApiKeyFromElasticCloudJsonFile();
log.info(`PROXY_URL is defined : ${PROXY_URL !== undefined}`);
log.info(`PROXY_CLIENT_ID is defined : ${PROXY_CLIENT_ID !== undefined}`);
log.info(`PROXY_SECRET is defined : ${PROXY_SECRET !== undefined}`);
log.info(`API_KEY is defined : ${API_KEY !== undefined}`);
let cloudHandler: ProjectHandler;
if (PROXY_URL && PROXY_CLIENT_ID && PROXY_SECRET && (await proxyHealthcheck(PROXY_URL))) {
log.info('Proxy service is up and running, so the tests will run using the proxyHandler.');
cloudHandler = new ProxyHandler(PROXY_URL, PROXY_CLIENT_ID, PROXY_SECRET);
} else if (API_KEY) {
log.info('Proxy service is unavailable, so the tests will run using the cloudHandler.');
cloudHandler = new CloudHandler(API_KEY, BASE_ENV_URL);
} else {
log.info('PROXY_URL or API KEY which are needed to create project could not be retrieved.');
@ -330,6 +349,12 @@ ${JSON.stringify(argv, null, 2)}
cypressConfigFile.env.grepTags = '@serverlessQA --@skipInServerless --@skipInServerlessMKI';
}
if (cypressConfigFile.env?.TOOLING_LOG_LEVEL) {
createToolingLogger.defaultLogLevel = cypressConfigFile.env.TOOLING_LOG_LEVEL;
}
// eslint-disable-next-line require-atomic-updates
log = prefixedOutputLogger('cy.parallel(svl)', createToolingLogger());
const tier: string = argv.tier;
const endpointAddon: boolean = argv.endpointAddon;
const cloudAddon: boolean = argv.cloudAddon;
@ -411,6 +436,7 @@ ${JSON.stringify(cypressConfigFile, null, 2)}
? getProductTypes(tier, endpointAddon, cloudAddon)
: (parseTestFileConfig(filePath).productTypes as ProductType[]);
log.info(`Running spec file: ${filePath}`);
log.info(`${id}: Creating project ${PROJECT_NAME}...`);
// Creating project for the test to run
const project = await cloudHandler.createSecurityProject(
@ -420,7 +446,7 @@ ${JSON.stringify(cypressConfigFile, null, 2)}
);
if (!project) {
log.info('Failed to create project.');
log.error('Failed to create project.');
// eslint-disable-next-line no-process-exit
return process.exit(1);
}
@ -437,7 +463,7 @@ ${JSON.stringify(cypressConfigFile, null, 2)}
const credentials = await cloudHandler.resetCredentials(project.id, id);
if (!credentials) {
log.info('Credentials could not be reset.');
log.error('Credentials could not be reset.');
// eslint-disable-next-line no-process-exit
return process.exit(1);
}
@ -460,16 +486,21 @@ ${JSON.stringify(cypressConfigFile, null, 2)}
// Wait until application is ready
await waitForKibanaLogin(project.kb_url, credentials);
// Check if proxy service is used to define which org executes the tests.
const proxyOrg =
cloudHandler instanceof ProxyHandler ? project.proxy_org_name : undefined;
log.info(`Proxy Organization used id : ${proxyOrg}`);
// Normalized the set of available env vars in cypress
const cyCustomEnv = {
CYPRESS_BASE_URL: project.kb_url,
BASE_URL: project.kb_url,
ELASTICSEARCH_URL: project.es_url,
ELASTICSEARCH_USERNAME: credentials.username,
ELASTICSEARCH_PASSWORD: credentials.password,
// Used in order to handle the correct role_users file loading.
PROXY_ORG: PROXY_URL ? project.proxy_org_name : undefined,
PROXY_ORG: proxyOrg,
KIBANA_URL: project.kb_url,
KIBANA_USERNAME: credentials.username,
@ -478,6 +509,7 @@ ${JSON.stringify(cypressConfigFile, null, 2)}
// Both CLOUD_SERVERLESS and IS_SERVERLESS are used by the cypress tests.
CLOUD_SERVERLESS: true,
IS_SERVERLESS: true,
CLOUD_QA_API_KEY: API_KEY,
// TEST_CLOUD is used by SvlUserManagerProvider to define if testing against cloud.
TEST_CLOUD: 1,
};

View file

@ -37,7 +37,7 @@ Before considering adding a new Cypress tests, please make sure you have added u
is to test that the user interface operates as expected, hence, you should not be using this tool to test REST API or data contracts.
First take a look to the [**Development Best Practices**](#development-best-practices) section.
Then check check [**Folder structure**](#folder-structure) section to know where is the best place to put your test, [**Test data**](#test-data) section if you need to create any type
Then check [**Folder structure**](#folder-structure) section to know where is the best place to put your test, [**Test data**](#test-data) section if you need to create any type
of data for your test, [**Running the tests**](#running-the-tests) to know how to execute the tests and [**Debugging your test**](#debugging-your-test) to debug your test if needed.
Please, before opening a PR with the new test, please make sure that the test fails. If you never see your test fail you dont know if your test is actually testing the right thing, or testing anything at all.
@ -45,7 +45,7 @@ Please, before opening a PR with the new test, please make sure that the test fa
Note that we use tags in order to select which tests we want to execute:
- `@serverless` includes a test in the Serverless test suite for PRs (the so-called first quality gate) and QA environment for the periodic pipeline. You need to explicitly add this tag to any test you want to run in CI for serverless.
- `@serverlessQA` includes a test in the Serverless test suite for the Kibana release process of serverless. You need to explicitly add this tag to any test you want yo run in CI for the second quality gate. These tests should be stable, otherviswe they will be blocking the release pipeline. They should be alsy critical enough, so that when they fail, there's a high chance of an SDH or blocker issue to be reported.
- `@serverlessQA` includes a test in the Serverless test suite for the Kibana release process of serverless. You need to explicitly add this tag to any test you want you run in CI for the second quality gate. These tests should be stable, otherwise they will be blocking the release pipeline. They should be also critical enough, so that when they fail, there's a high chance of an SDH or blocker issue to be reported.
- `@ess` includes a test in the normal, non-Serverless test suite. You need to explicitly add this tag to any test you want to run against a non-Serverless environment.
- `@skipInEss` excludes a test from the non-Serverless test suite. The test will not be executed as part for the PR process. All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
- `@skipInServerlessMKI` excludes a test from the execution on any MKI environment (even if it's tagged as `@serverless` or `@serverlessQA`). Could indicate many things, e.g. "the test is flaky in Serverless MKI", "the test has been temporarily excluded, see the comment above why". All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
@ -115,7 +115,8 @@ describe(
},
},
},
...
// ...
);
```
Note that this configuration doesn't work for local development. In this case, you need to update the configuration files: `../config` and `../serverless_config`, but you shouldn't commit these changes.
@ -245,7 +246,7 @@ cy.task('esArchiverUnload', { archiveName: 'overview'});
You can also use archives stored in `kibana/x-pack/test/functional/es_archives`. In order to do sow uste it on the tests as follow.
```typescript
cy.task('esArchiverLoad', { archiveName: 'security_solution/alias' }, type: 'ftr');
cy.task('esArchiverLoad', { archiveName: 'security_solution/alias', type: 'ftr'});
cy.task('esArchiverUnload', { archiveName: 'security_solution/alias', type:'ftr'});
```
@ -294,7 +295,7 @@ describe(
],
},
},
},
});
```
Per the way we set the environment during the execution process on CI, the above configuration is going to be valid when the test is executed on headless mode.
@ -431,7 +432,7 @@ describe(
],
},
},
},
});
```
For test developing or test debugging purposes on QA, you have avaialable the following options: