mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 09:48:58 -04:00
# Backport This will backport the following commits from `main` to `9.0`: - [[scout] use project deps as global hooks for parallel tests (#211409)](https://github.com/elastic/kibana/pull/211409) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2025-03-13T09:06:26Z","message":"[scout] use project deps as global hooks for parallel tests (#211409)\n\n## Summary\n\nCurrently we are using `globalSetup` [script in configuration\nfile](https://playwright.dev/docs/test-global-setup-teardown#option-2-configure-globalsetup-and-globalteardown)\nto ingest Elasticsearch data before running the tests in parallel\nagainst the same ES/Kibana instances.\n\nThis approach doesn't work well when you need to adjust `globalSetup`\nlogic based on specific condition, e.g. configuration file defining\nwhere servers are hosted, its credentials, etc.\n\nNot only global hook, but `ScoutConfig` fixture expects an argument to\ndefine where servers configuration is defined:\n\n\ncd502acea1/packages/kbn-scout/src/playwright/fixtures/worker/core_fixtures.ts (L65-L75)
\n\n`testInfo` is how Playwright exposes currently running configuration in\na form of `project` interface:\n[projects](https://playwright.dev/docs/test-projects) can be used to\ngroup tests, e.g. for specific envs or browsers.\n\nUnfortunately `testInfo` is not exposed in global scripts, because in\nPlaywright project design `globalSetup` scripts are run before multiple\nprojects and projects can have its own `setup` hooks via\n[dependencies](https://playwright.dev/docs/test-global-setup-teardown#option-1-project-dependencies):\n\n```\n {\n name: 'setup',\n testMatch: /global.setup\\.ts/,\n },\n {\n name: 'local',\n use: { ...devices['Desktop Chrome'], configName: 'local' },\n dependencies: 'setup',\n },\n``` \n\nWe already use project API to get `serversConfigDir` path, where we plan\nto store local and cloud server configurations. This PR proposes to\ndefine projects as `local` and `cloud` (maybe even separate `cloud-mki`,\n`cloud-ech`) as a way to provide playwright information about servers\nconfiguration.\n\nAdvantages:\n1. we can re-use existing fixtures as-is, without adding custom exported\nhelper functions for ES data ingestion\n2. project dependency is displayed as `setup` in Playwright report\n3. way better and simpler design for consumers:\n```\nimport { globalSetupHook } from '@kbn/scout';\n\nglobalSetupHook('Ingest data to Elasticsearch', async ({ esArchiver, log }) => {\n // add archives to load, if needed\n const archives = [\n testData.ES_ARCHIVES.LOGSTASH,\n ];\n\n log.debug('[setup] loading test data (only if indexes do not exist)...');\n for (const archive of archives) {\n await esArchiver.loadIfNeeded(archive);\n }\n});\n```\n4. it is supported by VSCode Playwright plugin\n<img width=\"1271\" alt=\"Screenshot 2025-02-17 at 11 26 12\"\nsrc=\"https://github.com/user-attachments/assets/ba7eeb38-d39d-4785-9c11-18647599ec4a\"\n/>\n\nI find it extremely useful because you don't need to change env var when\nyou want to switch b/w local or cloud run, all the configurations are\nloaded automatically and you just tick the checkbox!\n\nDisadvantages:\n1. it is important to run `playwright test` with `--project` flag to use\nthe proper configuration\n2. we have to define how `projects` are used for local and cloud\nconfiguration, and make sure it meets requirements of multiple teams. We\ncan expose a way to pass custom project definitions in\n`createPlaywrightConfig` function, but it might complicate the support\neffort when every Team has too many custom projects.\n3. `project` term is something we can't change and might be confusing\n4. Since it is a Playwright feature, we might not have consistency with\nAPI tests runner under Scout\n\nFor reviewers: \n\nPlaying with it locally might give a better understanding about the\npros/cons, especially with IDE playwright plugin installed.\n\nRunning servers with tests:\n```\nnode scripts/scout.js run-tests --serverless=oblt --testTarget=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n\nnode scripts/scout.js run-tests --serverless=oblt --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n```\n\nRunning test only requires passing `project` argument:\n\n```\nnpx playwright test --project=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n\nnpx playwright test --project=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts\n```\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: jennypavlova <jennypavlova94@gmail.com>","sha":"1b30686181df5410e92983ede4002ac5f693a164","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:obs-ux-infra_services","backport:version","test:scout","v9.1.0","v8.19.0"],"title":"[scout] use project deps as global hooks for parallel tests","number":211409,"url":"https://github.com/elastic/kibana/pull/211409","mergeCommit":{"message":"[scout] use project deps as global hooks for parallel tests (#211409)\n\n## Summary\n\nCurrently we are using `globalSetup` [script in configuration\nfile](https://playwright.dev/docs/test-global-setup-teardown#option-2-configure-globalsetup-and-globalteardown)\nto ingest Elasticsearch data before running the tests in parallel\nagainst the same ES/Kibana instances.\n\nThis approach doesn't work well when you need to adjust `globalSetup`\nlogic based on specific condition, e.g. configuration file defining\nwhere servers are hosted, its credentials, etc.\n\nNot only global hook, but `ScoutConfig` fixture expects an argument to\ndefine where servers configuration is defined:\n\n\ncd502acea1/packages/kbn-scout/src/playwright/fixtures/worker/core_fixtures.ts (L65-L75)
\n\n`testInfo` is how Playwright exposes currently running configuration in\na form of `project` interface:\n[projects](https://playwright.dev/docs/test-projects) can be used to\ngroup tests, e.g. for specific envs or browsers.\n\nUnfortunately `testInfo` is not exposed in global scripts, because in\nPlaywright project design `globalSetup` scripts are run before multiple\nprojects and projects can have its own `setup` hooks via\n[dependencies](https://playwright.dev/docs/test-global-setup-teardown#option-1-project-dependencies):\n\n```\n {\n name: 'setup',\n testMatch: /global.setup\\.ts/,\n },\n {\n name: 'local',\n use: { ...devices['Desktop Chrome'], configName: 'local' },\n dependencies: 'setup',\n },\n``` \n\nWe already use project API to get `serversConfigDir` path, where we plan\nto store local and cloud server configurations. This PR proposes to\ndefine projects as `local` and `cloud` (maybe even separate `cloud-mki`,\n`cloud-ech`) as a way to provide playwright information about servers\nconfiguration.\n\nAdvantages:\n1. we can re-use existing fixtures as-is, without adding custom exported\nhelper functions for ES data ingestion\n2. project dependency is displayed as `setup` in Playwright report\n3. way better and simpler design for consumers:\n```\nimport { globalSetupHook } from '@kbn/scout';\n\nglobalSetupHook('Ingest data to Elasticsearch', async ({ esArchiver, log }) => {\n // add archives to load, if needed\n const archives = [\n testData.ES_ARCHIVES.LOGSTASH,\n ];\n\n log.debug('[setup] loading test data (only if indexes do not exist)...');\n for (const archive of archives) {\n await esArchiver.loadIfNeeded(archive);\n }\n});\n```\n4. it is supported by VSCode Playwright plugin\n<img width=\"1271\" alt=\"Screenshot 2025-02-17 at 11 26 12\"\nsrc=\"https://github.com/user-attachments/assets/ba7eeb38-d39d-4785-9c11-18647599ec4a\"\n/>\n\nI find it extremely useful because you don't need to change env var when\nyou want to switch b/w local or cloud run, all the configurations are\nloaded automatically and you just tick the checkbox!\n\nDisadvantages:\n1. it is important to run `playwright test` with `--project` flag to use\nthe proper configuration\n2. we have to define how `projects` are used for local and cloud\nconfiguration, and make sure it meets requirements of multiple teams. We\ncan expose a way to pass custom project definitions in\n`createPlaywrightConfig` function, but it might complicate the support\neffort when every Team has too many custom projects.\n3. `project` term is something we can't change and might be confusing\n4. Since it is a Playwright feature, we might not have consistency with\nAPI tests runner under Scout\n\nFor reviewers: \n\nPlaying with it locally might give a better understanding about the\npros/cons, especially with IDE playwright plugin installed.\n\nRunning servers with tests:\n```\nnode scripts/scout.js run-tests --serverless=oblt --testTarget=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n\nnode scripts/scout.js run-tests --serverless=oblt --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n```\n\nRunning test only requires passing `project` argument:\n\n```\nnpx playwright test --project=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n\nnpx playwright test --project=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts\n```\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: jennypavlova <jennypavlova94@gmail.com>","sha":"1b30686181df5410e92983ede4002ac5f693a164"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211409","number":211409,"mergeCommit":{"message":"[scout] use project deps as global hooks for parallel tests (#211409)\n\n## Summary\n\nCurrently we are using `globalSetup` [script in configuration\nfile](https://playwright.dev/docs/test-global-setup-teardown#option-2-configure-globalsetup-and-globalteardown)\nto ingest Elasticsearch data before running the tests in parallel\nagainst the same ES/Kibana instances.\n\nThis approach doesn't work well when you need to adjust `globalSetup`\nlogic based on specific condition, e.g. configuration file defining\nwhere servers are hosted, its credentials, etc.\n\nNot only global hook, but `ScoutConfig` fixture expects an argument to\ndefine where servers configuration is defined:\n\n\ncd502acea1/packages/kbn-scout/src/playwright/fixtures/worker/core_fixtures.ts (L65-L75)
\n\n`testInfo` is how Playwright exposes currently running configuration in\na form of `project` interface:\n[projects](https://playwright.dev/docs/test-projects) can be used to\ngroup tests, e.g. for specific envs or browsers.\n\nUnfortunately `testInfo` is not exposed in global scripts, because in\nPlaywright project design `globalSetup` scripts are run before multiple\nprojects and projects can have its own `setup` hooks via\n[dependencies](https://playwright.dev/docs/test-global-setup-teardown#option-1-project-dependencies):\n\n```\n {\n name: 'setup',\n testMatch: /global.setup\\.ts/,\n },\n {\n name: 'local',\n use: { ...devices['Desktop Chrome'], configName: 'local' },\n dependencies: 'setup',\n },\n``` \n\nWe already use project API to get `serversConfigDir` path, where we plan\nto store local and cloud server configurations. This PR proposes to\ndefine projects as `local` and `cloud` (maybe even separate `cloud-mki`,\n`cloud-ech`) as a way to provide playwright information about servers\nconfiguration.\n\nAdvantages:\n1. we can re-use existing fixtures as-is, without adding custom exported\nhelper functions for ES data ingestion\n2. project dependency is displayed as `setup` in Playwright report\n3. way better and simpler design for consumers:\n```\nimport { globalSetupHook } from '@kbn/scout';\n\nglobalSetupHook('Ingest data to Elasticsearch', async ({ esArchiver, log }) => {\n // add archives to load, if needed\n const archives = [\n testData.ES_ARCHIVES.LOGSTASH,\n ];\n\n log.debug('[setup] loading test data (only if indexes do not exist)...');\n for (const archive of archives) {\n await esArchiver.loadIfNeeded(archive);\n }\n});\n```\n4. it is supported by VSCode Playwright plugin\n<img width=\"1271\" alt=\"Screenshot 2025-02-17 at 11 26 12\"\nsrc=\"https://github.com/user-attachments/assets/ba7eeb38-d39d-4785-9c11-18647599ec4a\"\n/>\n\nI find it extremely useful because you don't need to change env var when\nyou want to switch b/w local or cloud run, all the configurations are\nloaded automatically and you just tick the checkbox!\n\nDisadvantages:\n1. it is important to run `playwright test` with `--project` flag to use\nthe proper configuration\n2. we have to define how `projects` are used for local and cloud\nconfiguration, and make sure it meets requirements of multiple teams. We\ncan expose a way to pass custom project definitions in\n`createPlaywrightConfig` function, but it might complicate the support\neffort when every Team has too many custom projects.\n3. `project` term is something we can't change and might be confusing\n4. Since it is a Playwright feature, we might not have consistency with\nAPI tests runner under Scout\n\nFor reviewers: \n\nPlaying with it locally might give a better understanding about the\npros/cons, especially with IDE playwright plugin installed.\n\nRunning servers with tests:\n```\nnode scripts/scout.js run-tests --serverless=oblt --testTarget=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n\nnode scripts/scout.js run-tests --serverless=oblt --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n```\n\nRunning test only requires passing `project` argument:\n\n```\nnpx playwright test --project=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts\n\nnpx playwright test --project=local --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts\n```\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: jennypavlova <jennypavlova94@gmail.com>","sha":"1b30686181df5410e92983ede4002ac5f693a164"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
This commit is contained in:
parent
c5c18bbe5e
commit
1101afbf99
24 changed files with 165 additions and 68 deletions
|
@ -1474,7 +1474,8 @@ module.exports = {
|
|||
'playwright/no-slowed-test': 'error',
|
||||
'playwright/no-standalone-expect': 'error',
|
||||
'playwright/no-unsafe-references': 'error',
|
||||
'playwright/no-wait-for-selector': 'warn',
|
||||
'playwright/no-useless-await': 'error',
|
||||
'playwright/no-wait-for-selector': 'error',
|
||||
'playwright/max-nested-describe': ['error', { max: 1 }],
|
||||
'playwright/missing-playwright-await': 'error',
|
||||
'playwright/prefer-comparison-matcher': 'error',
|
||||
|
|
|
@ -12,10 +12,19 @@ import { getRunTarget, stripRunCommand } from './cli_processing';
|
|||
describe('cli_processing', () => {
|
||||
describe('stripRunCommand', () => {
|
||||
it(`should return the correct run command when started with 'npx'`, () => {
|
||||
const argv = ['npx', 'playwright', 'test', '--config', 'path/to/config', '--grep=@svlSearch'];
|
||||
const argv = [
|
||||
'npx',
|
||||
'playwright',
|
||||
'test',
|
||||
'--config',
|
||||
'path/to/config',
|
||||
'--project',
|
||||
'local',
|
||||
'--grep=@svlSearch',
|
||||
];
|
||||
|
||||
expect(stripRunCommand(argv)).toBe(
|
||||
'npx playwright test --config path/to/config --grep=@svlSearch'
|
||||
'npx playwright test --config path/to/config --project local --grep=@svlSearch'
|
||||
);
|
||||
});
|
||||
|
||||
|
@ -26,11 +35,13 @@ describe('cli_processing', () => {
|
|||
'test',
|
||||
'--config',
|
||||
'path/to/config',
|
||||
'--project',
|
||||
'local',
|
||||
'--grep=@svlSearch',
|
||||
];
|
||||
|
||||
expect(stripRunCommand(argv)).toBe(
|
||||
'npx playwright test --config path/to/config --grep=@svlSearch'
|
||||
'npx playwright test --config path/to/config --project local --grep=@svlSearch'
|
||||
);
|
||||
});
|
||||
|
||||
|
|
|
@ -186,9 +186,11 @@ If the servers are already running, you can execute tests independently using ei
|
|||
- Command Line: Use the following command to run tests:
|
||||
|
||||
```bash
|
||||
npx playwright test --config <plugin-path>/ui_tests/playwright.config.ts
|
||||
npx playwright test --config <plugin-path>/ui_tests/playwright.config.ts --project local
|
||||
```
|
||||
|
||||
We use `project` flag to define test target, where tests to be run: local servers or Elastic Cloud. Currently we only support local servers.
|
||||
|
||||
### Contributing
|
||||
|
||||
We welcome contributions to improve and extend `kbn-scout`. This guide will help you get started, add new features, and align with existing project standards.
|
||||
|
|
|
@ -12,11 +12,10 @@ export {
|
|||
expect,
|
||||
test,
|
||||
spaceTest,
|
||||
globalSetupHook,
|
||||
tags,
|
||||
createPlaywrightConfig,
|
||||
createLazyPageObject,
|
||||
ingestTestDataHook,
|
||||
ingestSynthtraceDataHook,
|
||||
} from './src/playwright';
|
||||
export type {
|
||||
ScoutPlaywrightOptions,
|
||||
|
|
|
@ -64,7 +64,8 @@ describe('createPlaywrightConfig', () => {
|
|||
expect(config.timeout).toBe(60000);
|
||||
expect(config.expect?.timeout).toBe(10000);
|
||||
expect(config.outputDir).toBe('./output/test-artifacts');
|
||||
expect(config.projects![0].name).toEqual('chromium');
|
||||
expect(config.projects).toHaveLength(1);
|
||||
expect(config.projects![0].name).toEqual('local');
|
||||
});
|
||||
|
||||
it('should return a Playwright configuration with Scout reporters', () => {
|
||||
|
@ -96,12 +97,17 @@ describe('createPlaywrightConfig', () => {
|
|||
]);
|
||||
});
|
||||
|
||||
it(`should override 'workers' count in Playwright configuration`, () => {
|
||||
it(`should override 'workers' count and add 'setup' project dependency`, () => {
|
||||
const testDir = './my_tests';
|
||||
const workers = 2;
|
||||
|
||||
const config = createPlaywrightConfig({ testDir, workers });
|
||||
expect(config.workers).toBe(workers);
|
||||
|
||||
expect(config.projects).toHaveLength(2);
|
||||
expect(config.projects![0].name).toEqual('setup');
|
||||
expect(config.projects![1].name).toEqual('local');
|
||||
expect(config.projects![1]).toHaveProperty('dependencies', ['setup']);
|
||||
});
|
||||
|
||||
it('should generate and cache runId in process.env.TEST_RUN_ID', () => {
|
||||
|
|
|
@ -27,9 +27,34 @@ export function createPlaywrightConfig(options: ScoutPlaywrightOptions): Playwri
|
|||
process.env.TEST_RUN_ID = runId;
|
||||
}
|
||||
|
||||
const scoutProjects: PlaywrightTestConfig<ScoutTestOptions>['projects'] = [
|
||||
{
|
||||
name: 'local',
|
||||
use: { ...devices['Desktop Chrome'], configName: 'local' },
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
* For parallel tests, we need to add a setup project that runs before the tests project.
|
||||
*/
|
||||
if (options.workers && options.workers > 1) {
|
||||
const parentProject = scoutProjects.find((p) => p.use?.configName);
|
||||
|
||||
scoutProjects.unshift({
|
||||
name: 'setup',
|
||||
use: parentProject?.use ? { ...parentProject.use } : {},
|
||||
testMatch: /global.setup\.ts/,
|
||||
});
|
||||
|
||||
scoutProjects.forEach((project) => {
|
||||
if (project.name !== 'setup') {
|
||||
project.dependencies = ['setup'];
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
return defineConfig<ScoutTestOptions>({
|
||||
testDir: options.testDir,
|
||||
globalSetup: options.globalSetup,
|
||||
/* Run tests in files in parallel */
|
||||
fullyParallel: false,
|
||||
/* Fail the build on CI if you accidentally left test.only in the source code. */
|
||||
|
@ -47,6 +72,7 @@ export function createPlaywrightConfig(options: ScoutPlaywrightOptions): Playwri
|
|||
],
|
||||
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
|
||||
use: {
|
||||
// 'configName' is not defined by default to enforce using '--project' flag when running the tests
|
||||
testIdAttribute: 'data-test-subj',
|
||||
serversConfigDir: SCOUT_SERVERS_ROOT,
|
||||
[VALID_CONFIG_MARKER]: true,
|
||||
|
@ -70,24 +96,6 @@ export function createPlaywrightConfig(options: ScoutPlaywrightOptions): Playwri
|
|||
|
||||
outputDir: './output/test-artifacts', // For other test artifacts (screenshots, videos, traces)
|
||||
|
||||
/* Configure projects for major browsers */
|
||||
projects: [
|
||||
{
|
||||
name: 'chromium',
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
},
|
||||
|
||||
// {
|
||||
// name: 'firefox',
|
||||
// use: { ...devices['Desktop Firefox'] },
|
||||
// },
|
||||
],
|
||||
|
||||
/* Run your local dev server before starting the tests */
|
||||
// webServer: {
|
||||
// command: 'npm run start',
|
||||
// url: 'http://127.0.0.1:3000',
|
||||
// reuseExistingServer: !process.env.CI,
|
||||
// },
|
||||
projects: scoutProjects,
|
||||
});
|
||||
}
|
||||
|
|
|
@ -8,7 +8,13 @@
|
|||
*/
|
||||
|
||||
import { mergeTests } from 'playwright/test';
|
||||
import { apiFixtures, coreWorkerFixtures, scoutSpaceParallelFixture } from './worker';
|
||||
import {
|
||||
apiFixtures,
|
||||
coreWorkerFixtures,
|
||||
esArchiverFixture,
|
||||
scoutSpaceParallelFixture,
|
||||
synthtraceFixture,
|
||||
} from './worker';
|
||||
import type {
|
||||
ApiParallelWorkerFixtures,
|
||||
EsClient,
|
||||
|
@ -52,3 +58,10 @@ export interface ScoutParallelWorkerFixtures extends ApiParallelWorkerFixtures {
|
|||
esClient: EsClient;
|
||||
scoutSpace: ScoutSpaceParallelFixture;
|
||||
}
|
||||
|
||||
export const globalSetup = mergeTests(
|
||||
coreWorkerFixtures,
|
||||
esArchiverFixture,
|
||||
synthtraceFixture,
|
||||
apiFixtures
|
||||
);
|
||||
|
|
|
@ -66,10 +66,15 @@ export const coreWorkerFixtures = base.extend<
|
|||
*/
|
||||
config: [
|
||||
({ log }, use, workerInfo) => {
|
||||
const configName = 'local';
|
||||
const projectUse = workerInfo.project.use as ScoutTestOptions;
|
||||
if (!projectUse.configName) {
|
||||
throw new Error(
|
||||
`Failed to read the 'configName' property. Make sure to run tests with '--project' flag and target enviroment (local or cloud),
|
||||
e.g. 'npx playwright test --project local --config <path_to_Playwright.config.ts>'`
|
||||
);
|
||||
}
|
||||
const serversConfigDir = projectUse.serversConfigDir;
|
||||
const configInstance = createScoutConfig(serversConfigDir, configName, log);
|
||||
const configInstance = createScoutConfig(serversConfigDir, projectUse.configName, log);
|
||||
|
||||
use(configInstance);
|
||||
},
|
||||
|
|
|
@ -8,7 +8,13 @@
|
|||
*/
|
||||
|
||||
import { Readable } from 'stream';
|
||||
import type { ApmFields, Fields, InfraDocument, OtelDocument } from '@kbn/apm-synthtrace-client';
|
||||
import type {
|
||||
ApmFields,
|
||||
Fields,
|
||||
InfraDocument,
|
||||
OtelDocument,
|
||||
SynthtraceGenerator,
|
||||
} from '@kbn/apm-synthtrace-client';
|
||||
import Url from 'url';
|
||||
import type { SynthtraceEsClient } from '@kbn/apm-synthtrace/src/lib/shared/base_client';
|
||||
import {
|
||||
|
@ -17,10 +23,9 @@ import {
|
|||
getOtelSynthtraceEsClient,
|
||||
} from '../../../common/services/synthtrace';
|
||||
import { coreWorkerFixtures } from './core_fixtures';
|
||||
import type { SynthtraceEvents } from '../../global_hooks/synthtrace_ingestion';
|
||||
|
||||
interface SynthtraceFixtureEsClient<TFields extends Fields> {
|
||||
index: (events: SynthtraceEvents<TFields>) => Promise<void>;
|
||||
index: (events: SynthtraceGenerator<TFields>) => Promise<void>;
|
||||
clean: SynthtraceEsClient<TFields>['clean'];
|
||||
}
|
||||
|
||||
|
@ -34,15 +39,12 @@ const useSynthtraceClient = async <TFields extends Fields>(
|
|||
client: SynthtraceEsClient<TFields>,
|
||||
use: (client: SynthtraceFixtureEsClient<TFields>) => Promise<void>
|
||||
) => {
|
||||
const index = async (events: SynthtraceEvents<TFields>) =>
|
||||
const index = async (events: SynthtraceGenerator<TFields>) =>
|
||||
await client.index(Readable.from(Array.from(events).flatMap((event) => event.serialize())));
|
||||
|
||||
const clean = async () => await client.clean();
|
||||
|
||||
await use({ index, clean });
|
||||
|
||||
// cleanup function after all tests have ran
|
||||
await client.clean();
|
||||
};
|
||||
|
||||
export const synthtraceFixture = coreWorkerFixtures.extend<{}, SynthtraceFixture>({
|
||||
|
|
|
@ -65,6 +65,9 @@ const getSynthtraceClient = (
|
|||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* @deprecated Use `globalSetupHook` and synthtrace fixtures instead
|
||||
*/
|
||||
export async function ingestSynthtraceDataHook(config: FullConfig, data: SynthtraceIngestionData) {
|
||||
const log = getLogger();
|
||||
|
||||
|
|
|
@ -7,7 +7,7 @@
|
|||
* License v3.0 only", or the "Server Side Public License, v 1".
|
||||
*/
|
||||
|
||||
import { scoutFixtures, scoutParallelFixtures } from './fixtures';
|
||||
import { scoutFixtures, scoutParallelFixtures, globalSetup } from './fixtures';
|
||||
|
||||
// Scout core fixtures: worker & test scope
|
||||
export const test = scoutFixtures;
|
||||
|
@ -15,6 +15,8 @@ export const test = scoutFixtures;
|
|||
// Scout core 'space aware' fixtures: worker & test scope
|
||||
export const spaceTest = scoutParallelFixtures;
|
||||
|
||||
export const globalSetupHook = globalSetup;
|
||||
|
||||
export { createPlaywrightConfig } from './config';
|
||||
export { createLazyPageObject } from './page_objects/utils';
|
||||
export { expect } from './expect';
|
||||
|
|
|
@ -72,6 +72,7 @@ describe('parseTestFlags', () => {
|
|||
expect(result).toEqual({
|
||||
mode: 'serverless=oblt',
|
||||
configPath: '/path/to/config',
|
||||
testTarget: 'local',
|
||||
headed: false,
|
||||
esFrom: undefined,
|
||||
installDir: undefined,
|
||||
|
@ -82,6 +83,7 @@ describe('parseTestFlags', () => {
|
|||
it(`should parse with correct config and stateful flags`, async () => {
|
||||
const flags = new FlagsReader({
|
||||
config: '/path/to/config',
|
||||
testTarget: 'local',
|
||||
stateful: true,
|
||||
logToFile: false,
|
||||
headed: true,
|
||||
|
@ -93,10 +95,41 @@ describe('parseTestFlags', () => {
|
|||
expect(result).toEqual({
|
||||
mode: 'stateful',
|
||||
configPath: '/path/to/config',
|
||||
testTarget: 'local',
|
||||
headed: true,
|
||||
esFrom: 'snapshot',
|
||||
installDir: undefined,
|
||||
logsDir: undefined,
|
||||
});
|
||||
});
|
||||
|
||||
it(`should throw an error with incorrect '--testTarget' flag`, async () => {
|
||||
const flags = new FlagsReader({
|
||||
config: '/path/to/config',
|
||||
testTarget: 'a',
|
||||
stateful: true,
|
||||
logToFile: false,
|
||||
headed: true,
|
||||
esFrom: 'snapshot',
|
||||
});
|
||||
|
||||
await expect(parseTestFlags(flags)).rejects.toThrow(
|
||||
'invalid --testTarget, expected one of "local", "cloud"'
|
||||
);
|
||||
});
|
||||
|
||||
it(`should throw an error with incorrect '--testTarget' flag set to 'cloud'`, async () => {
|
||||
const flags = new FlagsReader({
|
||||
config: '/path/to/config',
|
||||
testTarget: 'cloud',
|
||||
stateful: true,
|
||||
logToFile: false,
|
||||
headed: true,
|
||||
esFrom: 'snapshot',
|
||||
});
|
||||
validatePlaywrightConfigMock.mockResolvedValueOnce();
|
||||
await expect(parseTestFlags(flags)).rejects.toThrow(
|
||||
'Running tests against Cloud / MKI is not supported yet'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
@ -19,6 +19,7 @@ export interface RunTestsOptions {
|
|||
configPath: string;
|
||||
headed: boolean;
|
||||
mode: CliSupportedServerModes;
|
||||
testTarget: 'local' | 'cloud';
|
||||
esFrom: 'serverless' | 'source' | 'snapshot' | undefined;
|
||||
installDir: string | undefined;
|
||||
logsDir: string | undefined;
|
||||
|
@ -27,11 +28,12 @@ export interface RunTestsOptions {
|
|||
export const TEST_FLAG_OPTIONS: FlagOptions = {
|
||||
...SERVER_FLAG_OPTIONS,
|
||||
boolean: [...(SERVER_FLAG_OPTIONS.boolean || []), 'headed'],
|
||||
string: [...(SERVER_FLAG_OPTIONS.string || []), 'config'],
|
||||
default: { headed: false },
|
||||
string: [...(SERVER_FLAG_OPTIONS.string || []), 'config', 'testTarget'],
|
||||
default: { headed: false, testTarget: 'local' },
|
||||
help: `${SERVER_FLAG_OPTIONS.help}
|
||||
--config Playwright config file path
|
||||
--headed Run Playwright with browser head
|
||||
--testTarget Run tests agaist locally started servers or Cloud deployment / MKI project
|
||||
`,
|
||||
};
|
||||
|
||||
|
@ -39,6 +41,11 @@ export async function parseTestFlags(flags: FlagsReader) {
|
|||
const options = parseServerFlags(flags);
|
||||
const configPath = flags.string('config');
|
||||
const headed = flags.boolean('headed');
|
||||
const testTarget = flags.enum('testTarget', ['local', 'cloud']) || 'local';
|
||||
|
||||
if (testTarget === 'cloud') {
|
||||
throw createFlagError(`Running tests against Cloud / MKI is not supported yet`);
|
||||
}
|
||||
|
||||
if (!configPath) {
|
||||
throw createFlagError(`Path to playwright config is required: --config <file path>`);
|
||||
|
@ -51,5 +58,6 @@ export async function parseTestFlags(flags: FlagsReader) {
|
|||
...options,
|
||||
configPath,
|
||||
headed,
|
||||
testTarget,
|
||||
};
|
||||
}
|
||||
|
|
|
@ -32,7 +32,12 @@ export async function runTests(log: ToolingLog, options: RunTestsOptions) {
|
|||
const playwrightConfigPath = options.configPath;
|
||||
|
||||
const cmd = resolve(REPO_ROOT, './node_modules/.bin/playwright');
|
||||
const cmdArgs = ['test', `--config=${playwrightConfigPath}`, `--grep=${playwrightGrepTag}`];
|
||||
const cmdArgs = [
|
||||
'test',
|
||||
`--config=${playwrightConfigPath}`,
|
||||
`--grep=${playwrightGrepTag}`,
|
||||
`--project=${options.testTarget}`,
|
||||
];
|
||||
|
||||
await withProcRunner(log, async (procs) => {
|
||||
log.info(`scout: Validate Playwright config has tests`);
|
||||
|
@ -78,7 +83,7 @@ export async function runTests(log: ToolingLog, options: RunTestsOptions) {
|
|||
// wait for 5 seconds
|
||||
await silence(log, 5000);
|
||||
|
||||
// Running 'npx playwright test --config=${playwrightConfigPath}'
|
||||
// Running 'npx playwright test --config=${playwrightConfigPath} --project local'
|
||||
await procs.run(`playwright`, {
|
||||
cmd,
|
||||
args: [...cmdArgs, ...(options.headed ? ['--headed'] : [])],
|
||||
|
|
|
@ -13,14 +13,15 @@ export type Protocol = 'http' | 'https';
|
|||
|
||||
export const VALID_CONFIG_MARKER = Symbol('validConfig');
|
||||
|
||||
export type ScoutConfigName = 'local';
|
||||
|
||||
export interface ScoutTestOptions extends PlaywrightTestOptions {
|
||||
serversConfigDir: string;
|
||||
configName: ScoutConfigName;
|
||||
[VALID_CONFIG_MARKER]: boolean;
|
||||
}
|
||||
|
||||
export interface ScoutPlaywrightOptions
|
||||
extends Pick<PlaywrightTestConfig, 'testDir' | 'workers' | 'globalSetup'> {
|
||||
export interface ScoutPlaywrightOptions extends Pick<PlaywrightTestConfig, 'testDir' | 'workers'> {
|
||||
testDir: string;
|
||||
workers?: 1 | 2 | 3; // to keep performance consistent within test suites
|
||||
globalSetup?: string;
|
||||
}
|
||||
|
|
|
@ -52,7 +52,7 @@ export async function startServers(log: ToolingLog, options: StartServerOptions)
|
|||
'\n\n' +
|
||||
dedent`
|
||||
Elasticsearch and Kibana are ready for functional testing.
|
||||
Use 'npx playwright test --config <path_to_Playwright.config.ts>' to run tests'
|
||||
Use 'npx playwright test --config <path_to_Playwright.config.ts> --project local' to run tests'
|
||||
` +
|
||||
'\n\n'
|
||||
);
|
||||
|
|
|
@ -13,10 +13,10 @@ Then you can run the tests multiple times in another terminal with:
|
|||
|
||||
```bash
|
||||
// ESS
|
||||
npx playwright test --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts --grep @ess
|
||||
npx playwright test --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts --project local --grep @ess
|
||||
|
||||
// Serverless
|
||||
npx playwright test --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts --grep @svlSearch
|
||||
npx playwright test --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts --project local --grep @svlSearch
|
||||
|
||||
// @svlOblt, @svlSecurity
|
||||
```
|
||||
|
|
|
@ -9,7 +9,6 @@ import { createPlaywrightConfig } from '@kbn/scout';
|
|||
|
||||
// eslint-disable-next-line import/no-default-export
|
||||
export default createPlaywrightConfig({
|
||||
globalSetup: require.resolve('./parallel_tests/global_setup'),
|
||||
testDir: './parallel_tests',
|
||||
workers: 2,
|
||||
});
|
||||
|
|
|
@ -5,11 +5,10 @@
|
|||
* 2.0.
|
||||
*/
|
||||
|
||||
import { ingestTestDataHook } from '@kbn/scout';
|
||||
import { type FullConfig } from '@playwright/test';
|
||||
import { globalSetupHook } from '@kbn/scout';
|
||||
import { testData } from '../fixtures';
|
||||
|
||||
async function globalSetup(config: FullConfig) {
|
||||
globalSetupHook('Ingest data to Elasticsearch', async ({ esArchiver, log }) => {
|
||||
// add archives to load, if needed
|
||||
const archives = [
|
||||
testData.ES_ARCHIVES.LOGSTASH,
|
||||
|
@ -17,8 +16,8 @@ async function globalSetup(config: FullConfig) {
|
|||
testData.ES_ARCHIVES.ECOMMERCE,
|
||||
];
|
||||
|
||||
return ingestTestDataHook(config, archives);
|
||||
}
|
||||
|
||||
// eslint-disable-next-line import/no-default-export
|
||||
export default globalSetup;
|
||||
log.debug('[setup] loading test data (only if indexes do not exist)...');
|
||||
for (const archive of archives) {
|
||||
await esArchiver.loadIfNeeded(archive);
|
||||
}
|
||||
});
|
|
@ -17,10 +17,10 @@ export type {
|
|||
// re-export from @kbn/scout
|
||||
export {
|
||||
expect,
|
||||
globalSetupHook,
|
||||
tags,
|
||||
createPlaywrightConfig,
|
||||
createLazyPageObject,
|
||||
ingestTestDataHook,
|
||||
} from '@kbn/scout';
|
||||
|
||||
export type {
|
||||
|
@ -29,7 +29,6 @@ export type {
|
|||
KibanaUrl,
|
||||
ScoutLogger,
|
||||
ScoutPage,
|
||||
PageObjects,
|
||||
ScoutServerConfig,
|
||||
ScoutTestConfig,
|
||||
ScoutPlaywrightOptions,
|
||||
|
|
|
@ -16,20 +16,22 @@ Some tests are designed to run sequentially:
|
|||
|
||||
```bash
|
||||
// ESS
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts --grep @ess
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts --project=local --grep @ess
|
||||
|
||||
// Serverless
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts --grep @svlOblt
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts --project=local --grep @svlOblt
|
||||
```
|
||||
|
||||
Some tests are designed to run concurrently (preferred option):
|
||||
|
||||
```bash
|
||||
// ESS
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/parallel_playwright.config.ts --grep @ess
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/parallel_playwright.config.ts
|
||||
--project=local --grep @ess
|
||||
|
||||
// Serverless
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/parallel_playwright.config.ts --grep @svlOblt
|
||||
npx playwright test --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/parallel_playwright.config.ts
|
||||
--project=local --grep @svlOblt
|
||||
```
|
||||
|
||||
Test results are available in `x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/output`
|
||||
|
|
|
@ -5,7 +5,7 @@
|
|||
* 2.0.
|
||||
*/
|
||||
|
||||
import { createPlaywrightConfig } from '@kbn/scout';
|
||||
import { createPlaywrightConfig } from '@kbn/scout-oblt';
|
||||
|
||||
// eslint-disable-next-line import/no-default-export
|
||||
export default createPlaywrightConfig({
|
||||
|
|
|
@ -5,7 +5,7 @@
|
|||
* 2.0.
|
||||
*/
|
||||
|
||||
import { createPlaywrightConfig } from '@kbn/scout';
|
||||
import { createPlaywrightConfig } from '@kbn/scout-oblt';
|
||||
|
||||
// eslint-disable-next-line import/no-default-export
|
||||
export default createPlaywrightConfig({
|
||||
|
|
|
@ -7,7 +7,6 @@
|
|||
"**/*"
|
||||
],
|
||||
"kbn_references": [
|
||||
"@kbn/scout",
|
||||
"@kbn/scout-oblt"
|
||||
],
|
||||
"exclude": [
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue