Cypress baseline for osquery (#102265)

* Cypress baseline for osquery

* fix types

* Update visual_config.ts

Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

src/dev/typescript/projects.ts

@@ -22,6 +22,9 @@ export const PROJECTS = [
   new Project(resolve(REPO_ROOT, 'x-pack/plugins/security_solution/cypress/tsconfig.json'), {
     name: 'security_solution/cypress',
   }),
+  new Project(resolve(REPO_ROOT, 'x-pack/plugins/osquery/cypress/tsconfig.json'), {
+    name: 'osquery/cypress',
+  }),
   new Project(resolve(REPO_ROOT, 'x-pack/plugins/apm/e2e/tsconfig.json'), {
     name: 'apm/cypress',
     disableTypeCheck: true,

x-pack/plugins/osquery/cypress/README.md

@@ -0,0 +1,138 @@
+# Cypress Tests
+
+The `osquery/cypress` directory contains functional UI tests that execute using [Cypress](https://www.cypress.io/).
+
+## Running the tests
+
+There are currently three ways to run the tests, comprised of two execution modes and two target environments, which will be detailed below.
+
+### Execution modes
+
+#### Interactive mode
+
+When you run Cypress in interactive mode, an interactive runner is displayed that allows you to see commands as they execute while also viewing the application under test. For more information, please see [cypress documentation](https://docs.cypress.io/guides/core-concepts/test-runner.html#Overview).
+
+#### Headless mode
+
+A headless browser is a browser simulation program that does not have a user interface. These programs operate like any other browser, but do not display any UI. This is why meanwhile you are executing the tests on this mode you are not going to see the application under test. Just the output of the test is displayed on the terminal once the execution is finished.
+
+### Target environments
+
+#### FTR (CI)
+
+This is the configuration used by CI. It uses the FTR to spawn both a Kibana instance (http://localhost:5620) and an Elasticsearch instance (http://localhost:9220) with a preloaded minimum set of data (see preceding "Test data" section), and then executes cypress against this stack. You can find this configuration in `x-pack/test/security_solution_cypress`
+
+### Test Execution: Examples
+
+#### FTR + Headless (Chrome)
+
+Since this is how tests are run on CI, this will likely be the configuration you want to reproduce failures locally, etc.
+
+```shell
+# bootstrap kibana from the project root
+yarn kbn bootstrap
+
+# build the plugins/assets that cypress will execute against
+node scripts/build_kibana_platform_plugins
+
+# launch the cypress test runner
+cd x-pack/plugins/security_solution
+yarn cypress:run-as-ci
+```
+#### FTR + Interactive
+
+This is the preferred mode for developing new tests.
+
+```shell
+# bootstrap kibana from the project root
+yarn kbn bootstrap
+
+# build the plugins/assets that cypress will execute against
+node scripts/build_kibana_platform_plugins
+
+# launch the cypress test runner
+cd x-pack/plugins/security_solution
+yarn cypress:open-as-ci
+```
+
+Note that you can select the browser you want to use on the top right side of the interactive runner.
+
+## Folder Structure
+
+### integration/
+
+Cypress convention. Contains the specs that are going to be executed.
+
+### fixtures/
+
+Cypress convention. Fixtures are used as external pieces of static data when we stub responses.
+
+### plugins/
+
+Cypress convention. As a convenience, by default Cypress will automatically include the plugins file cypress/plugins/index.js before every single spec file it runs.
+
+### screens/
+
+Contains the elements we want to interact with in our tests.
+
+Each file inside the screens folder represents a screen in our application.
+
+### tasks/
+
+_Tasks_ are functions that may be reused across tests.
+
+Each file inside the tasks folder represents a screen of our application. 
+
+## Test data
+
+The data the tests need:
+
+- Is generated on the fly using our application APIs (preferred way)
+- Is ingested on the ELS instance using the `es_archive` utility
+
+### How to generate a new archive
+
+**Note:** As mentioned above, archives are only meant to contain external data, e.g. beats data. Due to the tendency for archived domain objects (rules, signals) to quickly become out of date, it is strongly suggested that you generate this data within the test, through interaction with either the UI or the API.
+
+We use es_archiver to manage the data that our Cypress tests need.
+
+1. Set up a clean instance of kibana and elasticsearch (if this is not possible, try to clean/minimize the data that you are going to archive).
+2. With the kibana and elasticsearch instance up and running, create the data that you need for your test.
+3. When you are sure that you have all the data you need run the following command from: `x-pack/plugins/security_solution`
+
+```sh
+node ../../../scripts/es_archiver save <nameOfTheFolderWhereDataIsSaved> <indexPatternsToBeSaved>  --dir ../../test/security_solution_cypress/es_archives --config ../../../test/functional/config.js --es-url http://<elasticsearchUsername>:<elasticsearchPassword>@<elasticsearchHost>:<elasticsearchPort>
+```
+
+Example:
+
+```sh
+node ../../../scripts/es_archiver save custom_rules ".kibana",".siem-signal*"  --dir ../../test/security_solution_cypress/es_archives --config ../../../test/functional/config.js --es-url http://elastic:changeme@localhost:9220
+```
+
+Note that the command will create the folder if it does not exist.
+
+## Development Best Practices
+
+### Clean up the state 
+
+Remember to clean up the state of the test after its execution, typically with the `cleanKibana` function. Be mindful of failure scenarios, as well: if your test fails, will it leave the environment in a recoverable state?
+
+### Minimize the use of es_archive
+
+When possible, create all the data that you need for executing the tests using the application APIS or the UI.
+
+### Speed up test execution time
+
+Loading the web page takes a big amount of time, in order to minimize that impact, the following points should be
+taken into consideration until another solution is implemented:
+
+- Group the tests that are similar in different contexts.
+- For every context login only once, clean the state between tests if needed without re-loading the page.
+- All tests in a spec file must be order-independent.
+
+Remember that minimizing the number of times the web page is loaded, we minimize as well the execution time.
+
+## Linting
+
+Optional linting rules for Cypress and linting setup can be found [here](https://github.com/cypress-io/eslint-plugin-cypress#usage)

x-pack/plugins/osquery/cypress/cypress.json

@@ -0,0 +1,14 @@
+{
+  "baseUrl": "http://localhost:5620",
+  "defaultCommandTimeout": 60000,
+  "execTimeout": 120000,
+  "pageLoadTimeout": 120000,
+  "nodeVersion": "system",
+  "retries": {
+    "runMode": 2
+  },
+  "trashAssetsBeforeRuns": false,
+  "video": false,
+  "viewportHeight": 900,
+  "viewportWidth": 1440
+}

x-pack/plugins/osquery/cypress/integration/osquery_manager.spec.ts

@@ -0,0 +1,29 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { HEADER } from '../screens/osquery';
+import { OSQUERY_NAVIGATION_LINK } from '../screens/navigation';
+
+import { INTEGRATIONS, OSQUERY, openNavigationFlyout, navigateTo } from '../tasks/navigation';
+import { addIntegration } from '../tasks/integrations';
+
+describe('Osquery Manager', () => {
+  before(() => {
+    navigateTo(INTEGRATIONS);
+    addIntegration('Osquery Manager');
+  });
+
+  it('Displays Osquery on the navigation flyout once installed ', () => {
+    openNavigationFlyout();
+    cy.get(OSQUERY_NAVIGATION_LINK).should('exist');
+  });
+
+  it('Displays Live queries history title when navigating to Osquery', () => {
+    navigateTo(OSQUERY);
+    cy.get(HEADER).should('have.text', 'Live queries history');
+  });
+});

x-pack/plugins/osquery/cypress/plugins/index.js

@@ -0,0 +1,29 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+/// <reference types="cypress" />
+// ***********************************************************
+// This example plugins/index.js can be used to load plugins
+//
+// You can change the location of this file or turn off loading
+// the plugins file with the 'pluginsFile' configuration option.
+//
+// You can read more here:
+// https://on.cypress.io/plugins-guide
+// ***********************************************************
+
+// This function is called when a project is opened or re-opened (e.g. due to
+// the project's config changing)
+
+/**
+ * @type {Cypress.PluginConfig}
+ */
+// eslint-disable-next-line @typescript-eslint/no-unused-vars
+module.exports = (_on, _config) => {
+  // `on` is used to hook into various events Cypress emits
+  // `config` is the resolved Cypress config
+};

x-pack/plugins/osquery/cypress/screens/integrations.ts

@@ -0,0 +1,10 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+export const ADD_POLICY_BTN = '[data-test-subj="addIntegrationPolicyButton"]';
+export const CREATE_PACKAGE_POLICY_SAVE_BTN = '[data-test-subj="createPackagePolicySaveButton"]';
+export const INTEGRATIONS_CARD = '.euiCard__titleAnchor';

x-pack/plugins/osquery/cypress/screens/navigation.ts

@@ -0,0 +1,9 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+export const TOGGLE_NAVIGATION_BTN = '[data-test-subj="toggleNavButton"]';
+export const OSQUERY_NAVIGATION_LINK = '[data-test-subj="collapsibleNavAppLink"] [title="Osquery"]';

x-pack/plugins/osquery/cypress/screens/osquery.ts

@@ -0,0 +1,8 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+export const HEADER = 'h1';

x-pack/plugins/osquery/cypress/support/commands.js

@@ -0,0 +1,32 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+// ***********************************************
+// This example commands.js shows you how to
+// create various custom commands and overwrite
+// existing commands.
+//
+// For more comprehensive examples of custom
+// commands please read more here:
+// https://on.cypress.io/custom-commands
+// ***********************************************
+//
+//
+// -- This is a parent command --
+// Cypress.Commands.add('login', (email, password) => { ... })
+//
+//
+// -- This is a child command --
+// Cypress.Commands.add('drag', { prevSubject: 'element'}, (subject, options) => { ... })
+//
+//
+// -- This is a dual command --
+// Cypress.Commands.add('dismiss', { prevSubject: 'optional'}, (subject, options) => { ... })
+//
+//
+// -- This will overwrite an existing command --
+// Cypress.Commands.overwrite('visit', (originalFn, url, options) => { ... })

x-pack/plugins/osquery/cypress/support/index.ts

@@ -0,0 +1,30 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+// ***********************************************************
+// This example support/index.js is processed and
+// loaded automatically before your test files.
+//
+// This is a great place to put global configuration and
+// behavior that modifies Cypress.
+//
+// You can change the location of this file or turn off
+// automatically serving support files with the
+// 'supportFile' configuration option.
+//
+// You can read more here:
+// https://on.cypress.io/configuration
+// ***********************************************************
+
+// Import commands.js using ES2015 syntax:
+import './commands';
+
+// Alternatively you can use CommonJS syntax:
+// require('./commands')
+Cypress.on('uncaught:exception', () => {
+  return false;
+});

x-pack/plugins/osquery/cypress/tasks/integrations.ts

@@ -0,0 +1,20 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import {
+  ADD_POLICY_BTN,
+  CREATE_PACKAGE_POLICY_SAVE_BTN,
+  INTEGRATIONS_CARD,
+} from '../screens/integrations';
+
+export const addIntegration = (integration: string) => {
+  cy.get(INTEGRATIONS_CARD).contains(integration).click();
+  cy.get(ADD_POLICY_BTN).click();
+  cy.get(CREATE_PACKAGE_POLICY_SAVE_BTN).click();
+  cy.get(CREATE_PACKAGE_POLICY_SAVE_BTN).should('not.exist');
+  cy.reload();
+};

x-pack/plugins/osquery/cypress/tasks/navigation.ts

@@ -0,0 +1,19 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { TOGGLE_NAVIGATION_BTN } from '../screens/navigation';
+
+export const INTEGRATIONS = 'app/integrations#/';
+export const OSQUERY = 'app/osquery/live_queries';
+
+export const navigateTo = (page: string) => {
+  cy.visit(page);
+};
+
+export const openNavigationFlyout = () => {
+  cy.get(TOGGLE_NAVIGATION_BTN).click();
+};

x-pack/plugins/osquery/cypress/tsconfig.json

@@ -0,0 +1,15 @@
+{
+    "extends": "../../../../tsconfig.base.json",
+    "exclude": [],
+    "include": [
+      "./**/*"
+    ],
+    "compilerOptions": {
+      "tsBuildInfoFile": "../../../../build/tsbuildinfo/osquery/cypress",
+      "types": [
+        "cypress",
+        "node"
+      ],
+      "resolveJsonModule": true,
+    },
+  }

x-pack/plugins/osquery/package.json

@@ -0,0 +1,13 @@
+{
+    "author": "Elastic",
+    "name": "osquery",
+    "version": "8.0.0",
+    "private": true,
+    "license": "Elastic-License",
+    "scripts": {
+      "cypress:open": "../../../node_modules/.bin/cypress open --config-file ./cypress/cypress.json",
+      "cypress:open-as-ci": "node ../../../scripts/functional_tests --config ../../test/osquery_cypress/visual_config.ts",
+      "cypress:run": "../../../node_modules/.bin/cypress run --config-file ./cypress/cypress.json",
+      "cypress:run-as-ci": "node ../../../scripts/functional_tests --config ../../test/osquery_cypress/cli_config.ts"
+    }
+}

x-pack/test/osquery_cypress/cli_config.ts

@@ -0,0 +1,19 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { FtrConfigProviderContext } from '@kbn/test';
+
+import { OsqueryCypressCliTestRunner } from './runner';
+
+export default async function ({ readConfigFile }: FtrConfigProviderContext) {
+  const osqueryCypressConfig = await readConfigFile(require.resolve('./config.ts'));
+  return {
+    ...osqueryCypressConfig.getAll(),
+
+    testRunner: OsqueryCypressCliTestRunner,
+  };
+}

x-pack/test/osquery_cypress/config.ts

@@ -0,0 +1,43 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { FtrConfigProviderContext } from '@kbn/test';
+
+import { CA_CERT_PATH } from '@kbn/dev-utils';
+
+export default async function ({ readConfigFile }: FtrConfigProviderContext) {
+  const kibanaCommonTestsConfig = await readConfigFile(
+    require.resolve('../../../test/common/config.js')
+  );
+  const xpackFunctionalTestsConfig = await readConfigFile(
+    require.resolve('../functional/config.js')
+  );
+
+  return {
+    ...kibanaCommonTestsConfig.getAll(),
+
+    esTestCluster: {
+      ...xpackFunctionalTestsConfig.get('esTestCluster'),
+      serverArgs: [
+        ...xpackFunctionalTestsConfig.get('esTestCluster.serverArgs'),
+        // define custom es server here
+        // API Keys is enabled at the top level
+        'xpack.security.enabled=true',
+      ],
+    },
+
+    kbnTestServer: {
+      ...xpackFunctionalTestsConfig.get('kbnTestServer'),
+      serverArgs: [
+        ...xpackFunctionalTestsConfig.get('kbnTestServer.serverArgs'),
+        '--csp.strict=false',
+        // define custom kibana server args here
+        `--elasticsearch.ssl.certificateAuthorities=${CA_CERT_PATH}`,
+      ],
+    },
+  };
+}

x-pack/test/osquery_cypress/ftr_provider_context.d.ts

@@ -0,0 +1,12 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { GenericFtrProviderContext } from '@kbn/test';
+
+import { services } from './services';
+
+export type FtrProviderContext = GenericFtrProviderContext<typeof services, {}>;

x-pack/test/osquery_cypress/runner.ts

@@ -0,0 +1,81 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { resolve } from 'path';
+import Url from 'url';
+
+import { withProcRunner } from '@kbn/dev-utils';
+
+import { FtrProviderContext } from './ftr_provider_context';
+
+export async function OsqueryCypressCliTestRunner({ getService }: FtrProviderContext) {
+  const log = getService('log');
+  const config = getService('config');
+
+  await withProcRunner(log, async (procs) => {
+    await procs.run('cypress', {
+      cmd: 'yarn',
+      args: ['cypress:run'],
+      cwd: resolve(__dirname, '../../plugins/osquery'),
+      env: {
+        FORCE_COLOR: '1',
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_baseUrl: Url.format(config.get('servers.kibana')),
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_protocol: config.get('servers.kibana.protocol'),
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_hostname: config.get('servers.kibana.hostname'),
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_configport: config.get('servers.kibana.port'),
+        CYPRESS_ELASTICSEARCH_URL: Url.format(config.get('servers.elasticsearch')),
+        CYPRESS_ELASTICSEARCH_USERNAME: config.get('servers.elasticsearch.username'),
+        CYPRESS_ELASTICSEARCH_PASSWORD: config.get('servers.elasticsearch.password'),
+        CYPRESS_KIBANA_URL: Url.format({
+          protocol: config.get('servers.kibana.protocol'),
+          hostname: config.get('servers.kibana.hostname'),
+          port: config.get('servers.kibana.port'),
+        }),
+        ...process.env,
+      },
+      wait: true,
+    });
+  });
+}
+
+export async function OsqueryCypressVisualTestRunner({ getService }: FtrProviderContext) {
+  const log = getService('log');
+  const config = getService('config');
+
+  await withProcRunner(log, async (procs) => {
+    await procs.run('cypress', {
+      cmd: 'yarn',
+      args: ['cypress:open'],
+      cwd: resolve(__dirname, '../../plugins/osquery'),
+      env: {
+        FORCE_COLOR: '1',
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_baseUrl: Url.format(config.get('servers.kibana')),
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_protocol: config.get('servers.kibana.protocol'),
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_hostname: config.get('servers.kibana.hostname'),
+        // eslint-disable-next-line @typescript-eslint/naming-convention
+        CYPRESS_configport: config.get('servers.kibana.port'),
+        CYPRESS_ELASTICSEARCH_URL: Url.format(config.get('servers.elasticsearch')),
+        CYPRESS_ELASTICSEARCH_USERNAME: config.get('servers.elasticsearch.username'),
+        CYPRESS_ELASTICSEARCH_PASSWORD: config.get('servers.elasticsearch.password'),
+        CYPRESS_KIBANA_URL: Url.format({
+          protocol: config.get('servers.kibana.protocol'),
+          hostname: config.get('servers.kibana.hostname'),
+          port: config.get('servers.kibana.port'),
+        }),
+        ...process.env,
+      },
+      wait: true,
+    });
+  });
+}

x-pack/test/osquery_cypress/services.ts

@@ -0,0 +1,8 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+export * from '../../../test/common/services';

x-pack/test/osquery_cypress/visual_config.ts

@@ -0,0 +1,19 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { FtrConfigProviderContext } from '@kbn/test';
+
+import { OsqueryCypressVisualTestRunner } from './runner';
+
+export default async function ({ readConfigFile }: FtrConfigProviderContext) {
+  const osqueryCypressConfig = await readConfigFile(require.resolve('./config.ts'));
+  return {
+    ...osqueryCypressConfig.getAll(),
+
+    testRunner: OsqueryCypressVisualTestRunner,
+  };
+}