mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 09:48:58 -04:00
# Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Test plan for prebuilt rule upgrade notifications (#212817)](https://github.com/elastic/kibana/pull/212817) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-03-10T11:50:12Z","message":"[Security Solution] Test plan for prebuilt rule upgrade notifications (#212817)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com/elastic/kibana/issues/202078,\nhttps://github.com/elastic/kibana/issues/210358\n\n## Summary\n\nWe started to rework and introduce functional changes to our existing\ntest plans for prebuilt rule customization, upgrade, and export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Creates a new test plan for prebuilt rule upgrade notifications on the\nRule Management, Rule Details, and Rule Editing pages. The filename is\n`prebuilt_rule_upgrade_notifications.md`.\n- Extracts the existing scenarios for upgrade notifications on the Rule\nManagement page from `prebuilt_rule_upgrade_without_preview.md` to\n`prebuilt_rule_upgrade_notifications.md`. Also, updates them according\nto the most recent UI behavior.\n- Adds new scenarios for upgrade notifications on the Rule Details page\nto `prebuilt_rule_upgrade_notifications.md`.\n- Adds new scenarios for upgrade notifications on the Rule Editing page\nto `prebuilt_rule_upgrade_notifications.md`.\n\nThe new test plan should be in line with the changes discussed in\nhttps://github.com/elastic/kibana/issues/210358.","sha":"da0480bde922b82741240a2aeeb1c33bd2497c66","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for prebuilt rule upgrade notifications","number":212817,"url":"https://github.com/elastic/kibana/pull/212817","mergeCommit":{"message":"[Security Solution] Test plan for prebuilt rule upgrade notifications (#212817)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com/elastic/kibana/issues/202078,\nhttps://github.com/elastic/kibana/issues/210358\n\n## Summary\n\nWe started to rework and introduce functional changes to our existing\ntest plans for prebuilt rule customization, upgrade, and export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Creates a new test plan for prebuilt rule upgrade notifications on the\nRule Management, Rule Details, and Rule Editing pages. The filename is\n`prebuilt_rule_upgrade_notifications.md`.\n- Extracts the existing scenarios for upgrade notifications on the Rule\nManagement page from `prebuilt_rule_upgrade_without_preview.md` to\n`prebuilt_rule_upgrade_notifications.md`. Also, updates them according\nto the most recent UI behavior.\n- Adds new scenarios for upgrade notifications on the Rule Details page\nto `prebuilt_rule_upgrade_notifications.md`.\n- Adds new scenarios for upgrade notifications on the Rule Editing page\nto `prebuilt_rule_upgrade_notifications.md`.\n\nThe new test plan should be in line with the changes discussed in\nhttps://github.com/elastic/kibana/issues/210358.","sha":"da0480bde922b82741240a2aeeb1c33bd2497c66"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/212817","number":212817,"mergeCommit":{"message":"[Security Solution] Test plan for prebuilt rule upgrade notifications (#212817)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com/elastic/kibana/issues/202078,\nhttps://github.com/elastic/kibana/issues/210358\n\n## Summary\n\nWe started to rework and introduce functional changes to our existing\ntest plans for prebuilt rule customization, upgrade, and export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Creates a new test plan for prebuilt rule upgrade notifications on the\nRule Management, Rule Details, and Rule Editing pages. The filename is\n`prebuilt_rule_upgrade_notifications.md`.\n- Extracts the existing scenarios for upgrade notifications on the Rule\nManagement page from `prebuilt_rule_upgrade_without_preview.md` to\n`prebuilt_rule_upgrade_notifications.md`. Also, updates them according\nto the most recent UI behavior.\n- Adds new scenarios for upgrade notifications on the Rule Details page\nto `prebuilt_rule_upgrade_notifications.md`.\n- Adds new scenarios for upgrade notifications on the Rule Editing page\nto `prebuilt_rule_upgrade_notifications.md`.\n\nThe new test plan should be in line with the changes discussed in\nhttps://github.com/elastic/kibana/issues/210358.","sha":"da0480bde922b82741240a2aeeb1c33bd2497c66"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
This commit is contained in:
Kibana Machine
GitHub
parent
06931baa86
commit
0f84615991
3 changed files with 334 additions and 60 deletions
|
|
@ -0,0 +1,333 @@
|
|||
# Test plan: prebuilt rule upgrade notifications <!-- omit from toc -->
|
||||
|
||||
**Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168).
|
||||
|
||||
> [!TIP]
|
||||
> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features).
|
||||
|
||||
## Summary <!-- omit from toc -->
|
||||
|
||||
This is a test plan for the functionality of showing notifications about:
|
||||
|
||||
- Rule Management page: some of the currently installed prebuilt rules can be upgraded to new versions.
|
||||
- Rule Details page: the rule shown on this page can be upgraded to a new version.
|
||||
- Rule Editing page: the rule shown on this page can be upgraded to a new version.
|
||||
|
||||
## Table of contents <!-- omit from toc -->
|
||||
|
||||
<!--
|
||||
Please use the "Markdown All in One" VS Code extension to keep the TOC in sync with the text:
|
||||
https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
|
||||
-->
|
||||
|
||||
- [Useful information](#useful-information)
|
||||
- [Tickets](#tickets)
|
||||
- [Terminology](#terminology)
|
||||
- [Requirements](#requirements)
|
||||
- [Assumptions](#assumptions)
|
||||
- [Technical requirements](#technical-requirements)
|
||||
- [Product requirements](#product-requirements)
|
||||
- [Scenarios](#scenarios)
|
||||
- [Rule upgrade notifications on the Rule Management page](#rule-upgrade-notifications-on-the-rule-management-page)
|
||||
- [**Scenario: User is NOT notified on the Rule Management page when no prebuilt rules are installed**](#scenario-user-is-not-notified-on-the-rule-management-page-when-no-prebuilt-rules-are-installed)
|
||||
- [**Scenario: User is NOT notified on the Rule Management page when all installed prebuilt rules are up to date**](#scenario-user-is-not-notified-on-the-rule-management-page-when-all-installed-prebuilt-rules-are-up-to-date)
|
||||
- [**Scenario: User is NOT notified on the Rule Management page until the package installation is completed**](#scenario-user-is-not-notified-on-the-rule-management-page-until-the-package-installation-is-completed)
|
||||
- [**Scenario: User is notified on the Rule Management page when there are some prebuilt rules to upgrade but there are no more prebuilt rules to install**](#scenario-user-is-notified-on-the-rule-management-page-when-there-are-some-prebuilt-rules-to-upgrade-but-there-are-no-more-prebuilt-rules-to-install)
|
||||
- [**Scenario: User is notified on the Rule Management page when there are some prebuilt rules to upgrade and some more prebuilt rules to install**](#scenario-user-is-notified-on-the-rule-management-page-when-there-are-some-prebuilt-rules-to-upgrade-and-some-more-prebuilt-rules-to-install)
|
||||
- [**Scenario: User can open the Rule Upgrade table on the Rule Management page**](#scenario-user-can-open-the-rule-upgrade-table-on-the-rule-management-page)
|
||||
- [**Scenario: User can dismiss the prebuilt rule upgrade callout on the Rule Management page**](#scenario-user-can-dismiss-the-prebuilt-rule-upgrade-callout-on-the-rule-management-page)
|
||||
- [Rule upgrade notifications on the Rule Details page](#rule-upgrade-notifications-on-the-rule-details-page)
|
||||
- [**Scenario: User is NOT notified on the Rule Details page when the rule is up to date**](#scenario-user-is-not-notified-on-the-rule-details-page-when-the-rule-is-up-to-date)
|
||||
- [**Scenario: User is notified on the Rule Details page when the rule is outdated and can be upgraded to a new version**](#scenario-user-is-notified-on-the-rule-details-page-when-the-rule-is-outdated-and-can-be-upgraded-to-a-new-version)
|
||||
- [**Scenario: User can open the Rule Upgrade flyout on the Rule Details page**](#scenario-user-can-open-the-rule-upgrade-flyout-on-the-rule-details-page)
|
||||
- [**Scenario: User cannot dismiss the prebuilt rule upgrade callout on the Rule Details page**](#scenario-user-cannot-dismiss-the-prebuilt-rule-upgrade-callout-on-the-rule-details-page)
|
||||
- [Rule upgrade notifications on the Rule Editing page](#rule-upgrade-notifications-on-the-rule-editing-page)
|
||||
- [**Scenario: User is NOT notified on the Rule Editing page when the rule is up to date**](#scenario-user-is-not-notified-on-the-rule-editing-page-when-the-rule-is-up-to-date)
|
||||
- [**Scenario: User is notified on the Rule Editing page when the rule is outdated and can be upgraded to a new version**](#scenario-user-is-notified-on-the-rule-editing-page-when-the-rule-is-outdated-and-can-be-upgraded-to-a-new-version)
|
||||
- [**Scenario: User can navigate from the Rule Editing page to the Rule Details page in order to upgrade the rule**](#scenario-user-can-navigate-from-the-rule-editing-page-to-the-rule-details-page-in-order-to-upgrade-the-rule)
|
||||
- [**Scenario: User cannot dismiss the prebuilt rule upgrade callout on the Rule Editing page**](#scenario-user-cannot-dismiss-the-prebuilt-rule-upgrade-callout-on-the-rule-editing-page)
|
||||
|
||||
## Useful information
|
||||
|
||||
### Tickets
|
||||
|
||||
- [Users can Customize Prebuilt Detection Rules](https://github.com/elastic/security-team/issues/1974) (internal)
|
||||
- [Users can Customize Prebuilt Detection Rules: Milestone 3](https://github.com/elastic/kibana/issues/174168)
|
||||
- [Tests for prebuilt rule upgrade workflow](https://github.com/elastic/kibana/issues/202078)
|
||||
|
||||
### Terminology
|
||||
|
||||
- [Common terminology](./prebuilt_rules_common_info.md#common-terminology).
|
||||
- **Callout to upgrade prebuilt rules**: a notification callout shown on the Rule Management page that encourages the user to keep prebuilt rules up-to-date and regularly upgrade them to their latest versions.
|
||||
- **Callout to upgrade the rule**: is either of the two callouts, depending on the context of a given scenario:
|
||||
- a notification callout shown on the Rule Details page that encourages the user to upgrade the rule to its latest version;
|
||||
- a notification callout shown on the Rule Editing page that encourages the user to upgrade the rule to its latest version before editing it.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Assumptions
|
||||
|
||||
Assumptions about test environments and scenarios outlined in this test plan.
|
||||
|
||||
- [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions).
|
||||
|
||||
### Technical requirements
|
||||
|
||||
Non-functional requirements for the functionality outlined in this test plan.
|
||||
|
||||
- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements).
|
||||
|
||||
### Product requirements
|
||||
|
||||
Functional requirements for the functionality outlined in this test plan.
|
||||
|
||||
- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements).
|
||||
|
||||
User stories for the Rule Management page:
|
||||
|
||||
- User can see a notification callout on the Rule Management page when some of the currently installed prebuilt rules can be upgraded to new versions. The callout encourages the user to keep prebuilt rules up-to-date and regularly upgrade them to their latest versions.
|
||||
- User can dismiss the callout.
|
||||
- User can see a Rule Updates tab on the Rule Management page when some of the currently installed prebuilt rules can be upgraded to new versions.
|
||||
- User can see the total number of rules available for upgrade on this tab.
|
||||
- User can click on the tab which opens the Rule Upgrade table.
|
||||
|
||||
User stories for the Rule Details page:
|
||||
|
||||
- User can see a notification callout on the Rule Details page when the rule shown on this page can be upgraded to a new version. The callout encourages the user to upgrade the rule.
|
||||
- User can open a Rule Upgrade flyout to review updates in the latest rule version and perform the upgrade.
|
||||
- User can't dismiss the callout.
|
||||
|
||||
User stories for the Rule Editing page:
|
||||
|
||||
- User can see a notification callout on the Rule Editing page when the rule shown on this page can be upgraded to a new version. The callout encourages the user to upgrade the rule before editing it.
|
||||
- User can navigate back to the Rule Details page where they can review updates in the latest rule version and perform the upgrade.
|
||||
- User can't dismiss the callout.
|
||||
|
||||
## Scenarios
|
||||
|
||||
### Rule upgrade notifications on the Rule Management page
|
||||
|
||||
#### **Scenario: User is NOT notified on the Rule Management page when no prebuilt rules are installed**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given no prebuilt rules are installed in Kibana
|
||||
When user opens the Rule Management page
|
||||
Then user should NOT see a callout to upgrade prebuilt rules
|
||||
And user should NOT see a Rule Updates tab
|
||||
And user should NOT see a number of rules available to upgrade
|
||||
```
|
||||
|
||||
#### **Scenario: User is NOT notified on the Rule Management page when all installed prebuilt rules are up to date**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given <X> prebuilt rules are installed in Kibana
|
||||
And all of them are up to date (no new versions are available)
|
||||
When user opens the Rule Management page
|
||||
Then user should NOT see a callout to upgrade prebuilt rules
|
||||
And user should NOT see a Rule Updates tab
|
||||
And user should NOT see a number of rules available to upgrade
|
||||
|
||||
Examples:
|
||||
| X |
|
||||
| 1 |
|
||||
| 100 |
|
||||
| 1250 |
|
||||
```
|
||||
|
||||
#### **Scenario: User is NOT notified on the Rule Management page until the package installation is completed**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + unit tests.
|
||||
|
||||
```Gherkin
|
||||
Given <X> prebuilt rules are installed in Kibana
|
||||
And for <Z> of the installed rules there are new versions available
|
||||
And prebuilt rules package is not installed
|
||||
When user opens the Rule Management page
|
||||
Then user should NOT see a callout to upgrade prebuilt rules
|
||||
And user should NOT see a Rule Updates tab
|
||||
And user should NOT see a number of rules available to upgrade
|
||||
When user waits until the the package installation is completed
|
||||
Then user should see the callout to upgrade prebuilt rules
|
||||
And user should see the Rule Updates tab
|
||||
And user should see the number of rules available to upgrade (<Z>)
|
||||
|
||||
Examples:
|
||||
| X | Z |
|
||||
| 1250 | 1 |
|
||||
| 1250 | 100 |
|
||||
| 1250 | 1250 |
|
||||
```
|
||||
|
||||
#### **Scenario: User is notified on the Rule Management page when there are some prebuilt rules to upgrade but there are no more prebuilt rules to install**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given <X> prebuilt rules are installed in Kibana
|
||||
And there are no more prebuilt rules available to install
|
||||
And for <Z> of the installed rules there are new versions available
|
||||
When user opens the Rule Management page
|
||||
Then user should see the callout to upgrade prebuilt rules
|
||||
And user should see the Rule Updates tab
|
||||
And user should see the number of rules available to upgrade (<Z>)
|
||||
|
||||
Examples:
|
||||
| X | Z |
|
||||
| 1250 | 1 |
|
||||
| 1250 | 100 |
|
||||
| 1250 | 1250 |
|
||||
```
|
||||
|
||||
#### **Scenario: User is notified on the Rule Management page when there are some prebuilt rules to upgrade and some more prebuilt rules to install**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given <X> prebuilt rules are installed in Kibana
|
||||
And there are <Y> more prebuilt rules available to install
|
||||
And for <Z> of the installed rules there are new versions available
|
||||
When user opens the Rule Management page
|
||||
Then user should see the callout to upgrade prebuilt rules
|
||||
And user should see the Rule Updates tab
|
||||
And user should see the number of rules available to upgrade (<Z>)
|
||||
|
||||
Examples:
|
||||
| X | Y | Z |
|
||||
| 1 | 1249 | 1 |
|
||||
| 1249 | 1 | 100 |
|
||||
| 1249 | 1 | 1249 |
|
||||
```
|
||||
|
||||
#### **Scenario: User can open the Rule Upgrade table on the Rule Management page**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given some prebuilt rules are installed in Kibana
|
||||
And for some of the installed rules there are new versions available
|
||||
When user opens the Rule Management page
|
||||
Then user should see the Rule Updates tab
|
||||
When user clicks on the Rule Updates tab
|
||||
Then the Rule Upgrade table should be displayed
|
||||
```
|
||||
|
||||
#### **Scenario: User can dismiss the prebuilt rule upgrade callout on the Rule Management page**
|
||||
|
||||
Currently, we store the callout's state only in the browser memory. This means that a dismissed callout will show up again after the page refresh.
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + unit tests for the callout component.
|
||||
|
||||
```Gherkin
|
||||
Given some prebuilt rules are installed in Kibana
|
||||
And for some of the installed rules there are new versions available
|
||||
When user opens the Rule Management page
|
||||
Then user should see the callout to upgrade prebuilt rules
|
||||
When user clicks on the Dismiss button
|
||||
Then the callout should disappear
|
||||
When user refreshes the page
|
||||
Then the callout should appear again
|
||||
```
|
||||
|
||||
### Rule upgrade notifications on the Rule Details page
|
||||
|
||||
#### **Scenario: User is NOT notified on the Rule Details page when the rule is up to date**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is up to date (no new versions are available for this rule)
|
||||
When user opens the Rule Details page
|
||||
Then user should NOT see the callout to upgrade the rule
|
||||
```
|
||||
|
||||
#### **Scenario: User is notified on the Rule Details page when the rule is outdated and can be upgraded to a new version**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is outdated (a new version is available for this rule)
|
||||
When user opens the Rule Details page
|
||||
Then user should see the callout to upgrade the rule
|
||||
```
|
||||
|
||||
#### **Scenario: User can open the Rule Upgrade flyout on the Rule Details page**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is outdated (a new version is available for this rule)
|
||||
When user opens the Rule Details page
|
||||
Then user should see the callout to upgrade the rule
|
||||
When user clicks on the callout's CTA button
|
||||
Then the Rule Upgrade flyout should be displayed
|
||||
```
|
||||
|
||||
#### **Scenario: User cannot dismiss the prebuilt rule upgrade callout on the Rule Details page**
|
||||
|
||||
**Automation**: unit tests for the callout component.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is outdated (a new version is available for this rule)
|
||||
When user opens the Rule Details page
|
||||
Then user should see the callout to upgrade the rule
|
||||
And user should NOT see any Dismiss buttons in it
|
||||
And user should NOT be able to dismiss it
|
||||
```
|
||||
|
||||
### Rule upgrade notifications on the Rule Editing page
|
||||
|
||||
#### **Scenario: User is NOT notified on the Rule Editing page when the rule is up to date**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is up to date (no new versions are available for this rule)
|
||||
When user opens the Rule Editing page
|
||||
Then user should NOT see the callout to upgrade the rule
|
||||
```
|
||||
|
||||
#### **Scenario: User is notified on the Rule Editing page when the rule is outdated and can be upgraded to a new version**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is outdated (a new version is available for this rule)
|
||||
When user opens the Rule Editing page
|
||||
Then user should see the callout to upgrade the rule
|
||||
```
|
||||
|
||||
#### **Scenario: User can navigate from the Rule Editing page to the Rule Details page in order to upgrade the rule**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is outdated (a new version is available for this rule)
|
||||
When user opens the Rule Editing page
|
||||
Then user should see the callout to upgrade the rule
|
||||
When user clicks on the callout's CTA button
|
||||
Then user should be navigated to the Rule Details page
|
||||
```
|
||||
|
||||
#### **Scenario: User cannot dismiss the prebuilt rule upgrade callout on the Rule Editing page**
|
||||
|
||||
**Automation**: unit tests for the callout component.
|
||||
|
||||
```Gherkin
|
||||
Given a prebuilt rule is installed in Kibana
|
||||
And the rule is outdated (a new version is available for this rule)
|
||||
When user opens the Rule Editing page
|
||||
Then user should see the callout to upgrade the rule
|
||||
And user should NOT see any Dismiss buttons in it
|
||||
And user should NOT be able to dismiss it
|
||||
```
|
||||
|
|
@ -29,10 +29,6 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
|
|||
- [Technical requirements](#technical-requirements)
|
||||
- [Product requirements](#product-requirements)
|
||||
- [Scenarios](#scenarios)
|
||||
- [Rule upgrade notifications on the Rule Management page](#rule-upgrade-notifications-on-the-rule-management-page)
|
||||
- [**Scenario: User is NOT notified when all installed prebuilt rules are up to date**](#scenario-user-is-not-notified-when-all-installed-prebuilt-rules-are-up-to-date)
|
||||
- [**Scenario: User is notified when some prebuilt rules can be upgraded**](#scenario-user-is-notified-when-some-prebuilt-rules-can-be-upgraded)
|
||||
- [**Scenario: User is notified when both rules to install and upgrade are available**](#scenario-user-is-notified-when-both-rules-to-install-and-upgrade-are-available)
|
||||
- [Rule upgrade workflow: individual updates from Rule Updates table](#rule-upgrade-workflow-individual-updates-from-rule-updates-table)
|
||||
- [**Scenario: User can upgrade conflict-free prebuilt rules one by one**](#scenario-user-can-upgrade-conflict-free-prebuilt-rules-one-by-one)
|
||||
- [**Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts**](#scenario-user-cannot-upgrade-prebuilt-rules-one-by-one-from-rules-update-table-if-they-have-conflicts)
|
||||
|
|
@ -144,50 +140,6 @@ User stories, misc:
|
|||
|
||||
## Scenarios
|
||||
|
||||
### Rule upgrade notifications on the Rule Management page
|
||||
|
||||
#### **Scenario: User is NOT notified when all installed prebuilt rules are up to date**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given all the latest prebuilt rules are installed in Kibana
|
||||
When user opens the Rule Management page
|
||||
And user should NOT see a CTA to upgrade prebuilt rules
|
||||
And user should NOT see a number of rules available to upgrade
|
||||
And user should NOT see the Rule Updates table
|
||||
```
|
||||
|
||||
#### **Scenario: User is notified when some prebuilt rules can be upgraded**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given X prebuilt rules are installed in Kibana
|
||||
And there are no more prebuilt rules available to install
|
||||
And for Z of the installed rules there are new versions available
|
||||
When user opens the Rule Management page
|
||||
Then user should NOT see a CTA to install prebuilt rules
|
||||
And user should NOT see a number of rules available to install
|
||||
And user should see a CTA to upgrade prebuilt rules
|
||||
And user should see the number of rules available to upgrade (Z)
|
||||
```
|
||||
|
||||
#### **Scenario: User is notified when both rules to install and upgrade are available**
|
||||
|
||||
**Automation**: 1 e2e test with mock rules + 1 integration test with mock rules for the /status endpoint.
|
||||
|
||||
```Gherkin
|
||||
Given X prebuilt rules are installed in Kibana
|
||||
And there are Y more prebuilt rules available to install
|
||||
And for Z of the installed rules there are new versions available
|
||||
When user opens the Rule Management page
|
||||
Then user should see a CTA to install prebuilt rules
|
||||
And user should see the number of rules available to install (Y)
|
||||
And user should see a CTA to upgrade prebuilt rules
|
||||
And user should see the number of rules available to upgrade (Z)
|
||||
```
|
||||
|
||||
### Rule upgrade workflow: individual updates from Rule Updates table
|
||||
|
||||
#### **Scenario: User can upgrade conflict-free prebuilt rules one by one**
|
||||
|
|
@ -735,18 +687,6 @@ Then the rule bound data should be preserved
|
|||
|
||||
Examples: generated alerts, exception lists (rule exception list, shared exception list, endpoint exception list), timeline reference, actions, enabled state, execution results and execution events.
|
||||
|
||||
### Rule upgrade workflow: misc cases
|
||||
|
||||
#### **Scenario: User doesn't see the Rule Updates tab until the package installation is completed**
|
||||
|
||||
**Automation**: unit tests.
|
||||
|
||||
```Gherkin
|
||||
Given prebuilt rules package is not installed
|
||||
When user opens the Rule Management page
|
||||
Then user should NOT see the Rule Updates tab until the package installation is completed and there are rules available for upgrade
|
||||
```
|
||||
|
||||
### Error handling
|
||||
|
||||
#### **Scenario: Error is handled when any upgrade operation on prebuilt rules fails**
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ Learn how users can edit and customize prebuilt rules:
|
|||
|
||||
Learn how users can upgrade prebuilt rules:
|
||||
|
||||
- [Test plan: prebuilt rule upgrade notifications](./prebuilt_rule_upgrade_notifications.md)
|
||||
- [Test plan: upgrading prebuilt rules one-by-one or in bulk without preview](./prebuilt_rule_upgrade_without_preview.md)
|
||||
- [Test plan: upgrading prebuilt rules one-by-one with preview](./prebuilt_rule_upgrade_with_preview.md)
|
||||
- [Test plan: diff algorithms for upgrading prebuilt rules](./prebuilt_rule_upgrade_diff_algorithms.md)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue