mirror of
https://github.com/elastic/kibana.git
synced 2025-06-27 18:51:07 -04:00
d5fd19058f
# Backport This will backport the following commits from `main` to `9.0`: - [[Security solution][AI assistant] bug: encode security labs content to prevent antivirus false positive (#221656)](https://github.com/elastic/kibana/pull/221656) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Kenneth Kreindler","email":"42113355+KDKHD@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-06-06T09:16:48Z","message":"[Security solution][AI assistant] bug: encode security labs content to prevent antivirus false positive (#221656)\n\n## Summary\n\nFixes: https://github.com/elastic/kibana/issues/202114\n\nSummarize your PR. If it involves visual changes include a screenshot or\ngif.\n\nThis PR addresses the following\n[issue](https://github.com/elastic/kibana/issues/202114).\n\n#### Problem \nThe security labs' content is triggering false positive anti-virus\nalerts from [ESET cyber\nsecurity](https://www.eset.com/uk/home/cyber-security/?srsltid=AfmBOorLSTn6FfQXm9h4rm2nDpma91Q9-IfHmeUg4TuRL4TvMF9xB-Cc).\nThis is because the content contains specific words and YARA rules that\nthe antivirus picks up as malware. The content is not dangerous.\n\n#### Solution \nEncrypt the content so that the Yara rules do not trigger alerts. The\ncontent is encrypted with AES-256-ECB and the key `ELASTIC`. The\nencryption is not secure and does not need to be secure (we just want to\nobfuscate the content).\n\n#### Considerations\n- An alternative approach to fixing this issue that was considered was\nbuilding an integration so that the content is only imported after\nKibana has been started. As a team, we decided against this because it\nis convenient for airgapped systems to have the secure labs content\nshipped with the installation.\n- It would be great to test this fix against many antivirus providers,\nhowever, I haven't found a tool that lets me do this quickly.\n\n## How to test\n#### Verify the security labs content is encoded\n- Check out the PR\n- Run the following\n```bash\nyarn kbn bootstrap\nyarn build\n```\n- The build may not fully complete locally; however, you should still be\nable to see the build files at `/build`. Inspect the content of the\nfollowing folder:\n`build/kibana/node_modules/@kbn/elastic-assistant-plugin/server/knowledge_base/security_labs`\nand verify that only files ending with `.encoded.md` exist. The content\nof these files should not be human-readable.\n\n\n\n\n#### Verify the security labs content is encoded in the cloud/serverless\nbuild (optional)\n- Log into docker.elastic.co in your terminal. Do this by heading to\nhttps://docker-auth.elastic.co/ (more info\n[here](https://elasticprod.service-now.com/esc?id=kb_article&table=kb_knowledge&sys_id=e8d361c747abb910ffad4438946d439a&recordUrl=kb_view.do%3Fsysparm_article%3DKB0012946))\n- Use the latest CI run to get the serverless docker image and start an\ninteractive shell with it:\n```bash\ndocker run -it --rm docker.elastic.co/kibana-ci/kibana-serverless:pr-221656-ae41a481bbfc sh\n```\n- Inspect the contents of the following directory and verify that only\n`.encoded.md` files exist.\n```bash\ncd node_modules/@kbn/elastic-assistant-plugin/server/knowledge_base/security_labs/\nls\n```\n- Repeat the last 2 steps for the cloud deployment image (also found in\nthe CI)\n\n<img width=\"1233\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5d308537-b0a9-4bd0-a449-6ddae1ca1bfb\"\n/>\n\n#### Verify security labs content can be installed\n- Start Kibana locally\n- Head over to\n`http://localhost:5601/app/management/kibana/securityAiAssistantManagement?tab=knowledge_base`\nand install the knowledge base.\n\n<img width=\"1454\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e8c4b557-ea45-4c58-96ff-aacce47e9982\"\n/>\n\n- Check that the security labs content is being installed\n\n<img width=\"1456\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e7c2b128-ee94-436e-b4e0-1d48cb9d19cb\"\n/>\n\n- Once the security labs content is installed, go to the security AI\nassistant and ask the following: `Which malware is mentioned in the\nsecurity labs content. Include citations`\n- Verify the assistant gives a proper response and the citation links to\nthe Elastic Security Labs page.\n\n<img width=\"859\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e89e523c-3054-4b8b-a3c3-2857cb1ed8cd\"\n/>\n\n#### Verify antivirus does not trigger (optional)\n- Download and install ESET Cyber security trial from\n[here](https://www.eset.com/uk/home/cyber-security/?srsltid=AfmBOorLSTn6FfQXm9h4rm2nDpma91Q9-IfHmeUg4TuRL4TvMF9xB-Cc)\n- Open up the ESET\n- Click on `custom scan` and run the scan on the folder `build/kibana`\n\n\n\n\n- Expect no alerts to be triggered\n\n\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [X] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [X]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [X] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [X] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [X] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [X] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [X] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\nCo-authored-by: Garrett Spong <garrett.spong@elastic.co>\nCo-authored-by: Garrett Spong <spong@users.noreply.github.com>","sha":"973c8f30a69fa785a149cc50747d99c1db4ce7b6","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","ci:cloud-deploy","ci:project-deploy-security","Team:Security Generative AI","backport:version","v9.1.0","v9.0.3","v8.18.3","v8.17.8"],"title":"[Security solution][AI assistant] bug: encode security labs content to prevent antivirus false positive","number":221656,"url":"https://github.com/elastic/kibana/pull/221656","mergeCommit":{"message":"[Security solution][AI assistant] bug: encode security labs content to prevent antivirus false positive (#221656)\n\n## Summary\n\nFixes: https://github.com/elastic/kibana/issues/202114\n\nSummarize your PR. If it involves visual changes include a screenshot or\ngif.\n\nThis PR addresses the following\n[issue](https://github.com/elastic/kibana/issues/202114).\n\n#### Problem \nThe security labs' content is triggering false positive anti-virus\nalerts from [ESET cyber\nsecurity](https://www.eset.com/uk/home/cyber-security/?srsltid=AfmBOorLSTn6FfQXm9h4rm2nDpma91Q9-IfHmeUg4TuRL4TvMF9xB-Cc).\nThis is because the content contains specific words and YARA rules that\nthe antivirus picks up as malware. The content is not dangerous.\n\n#### Solution \nEncrypt the content so that the Yara rules do not trigger alerts. The\ncontent is encrypted with AES-256-ECB and the key `ELASTIC`. The\nencryption is not secure and does not need to be secure (we just want to\nobfuscate the content).\n\n#### Considerations\n- An alternative approach to fixing this issue that was considered was\nbuilding an integration so that the content is only imported after\nKibana has been started. As a team, we decided against this because it\nis convenient for airgapped systems to have the secure labs content\nshipped with the installation.\n- It would be great to test this fix against many antivirus providers,\nhowever, I haven't found a tool that lets me do this quickly.\n\n## How to test\n#### Verify the security labs content is encoded\n- Check out the PR\n- Run the following\n```bash\nyarn kbn bootstrap\nyarn build\n```\n- The build may not fully complete locally; however, you should still be\nable to see the build files at `/build`. Inspect the content of the\nfollowing folder:\n`build/kibana/node_modules/@kbn/elastic-assistant-plugin/server/knowledge_base/security_labs`\nand verify that only files ending with `.encoded.md` exist. The content\nof these files should not be human-readable.\n\n\n\n\n#### Verify the security labs content is encoded in the cloud/serverless\nbuild (optional)\n- Log into docker.elastic.co in your terminal. Do this by heading to\nhttps://docker-auth.elastic.co/ (more info\n[here](https://elasticprod.service-now.com/esc?id=kb_article&table=kb_knowledge&sys_id=e8d361c747abb910ffad4438946d439a&recordUrl=kb_view.do%3Fsysparm_article%3DKB0012946))\n- Use the latest CI run to get the serverless docker image and start an\ninteractive shell with it:\n```bash\ndocker run -it --rm docker.elastic.co/kibana-ci/kibana-serverless:pr-221656-ae41a481bbfc sh\n```\n- Inspect the contents of the following directory and verify that only\n`.encoded.md` files exist.\n```bash\ncd node_modules/@kbn/elastic-assistant-plugin/server/knowledge_base/security_labs/\nls\n```\n- Repeat the last 2 steps for the cloud deployment image (also found in\nthe CI)\n\n<img width=\"1233\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5d308537-b0a9-4bd0-a449-6ddae1ca1bfb\"\n/>\n\n#### Verify security labs content can be installed\n- Start Kibana locally\n- Head over to\n`http://localhost:5601/app/management/kibana/securityAiAssistantManagement?tab=knowledge_base`\nand install the knowledge base.\n\n<img width=\"1454\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e8c4b557-ea45-4c58-96ff-aacce47e9982\"\n/>\n\n- Check that the security labs content is being installed\n\n<img width=\"1456\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e7c2b128-ee94-436e-b4e0-1d48cb9d19cb\"\n/>\n\n- Once the security labs content is installed, go to the security AI\nassistant and ask the following: `Which malware is mentioned in the\nsecurity labs content. Include citations`\n- Verify the assistant gives a proper response and the citation links to\nthe Elastic Security Labs page.\n\n<img width=\"859\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e89e523c-3054-4b8b-a3c3-2857cb1ed8cd\"\n/>\n\n#### Verify antivirus does not trigger (optional)\n- Download and install ESET Cyber security trial from\n[here](https://www.eset.com/uk/home/cyber-security/?srsltid=AfmBOorLSTn6FfQXm9h4rm2nDpma91Q9-IfHmeUg4TuRL4TvMF9xB-Cc)\n- Open up the ESET\n- Click on `custom scan` and run the scan on the folder `build/kibana`\n\n\n\n\n- Expect no alerts to be triggered\n\n\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [X] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [X]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [X] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [X] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [X] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [X] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [X] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\nCo-authored-by: Garrett Spong <garrett.spong@elastic.co>\nCo-authored-by: Garrett Spong <spong@users.noreply.github.com>","sha":"973c8f30a69fa785a149cc50747d99c1db4ce7b6"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/221656","number":221656,"mergeCommit":{"message":"[Security solution][AI assistant] bug: encode security labs content to prevent antivirus false positive (#221656)\n\n## Summary\n\nFixes: https://github.com/elastic/kibana/issues/202114\n\nSummarize your PR. If it involves visual changes include a screenshot or\ngif.\n\nThis PR addresses the following\n[issue](https://github.com/elastic/kibana/issues/202114).\n\n#### Problem \nThe security labs' content is triggering false positive anti-virus\nalerts from [ESET cyber\nsecurity](https://www.eset.com/uk/home/cyber-security/?srsltid=AfmBOorLSTn6FfQXm9h4rm2nDpma91Q9-IfHmeUg4TuRL4TvMF9xB-Cc).\nThis is because the content contains specific words and YARA rules that\nthe antivirus picks up as malware. The content is not dangerous.\n\n#### Solution \nEncrypt the content so that the Yara rules do not trigger alerts. The\ncontent is encrypted with AES-256-ECB and the key `ELASTIC`. The\nencryption is not secure and does not need to be secure (we just want to\nobfuscate the content).\n\n#### Considerations\n- An alternative approach to fixing this issue that was considered was\nbuilding an integration so that the content is only imported after\nKibana has been started. As a team, we decided against this because it\nis convenient for airgapped systems to have the secure labs content\nshipped with the installation.\n- It would be great to test this fix against many antivirus providers,\nhowever, I haven't found a tool that lets me do this quickly.\n\n## How to test\n#### Verify the security labs content is encoded\n- Check out the PR\n- Run the following\n```bash\nyarn kbn bootstrap\nyarn build\n```\n- The build may not fully complete locally; however, you should still be\nable to see the build files at `/build`. Inspect the content of the\nfollowing folder:\n`build/kibana/node_modules/@kbn/elastic-assistant-plugin/server/knowledge_base/security_labs`\nand verify that only files ending with `.encoded.md` exist. The content\nof these files should not be human-readable.\n\n\n\n\n#### Verify the security labs content is encoded in the cloud/serverless\nbuild (optional)\n- Log into docker.elastic.co in your terminal. Do this by heading to\nhttps://docker-auth.elastic.co/ (more info\n[here](https://elasticprod.service-now.com/esc?id=kb_article&table=kb_knowledge&sys_id=e8d361c747abb910ffad4438946d439a&recordUrl=kb_view.do%3Fsysparm_article%3DKB0012946))\n- Use the latest CI run to get the serverless docker image and start an\ninteractive shell with it:\n```bash\ndocker run -it --rm docker.elastic.co/kibana-ci/kibana-serverless:pr-221656-ae41a481bbfc sh\n```\n- Inspect the contents of the following directory and verify that only\n`.encoded.md` files exist.\n```bash\ncd node_modules/@kbn/elastic-assistant-plugin/server/knowledge_base/security_labs/\nls\n```\n- Repeat the last 2 steps for the cloud deployment image (also found in\nthe CI)\n\n<img width=\"1233\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/5d308537-b0a9-4bd0-a449-6ddae1ca1bfb\"\n/>\n\n#### Verify security labs content can be installed\n- Start Kibana locally\n- Head over to\n`http://localhost:5601/app/management/kibana/securityAiAssistantManagement?tab=knowledge_base`\nand install the knowledge base.\n\n<img width=\"1454\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e8c4b557-ea45-4c58-96ff-aacce47e9982\"\n/>\n\n- Check that the security labs content is being installed\n\n<img width=\"1456\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e7c2b128-ee94-436e-b4e0-1d48cb9d19cb\"\n/>\n\n- Once the security labs content is installed, go to the security AI\nassistant and ask the following: `Which malware is mentioned in the\nsecurity labs content. Include citations`\n- Verify the assistant gives a proper response and the citation links to\nthe Elastic Security Labs page.\n\n<img width=\"859\" alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/e89e523c-3054-4b8b-a3c3-2857cb1ed8cd\"\n/>\n\n#### Verify antivirus does not trigger (optional)\n- Download and install ESET Cyber security trial from\n[here](https://www.eset.com/uk/home/cyber-security/?srsltid=AfmBOorLSTn6FfQXm9h4rm2nDpma91Q9-IfHmeUg4TuRL4TvMF9xB-Cc)\n- Open up the ESET\n- Click on `custom scan` and run the scan on the folder `build/kibana`\n\n\n\n\n- Expect no alerts to be triggered\n\n\n\n\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [X] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [X]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [X] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [X] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [X] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [X] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [X] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\nCo-authored-by: Garrett Spong <garrett.spong@elastic.co>\nCo-authored-by: Garrett Spong <spong@users.noreply.github.com>","sha":"973c8f30a69fa785a149cc50747d99c1db4ce7b6"}},{"branch":"9.0","label":"v9.0.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.17","label":"v8.17.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Garrett Spong <garrett.spong@elastic.co> Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
18 lines
661 B
TypeScript
18 lines
661 B
TypeScript
/*
|
|
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
|
* or more contributor license agreements. Licensed under the Elastic License
|
|
* 2.0; you may not use this file except in compliance with the Elastic License
|
|
* 2.0.
|
|
*/
|
|
|
|
export { encryptSecurityLabsContent, decryptSecurityLabsContent } from './src/utils';
|
|
|
|
/**
|
|
* Micromatch pattern for plain text markdown files in the security labs content.
|
|
*/
|
|
export const PLAIN_TEXT_FILE_MICROMATCH_PATTERN = ['*.md', '!*.encoded.md'];
|
|
|
|
/**
|
|
* Micromatch pattern for encoded markdown files in the security labs content.
|
|
*/
|
|
export const ENCODED_FILE_MICROMATCH_PATTERN = ['*.encoded.md'];
|