# Backport
This will backport the following commits from `main` to `8.9`:
- [[FTR] Implement browser network condition utils
(#163633)](https://github.com/elastic/kibana/pull/163633)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Marco Antonio
Ghiani","email":"marcoantonio.ghiani01@gmail.com"},"sourceCommit":{"committedDate":"2023-08-11T12:24:06Z","message":"[FTR]
Implement browser network condition utils (#163633)\n\n## 📓
Summary\r\n\r\nThe PR implements some utilities into the `browser`
service to allow\r\ncontrolling the network conditions during the
execution of a functional\r\ntest.\r\n\r\n###
`getNetworkConditions`\r\n\r\nReturns the current network simulation
options. If none conditions are\r\npreviously set, it returns
`undefined`\r\n\r\n**N.B.**: _if the testing environment is not a
Chromium browser, it\r\nthrows an error that can be easily caught to
manually skip the test or\r\nhandle a fallback
scenario._\r\n\r\n\r\n```ts\r\nit('should display a loading skeleton
while loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n const networkConditions
= await browser.getNetworkConditions(); // undefined\r\n\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n\r\n const networkConditions
= await browser.getNetworkConditions();\r\n // {\r\n // offline:
false,\r\n // latency: 2000,\r\n // download_throughput: 50000,\r\n //
upload_throughput: 50000,\r\n // }\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
`setNetworkConditions`\r\n\r\nSet the desired network conditions.\r\nIt
supports different presets that match the [network profiles
provided\r\nby
Chrome\r\ndebugger](da276a3fae/front_end/core/sdk/NetworkManager.ts (L363-L393)):\r\n-
`NO_THROTTLING`\r\n- `FAST_3G`\r\n- `SLOW_3G`\r\n- `OFFLINE`\r\n-
`CLOUD_USER` (pre-existing)\r\nIt also accepts ad-hoc options to
configure more specifically the\r\nnetwork conditions.\r\n\r\n**N.B.**:
_if the testing environment is not a Chromium browser, it\r\nthrows an
error that can be easily caught to manually skip the test or\r\nhandle a
fallback scenario._\r\n\r\n```ts\r\nit('should display a loading
skeleton while loading', async function () {\r\n // Skip the test in
case network condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('NO_THROTTLING');\r\n await
browser.setNetworkConditions('FAST_3G');\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n await
browser.setNetworkConditions('OFFLINE');\r\n await
browser.setNetworkConditions('CLOUD_USER');\r\n await
browser.setNetworkConditions({\r\n offline: false,\r\n latency: 5, //
Additional latency (ms).\r\n download_throughput: 500 * 1024, // Maximal
aggregated download throughput.\r\n upload_throughput: 500 * 1024, //
Maximal aggregated upload throughput.\r\n });\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
restoreNetworkConditions\r\n\r\nRestore the original network conditions,
setting to `NO_THROTTLING`.\r\nThe native implementation of
`deleteNetworkConditions` exposed by\r\nselenium is unofficial and
didn't consistently work, the recommended\r\napproach by the google dev
tools team is to restore the connection\r\nsetting the no throttling
profile.\r\n\r\n**N.B.**: _if the testing environment is not a Chromium
browser, it\r\nthrows an error that can be easily caught to manually
skip the test or\r\nhandle a fallback
scenario._\r\n\r\n```ts\r\nit('should display a loading skeleton while
loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('SLOW_3G'); // Slow down network
conditions\r\n \r\n // Do your assertions\r\n\r\n await
browser.restoreNetworkConditions(); // Restore network conditions\r\n }
catch (error) {\r\n this.skip();\r\n
}\r\n});\r\n```\r\n\r\nCo-authored-by: Marco Antonio Ghiani
<marcoantonio.ghiani@elastic.co>\r\nCo-authored-by: Dzmitry Lemechko
<dzmitry.lemechko@elastic.co>","sha":"aa45152a4e787f36014675b864fcc0fce7bd6758","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.10.0","v7.17.13","v8.9.2"],"number":163633,"url":"https://github.com/elastic/kibana/pull/163633","mergeCommit":{"message":"[FTR]
Implement browser network condition utils (#163633)\n\n## 📓
Summary\r\n\r\nThe PR implements some utilities into the `browser`
service to allow\r\ncontrolling the network conditions during the
execution of a functional\r\ntest.\r\n\r\n###
`getNetworkConditions`\r\n\r\nReturns the current network simulation
options. If none conditions are\r\npreviously set, it returns
`undefined`\r\n\r\n**N.B.**: _if the testing environment is not a
Chromium browser, it\r\nthrows an error that can be easily caught to
manually skip the test or\r\nhandle a fallback
scenario._\r\n\r\n\r\n```ts\r\nit('should display a loading skeleton
while loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n const networkConditions
= await browser.getNetworkConditions(); // undefined\r\n\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n\r\n const networkConditions
= await browser.getNetworkConditions();\r\n // {\r\n // offline:
false,\r\n // latency: 2000,\r\n // download_throughput: 50000,\r\n //
upload_throughput: 50000,\r\n // }\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
`setNetworkConditions`\r\n\r\nSet the desired network conditions.\r\nIt
supports different presets that match the [network profiles
provided\r\nby
Chrome\r\ndebugger](da276a3fae/front_end/core/sdk/NetworkManager.ts (L363-L393)):\r\n-
`NO_THROTTLING`\r\n- `FAST_3G`\r\n- `SLOW_3G`\r\n- `OFFLINE`\r\n-
`CLOUD_USER` (pre-existing)\r\nIt also accepts ad-hoc options to
configure more specifically the\r\nnetwork conditions.\r\n\r\n**N.B.**:
_if the testing environment is not a Chromium browser, it\r\nthrows an
error that can be easily caught to manually skip the test or\r\nhandle a
fallback scenario._\r\n\r\n```ts\r\nit('should display a loading
skeleton while loading', async function () {\r\n // Skip the test in
case network condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('NO_THROTTLING');\r\n await
browser.setNetworkConditions('FAST_3G');\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n await
browser.setNetworkConditions('OFFLINE');\r\n await
browser.setNetworkConditions('CLOUD_USER');\r\n await
browser.setNetworkConditions({\r\n offline: false,\r\n latency: 5, //
Additional latency (ms).\r\n download_throughput: 500 * 1024, // Maximal
aggregated download throughput.\r\n upload_throughput: 500 * 1024, //
Maximal aggregated upload throughput.\r\n });\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
restoreNetworkConditions\r\n\r\nRestore the original network conditions,
setting to `NO_THROTTLING`.\r\nThe native implementation of
`deleteNetworkConditions` exposed by\r\nselenium is unofficial and
didn't consistently work, the recommended\r\napproach by the google dev
tools team is to restore the connection\r\nsetting the no throttling
profile.\r\n\r\n**N.B.**: _if the testing environment is not a Chromium
browser, it\r\nthrows an error that can be easily caught to manually
skip the test or\r\nhandle a fallback
scenario._\r\n\r\n```ts\r\nit('should display a loading skeleton while
loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('SLOW_3G'); // Slow down network
conditions\r\n \r\n // Do your assertions\r\n\r\n await
browser.restoreNetworkConditions(); // Restore network conditions\r\n }
catch (error) {\r\n this.skip();\r\n
}\r\n});\r\n```\r\n\r\nCo-authored-by: Marco Antonio Ghiani
<marcoantonio.ghiani@elastic.co>\r\nCo-authored-by: Dzmitry Lemechko
<dzmitry.lemechko@elastic.co>","sha":"aa45152a4e787f36014675b864fcc0fce7bd6758"}},"sourceBranch":"main","suggestedTargetBranches":["7.17","8.9"],"targetPullRequestStates":[{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/163633","number":163633,"mergeCommit":{"message":"[FTR]
Implement browser network condition utils (#163633)\n\n## 📓
Summary\r\n\r\nThe PR implements some utilities into the `browser`
service to allow\r\ncontrolling the network conditions during the
execution of a functional\r\ntest.\r\n\r\n###
`getNetworkConditions`\r\n\r\nReturns the current network simulation
options. If none conditions are\r\npreviously set, it returns
`undefined`\r\n\r\n**N.B.**: _if the testing environment is not a
Chromium browser, it\r\nthrows an error that can be easily caught to
manually skip the test or\r\nhandle a fallback
scenario._\r\n\r\n\r\n```ts\r\nit('should display a loading skeleton
while loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n const networkConditions
= await browser.getNetworkConditions(); // undefined\r\n\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n\r\n const networkConditions
= await browser.getNetworkConditions();\r\n // {\r\n // offline:
false,\r\n // latency: 2000,\r\n // download_throughput: 50000,\r\n //
upload_throughput: 50000,\r\n // }\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
`setNetworkConditions`\r\n\r\nSet the desired network conditions.\r\nIt
supports different presets that match the [network profiles
provided\r\nby
Chrome\r\ndebugger](da276a3fae/front_end/core/sdk/NetworkManager.ts (L363-L393)):\r\n-
`NO_THROTTLING`\r\n- `FAST_3G`\r\n- `SLOW_3G`\r\n- `OFFLINE`\r\n-
`CLOUD_USER` (pre-existing)\r\nIt also accepts ad-hoc options to
configure more specifically the\r\nnetwork conditions.\r\n\r\n**N.B.**:
_if the testing environment is not a Chromium browser, it\r\nthrows an
error that can be easily caught to manually skip the test or\r\nhandle a
fallback scenario._\r\n\r\n```ts\r\nit('should display a loading
skeleton while loading', async function () {\r\n // Skip the test in
case network condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('NO_THROTTLING');\r\n await
browser.setNetworkConditions('FAST_3G');\r\n await
browser.setNetworkConditions('SLOW_3G');\r\n await
browser.setNetworkConditions('OFFLINE');\r\n await
browser.setNetworkConditions('CLOUD_USER');\r\n await
browser.setNetworkConditions({\r\n offline: false,\r\n latency: 5, //
Additional latency (ms).\r\n download_throughput: 500 * 1024, // Maximal
aggregated download throughput.\r\n upload_throughput: 500 * 1024, //
Maximal aggregated upload throughput.\r\n });\r\n } catch (error) {\r\n
this.skip();\r\n }\r\n});\r\n```\r\n\r\n###
restoreNetworkConditions\r\n\r\nRestore the original network conditions,
setting to `NO_THROTTLING`.\r\nThe native implementation of
`deleteNetworkConditions` exposed by\r\nselenium is unofficial and
didn't consistently work, the recommended\r\napproach by the google dev
tools team is to restore the connection\r\nsetting the no throttling
profile.\r\n\r\n**N.B.**: _if the testing environment is not a Chromium
browser, it\r\nthrows an error that can be easily caught to manually
skip the test or\r\nhandle a fallback
scenario._\r\n\r\n```ts\r\nit('should display a loading skeleton while
loading', async function () {\r\n // Skip the test in case network
condition utils are not available\r\n try {\r\n await
browser.setNetworkConditions('SLOW_3G'); // Slow down network
conditions\r\n \r\n // Do your assertions\r\n\r\n await
browser.restoreNetworkConditions(); // Restore network conditions\r\n }
catch (error) {\r\n this.skip();\r\n
}\r\n});\r\n```\r\n\r\nCo-authored-by: Marco Antonio Ghiani
<marcoantonio.ghiani@elastic.co>\r\nCo-authored-by: Dzmitry Lemechko
<dzmitry.lemechko@elastic.co>","sha":"aa45152a4e787f36014675b864fcc0fce7bd6758"}},{"branch":"7.17","label":"v7.17.13","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.9","label":"v8.9.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani01@gmail.com>
# Backport
This will backport the following commits from `main` to `8.9`:
- [[ftr] Improve FTR error handling for NoSuchSessionError
(#161025)](https://github.com/elastic/kibana/pull/161025)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2023-07-04T16:26:57Z","message":"[ftr]
Improve FTR error handling for NoSuchSessionError (#161025)\n\n##
Summary\r\n\r\nSometimes browser/driver process dies during test run on
CI and FTR\r\nfails with errors cascade, good example
is\r\n[here](https://buildkite.com/elastic/kibana-pull-request/builds/138535#0188fd74-9adf-4011-8168-1bdc6e3d0f17)\r\n\r\n\r\n\r\nCurrent
behaviour on `main`: FTR lifecycle hooks, defined
in\r\n[remote](57aea91fae/test/functional/services/remote/remote.ts)\r\nservice,
has no access to the information about test suite run and\r\nparticular
test failure. These hooks are related to WebDriver (browser)\r\nstate
management and suppose to reset it to default state.\r\nCurrently we
silently fail screenshot taking which means tests execution\r\nis
continued even if `--bail` flag is passed. It ends with cascade
of\r\nfailures with the same error `NoSuchSessionError: invalid session
id`\r\n\r\n<details>\r\n\r\n<summary>FTR output on
failure</summary>\r\n\r\n```\r\n └- ✖ fail: discover/group1 discover
test query should show correct time range string by timepicker\r\n │
Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 23, 2015 @
18:31:44.000'\r\n │ at Assertion.assert (expect.js💯11)\r\n │ at
Assertion.apply (expect.js:227:8)\r\n │ at Assertion.be
(expect.js:69:22)\r\n │ at Context.<anonymous> (_discover.ts:53:31)\r\n
│ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n └-> \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n └- ✖ fail:
discover/group1 discover test query \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n └->
\"after all\" hook in \"discover test\"\r\n │ debg Cleaning all saved
objects { space: undefined }\r\n │ succ deleted 2 objects\r\n └->
\"after all\" hook: afterTestSuite.trigger in \"discover test\"\r\n └- ✖
fail: discover/group1 discover test \"after all\" hook:
afterTestSuite.trigger in \"discover test\"\r\n │ NoSuchSessionError:
invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n └->
\"after all\" hook: unloadMakelogs in \"discover/group1\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"mappings.json\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.22\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.20\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.21\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"data.json.gz\"\r\n └-> \"after all\" hook:
afterTestSuite.trigger in \"discover/group1\"\r\n └- ✖ fail:
discover/group1 \"after all\" hook: afterTestSuite.trigger in
\"discover/group1\"\r\n │ NoSuchSessionError: invalid session id\r\n │
at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
runMicrotasks (<anonymous>)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Task.exec
(prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │\r\n │0 passing
(15.7s)\r\n │4 failing\r\n │\r\n │1) discover/group1\r\n │ discover
test\r\n │ query\r\n │ should show correct time range string by
timepicker:\r\n │\r\n │ Error: expected 'Sep 19, 2015 @ 06:31:44.000' to
equal 'Sep 23, 2015 @ 18:31:44.000'\r\n │ at Assertion.assert
(expect.js💯11)\r\n │ at Assertion.apply (expect.js:227:8)\r\n │ at
Assertion.be (expect.js:69:22)\r\n │ at Context.<anonymous>
(_discover.ts:53:31)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n │2) discover/group1\r\n │
discover test\r\n │ query\r\n │ \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\":\r\n │\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │3)
discover/group1\r\n │ discover test\r\n │ \"after all\" hook:
afterTestSuite.trigger in \"discover test\":\r\n │\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │4)
discover/group1\r\n │ \"after all\" hook: afterTestSuite.trigger in
\"discover/group1\":\r\n │\r\n │ NoSuchSessionError: invalid session
id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
runMicrotasks (<anonymous>)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Task.exec
(prevent_parallel_calls.ts:28:20)\r\n```\r\n\r\n</details>\r\n\r\nThis
PR change: I didn't find a good reason why we need to fail
silently\r\non screenshot taking. I added a check WebDriver session
status with\r\n`hasOpenWindow` and take failure artefacts only if is
still valid.\r\nNext change is to fail FTR after hooks related to
WebDriver silently:\r\nthere is no help having cascade of the repeated
stacktrace so I wrap\r\nWebDriver call in hooks with `tryWebDriverCall`
that catches the error\r\nand only prints it for
visibility.\r\n\r\n<details>\r\n\r\n<summary>FTR new output on
failure</summary>\r\n\r\n```\r\n │ERROR WebDriver session is no longer
valid.\r\n │ Probably Chrome process crashed when it tried to use more
memory than what was available.\r\n │ERROR Browser is closed, no
artifacts were captured for the failure\r\n └- ✖ fail: discover/group1
discover test query should show correct time range string by
timepicker\r\n │ Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal
'Sep 23, 2015 @ 18:31:44.000'\r\n │ at Assertion.assert
(expect.js💯11)\r\n │ at Assertion.apply (expect.js:227:8)\r\n │ at
Assertion.be (expect.js:69:22)\r\n │ at Context.<anonymous>
(_discover.ts:53:31)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n └-> \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n │ERROR WebDriver
session is no longer valid\r\n └-> \"after all\" hook in \"discover
test\"\r\n │ debg Cleaning all saved objects { space: undefined }\r\n │
warn browser[SEVERE] ERROR FETCHING BROWSR LOGS: This driver instance
does not have a valid session ID (did you call WebDriver.quit()?) and
may no longer be used.\r\n │ succ deleted 2 objects\r\n └-> \"after
all\" hook: afterTestSuite.trigger in \"discover test\"\r\n │ERROR
WebDriver session is no longer valid\r\n └-> \"after all\" hook:
unloadMakelogs in \"discover/group1\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"mappings.json\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.22\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.20\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.21\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"data.json.gz\"\r\n └-> \"after all\" hook:
afterTestSuite.trigger in \"discover/group1\"\r\n │ERROR WebDriver
session is no longer valid\r\n\r\n0 passing (16.2s)\r\n1
failing\r\n\r\n1) discover/group1\r\n discover test\r\n query\r\n should
show correct time range string by timepicker:\r\n\r\n Error: expected
'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 23, 2015 @ 18:31:44.000'\r\n
at Assertion.assert (expect.js💯11)\r\n at Assertion.apply
(expect.js:227:8)\r\n at Assertion.be (expect.js:69:22)\r\n at
Context.<anonymous> (_discover.ts:53:31)\r\n at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n
at Object.apply
(wrap_function.js:73:16)\r\n```\r\n\r\n</details>\r\n\r\n\r\nFlaky-test-runner
verification: started 100x to hopefully catch invalid\r\nsession on
CI\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2539\r\nNote:
locally I was simulating it by calling `this.driver.close()` to\r\nclose
browser before screenshot
taking","sha":"8a95bf7fabe34c601725313c922f95ee0fa641e5","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.9.0","v8.10.0","v8.8.3"],"number":161025,"url":"https://github.com/elastic/kibana/pull/161025","mergeCommit":{"message":"[ftr]
Improve FTR error handling for NoSuchSessionError (#161025)\n\n##
Summary\r\n\r\nSometimes browser/driver process dies during test run on
CI and FTR\r\nfails with errors cascade, good example
is\r\n[here](https://buildkite.com/elastic/kibana-pull-request/builds/138535#0188fd74-9adf-4011-8168-1bdc6e3d0f17)\r\n\r\n\r\n\r\nCurrent
behaviour on `main`: FTR lifecycle hooks, defined
in\r\n[remote](57aea91fae/test/functional/services/remote/remote.ts)\r\nservice,
has no access to the information about test suite run and\r\nparticular
test failure. These hooks are related to WebDriver (browser)\r\nstate
management and suppose to reset it to default state.\r\nCurrently we
silently fail screenshot taking which means tests execution\r\nis
continued even if `--bail` flag is passed. It ends with cascade
of\r\nfailures with the same error `NoSuchSessionError: invalid session
id`\r\n\r\n<details>\r\n\r\n<summary>FTR output on
failure</summary>\r\n\r\n```\r\n └- ✖ fail: discover/group1 discover
test query should show correct time range string by timepicker\r\n │
Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 23, 2015 @
18:31:44.000'\r\n │ at Assertion.assert (expect.js💯11)\r\n │ at
Assertion.apply (expect.js:227:8)\r\n │ at Assertion.be
(expect.js:69:22)\r\n │ at Context.<anonymous> (_discover.ts:53:31)\r\n
│ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n └-> \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n └- ✖ fail:
discover/group1 discover test query \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n └->
\"after all\" hook in \"discover test\"\r\n │ debg Cleaning all saved
objects { space: undefined }\r\n │ succ deleted 2 objects\r\n └->
\"after all\" hook: afterTestSuite.trigger in \"discover test\"\r\n └- ✖
fail: discover/group1 discover test \"after all\" hook:
afterTestSuite.trigger in \"discover test\"\r\n │ NoSuchSessionError:
invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n └->
\"after all\" hook: unloadMakelogs in \"discover/group1\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"mappings.json\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.22\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.20\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.21\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"data.json.gz\"\r\n └-> \"after all\" hook:
afterTestSuite.trigger in \"discover/group1\"\r\n └- ✖ fail:
discover/group1 \"after all\" hook: afterTestSuite.trigger in
\"discover/group1\"\r\n │ NoSuchSessionError: invalid session id\r\n │
at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
runMicrotasks (<anonymous>)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Task.exec
(prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │\r\n │0 passing
(15.7s)\r\n │4 failing\r\n │\r\n │1) discover/group1\r\n │ discover
test\r\n │ query\r\n │ should show correct time range string by
timepicker:\r\n │\r\n │ Error: expected 'Sep 19, 2015 @ 06:31:44.000' to
equal 'Sep 23, 2015 @ 18:31:44.000'\r\n │ at Assertion.assert
(expect.js💯11)\r\n │ at Assertion.apply (expect.js:227:8)\r\n │ at
Assertion.be (expect.js:69:22)\r\n │ at Context.<anonymous>
(_discover.ts:53:31)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n │2) discover/group1\r\n │
discover test\r\n │ query\r\n │ \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\":\r\n │\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │3)
discover/group1\r\n │ discover test\r\n │ \"after all\" hook:
afterTestSuite.trigger in \"discover test\":\r\n │\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │4)
discover/group1\r\n │ \"after all\" hook: afterTestSuite.trigger in
\"discover/group1\":\r\n │\r\n │ NoSuchSessionError: invalid session
id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
runMicrotasks (<anonymous>)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Task.exec
(prevent_parallel_calls.ts:28:20)\r\n```\r\n\r\n</details>\r\n\r\nThis
PR change: I didn't find a good reason why we need to fail
silently\r\non screenshot taking. I added a check WebDriver session
status with\r\n`hasOpenWindow` and take failure artefacts only if is
still valid.\r\nNext change is to fail FTR after hooks related to
WebDriver silently:\r\nthere is no help having cascade of the repeated
stacktrace so I wrap\r\nWebDriver call in hooks with `tryWebDriverCall`
that catches the error\r\nand only prints it for
visibility.\r\n\r\n<details>\r\n\r\n<summary>FTR new output on
failure</summary>\r\n\r\n```\r\n │ERROR WebDriver session is no longer
valid.\r\n │ Probably Chrome process crashed when it tried to use more
memory than what was available.\r\n │ERROR Browser is closed, no
artifacts were captured for the failure\r\n └- ✖ fail: discover/group1
discover test query should show correct time range string by
timepicker\r\n │ Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal
'Sep 23, 2015 @ 18:31:44.000'\r\n │ at Assertion.assert
(expect.js💯11)\r\n │ at Assertion.apply (expect.js:227:8)\r\n │ at
Assertion.be (expect.js:69:22)\r\n │ at Context.<anonymous>
(_discover.ts:53:31)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n └-> \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n │ERROR WebDriver
session is no longer valid\r\n └-> \"after all\" hook in \"discover
test\"\r\n │ debg Cleaning all saved objects { space: undefined }\r\n │
warn browser[SEVERE] ERROR FETCHING BROWSR LOGS: This driver instance
does not have a valid session ID (did you call WebDriver.quit()?) and
may no longer be used.\r\n │ succ deleted 2 objects\r\n └-> \"after
all\" hook: afterTestSuite.trigger in \"discover test\"\r\n │ERROR
WebDriver session is no longer valid\r\n └-> \"after all\" hook:
unloadMakelogs in \"discover/group1\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"mappings.json\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.22\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.20\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.21\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"data.json.gz\"\r\n └-> \"after all\" hook:
afterTestSuite.trigger in \"discover/group1\"\r\n │ERROR WebDriver
session is no longer valid\r\n\r\n0 passing (16.2s)\r\n1
failing\r\n\r\n1) discover/group1\r\n discover test\r\n query\r\n should
show correct time range string by timepicker:\r\n\r\n Error: expected
'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 23, 2015 @ 18:31:44.000'\r\n
at Assertion.assert (expect.js💯11)\r\n at Assertion.apply
(expect.js:227:8)\r\n at Assertion.be (expect.js:69:22)\r\n at
Context.<anonymous> (_discover.ts:53:31)\r\n at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n
at Object.apply
(wrap_function.js:73:16)\r\n```\r\n\r\n</details>\r\n\r\n\r\nFlaky-test-runner
verification: started 100x to hopefully catch invalid\r\nsession on
CI\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2539\r\nNote:
locally I was simulating it by calling `this.driver.close()` to\r\nclose
browser before screenshot
taking","sha":"8a95bf7fabe34c601725313c922f95ee0fa641e5"}},"sourceBranch":"main","suggestedTargetBranches":["8.9","8.8"],"targetPullRequestStates":[{"branch":"8.9","label":"v8.9.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/161025","number":161025,"mergeCommit":{"message":"[ftr]
Improve FTR error handling for NoSuchSessionError (#161025)\n\n##
Summary\r\n\r\nSometimes browser/driver process dies during test run on
CI and FTR\r\nfails with errors cascade, good example
is\r\n[here](https://buildkite.com/elastic/kibana-pull-request/builds/138535#0188fd74-9adf-4011-8168-1bdc6e3d0f17)\r\n\r\n\r\n\r\nCurrent
behaviour on `main`: FTR lifecycle hooks, defined
in\r\n[remote](57aea91fae/test/functional/services/remote/remote.ts)\r\nservice,
has no access to the information about test suite run and\r\nparticular
test failure. These hooks are related to WebDriver (browser)\r\nstate
management and suppose to reset it to default state.\r\nCurrently we
silently fail screenshot taking which means tests execution\r\nis
continued even if `--bail` flag is passed. It ends with cascade
of\r\nfailures with the same error `NoSuchSessionError: invalid session
id`\r\n\r\n<details>\r\n\r\n<summary>FTR output on
failure</summary>\r\n\r\n```\r\n └- ✖ fail: discover/group1 discover
test query should show correct time range string by timepicker\r\n │
Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 23, 2015 @
18:31:44.000'\r\n │ at Assertion.assert (expect.js💯11)\r\n │ at
Assertion.apply (expect.js:227:8)\r\n │ at Assertion.be
(expect.js:69:22)\r\n │ at Context.<anonymous> (_discover.ts:53:31)\r\n
│ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n └-> \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n └- ✖ fail:
discover/group1 discover test query \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n └->
\"after all\" hook in \"discover test\"\r\n │ debg Cleaning all saved
objects { space: undefined }\r\n │ succ deleted 2 objects\r\n └->
\"after all\" hook: afterTestSuite.trigger in \"discover test\"\r\n └- ✖
fail: discover/group1 discover test \"after all\" hook:
afterTestSuite.trigger in \"discover test\"\r\n │ NoSuchSessionError:
invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n └->
\"after all\" hook: unloadMakelogs in \"discover/group1\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"mappings.json\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.22\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.20\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.21\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"data.json.gz\"\r\n └-> \"after all\" hook:
afterTestSuite.trigger in \"discover/group1\"\r\n └- ✖ fail:
discover/group1 \"after all\" hook: afterTestSuite.trigger in
\"discover/group1\"\r\n │ NoSuchSessionError: invalid session id\r\n │
at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
runMicrotasks (<anonymous>)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Task.exec
(prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │\r\n │0 passing
(15.7s)\r\n │4 failing\r\n │\r\n │1) discover/group1\r\n │ discover
test\r\n │ query\r\n │ should show correct time range string by
timepicker:\r\n │\r\n │ Error: expected 'Sep 19, 2015 @ 06:31:44.000' to
equal 'Sep 23, 2015 @ 18:31:44.000'\r\n │ at Assertion.assert
(expect.js💯11)\r\n │ at Assertion.apply (expect.js:227:8)\r\n │ at
Assertion.be (expect.js:69:22)\r\n │ at Context.<anonymous>
(_discover.ts:53:31)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n │2) discover/group1\r\n │
discover test\r\n │ query\r\n │ \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\":\r\n │\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │3)
discover/group1\r\n │ discover test\r\n │ \"after all\" hook:
afterTestSuite.trigger in \"discover test\":\r\n │\r\n │
NoSuchSessionError: invalid session id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n │
at Task.exec (prevent_parallel_calls.ts:28:20)\r\n │\r\n │\r\n │4)
discover/group1\r\n │ \"after all\" hook: afterTestSuite.trigger in
\"discover/group1\":\r\n │\r\n │ NoSuchSessionError: invalid session
id\r\n │ at Object.throwDecodedError
(node_modules/selenium-webdriver/lib/error.js:524:15)\r\n │ at
parseHttpResponse
(node_modules/selenium-webdriver/lib/http.js:601:13)\r\n │ at
Executor.execute
(node_modules/selenium-webdriver/lib/http.js:529:28)\r\n │ at
runMicrotasks (<anonymous>)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Task.exec
(prevent_parallel_calls.ts:28:20)\r\n```\r\n\r\n</details>\r\n\r\nThis
PR change: I didn't find a good reason why we need to fail
silently\r\non screenshot taking. I added a check WebDriver session
status with\r\n`hasOpenWindow` and take failure artefacts only if is
still valid.\r\nNext change is to fail FTR after hooks related to
WebDriver silently:\r\nthere is no help having cascade of the repeated
stacktrace so I wrap\r\nWebDriver call in hooks with `tryWebDriverCall`
that catches the error\r\nand only prints it for
visibility.\r\n\r\n<details>\r\n\r\n<summary>FTR new output on
failure</summary>\r\n\r\n```\r\n │ERROR WebDriver session is no longer
valid.\r\n │ Probably Chrome process crashed when it tried to use more
memory than what was available.\r\n │ERROR Browser is closed, no
artifacts were captured for the failure\r\n └- ✖ fail: discover/group1
discover test query should show correct time range string by
timepicker\r\n │ Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal
'Sep 23, 2015 @ 18:31:44.000'\r\n │ at Assertion.assert
(expect.js💯11)\r\n │ at Assertion.apply (expect.js:227:8)\r\n │ at
Assertion.be (expect.js:69:22)\r\n │ at Context.<anonymous>
(_discover.ts:53:31)\r\n │ at processTicksAndRejections
(node:internal/process/task_queues:96:5)\r\n │ at Object.apply
(wrap_function.js:73:16)\r\n │\r\n │\r\n └-> \"after all\" hook:
afterTestSuite.trigger for \"should reload the saved search with
persisted query to show the initial hit count\"\r\n │ERROR WebDriver
session is no longer valid\r\n └-> \"after all\" hook in \"discover
test\"\r\n │ debg Cleaning all saved objects { space: undefined }\r\n │
warn browser[SEVERE] ERROR FETCHING BROWSR LOGS: This driver instance
does not have a valid session ID (did you call WebDriver.quit()?) and
may no longer be used.\r\n │ succ deleted 2 objects\r\n └-> \"after
all\" hook: afterTestSuite.trigger in \"discover test\"\r\n │ERROR
WebDriver session is no longer valid\r\n └-> \"after all\" hook:
unloadMakelogs in \"discover/group1\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"mappings.json\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.22\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.20\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Deleted
existing index \"logstash-2015.09.21\"\r\n │ info
[test/functional/fixtures/es_archiver/logstash_functional] Unloading
indices from \"data.json.gz\"\r\n └-> \"after all\" hook:
afterTestSuite.trigger in \"discover/group1\"\r\n │ERROR WebDriver
session is no longer valid\r\n\r\n0 passing (16.2s)\r\n1
failing\r\n\r\n1) discover/group1\r\n discover test\r\n query\r\n should
show correct time range string by timepicker:\r\n\r\n Error: expected
'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 23, 2015 @ 18:31:44.000'\r\n
at Assertion.assert (expect.js💯11)\r\n at Assertion.apply
(expect.js:227:8)\r\n at Assertion.be (expect.js:69:22)\r\n at
Context.<anonymous> (_discover.ts:53:31)\r\n at
processTicksAndRejections (node:internal/process/task_queues:96:5)\r\n
at Object.apply
(wrap_function.js:73:16)\r\n```\r\n\r\n</details>\r\n\r\n\r\nFlaky-test-runner
verification: started 100x to hopefully catch invalid\r\nsession on
CI\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2539\r\nNote:
locally I was simulating it by calling `this.driver.close()` to\r\nclose
browser before screenshot
taking","sha":"8a95bf7fabe34c601725313c922f95ee0fa641e5"}},{"branch":"8.8","label":"v8.8.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## Summary
Fixes#158167
The issue occurred because we were using a number called "index" as the
React key prop. When we removed the first element, the second one took
its place, but React still showed the removed element.
To fix this problem, we found a solution by using the uuid field that
each action item has. We now generate it for each new action we create
in the action form and use that as key
I was told to add @pmuellr as you might know if we are missing something
🙇
### Test: why xpath selector?
I had to use the xpath selector to fix a problem we had. The problem was
between two actions we set up. They look the same, but the body content
is the only different (I've attached a screenshot for more details).
We use a third party component for these actions. This component doesn't
have any "value" attribute and doesn't add anything besides the HTML
text. I tried to find other useful details but couldn't find any.
The problem comes up when we try to delete one of the actions. To fix
it, we needed to check if that component was missing. We already have
tools that can look for missing components, but they don't work with the
xpath selector. So, I added a new function that can do this. Now, we can
use the xpath selector to look for missing components and fix the
problem
<details>
<summary>See Screenshot</summary>
<img
src="5447795d-0281-4847-aa85-76d0e5fdec3d"/>
</details>
```[tasklist]
- [x] Make sure that it's ok to generate the uuid
- [x] Test
- [x] Do we need to backport? Versions?
```
## Summary
Starting with Chrome v113 we noticed that `_area_chart.ts` suite became
flaky #156821 failing with
`WebDriverError: unknown error: unhandled inspector error:
{"code":-32000,"message":"No node with given id found"}`
Updating chromedriver to v113 did not solve the issue and more tests
started to fail with the same error.
It happens occasionally when driver returns unhandled error instead of
StaleElementReferenceException. This PR adds the error to the
`RETRY_ON_ERRORS` list, so that FTR can search for the element again and
re-try the action on it:
```
│ debg getVisibleText: elementId=29C3E81151C86107290DD8F020524333_element_290
│ debg Chromedriver issue #4440, WebElementWrapper.getVisibleText: WebDriverError: unknown error: unhandled inspector error: {"code":-32000,"message":"No node with given id found"}
│ (Session info: chrome=113.0.5672.63)
│ debg current ElementID=29C3E81151C86107290DD8F020524333_element_290
│ debg new ElementID=29C3E81151C86107290DD8F020524333_element_293
│ debg Searching again for the element 'By(css selector, [data-test-subj="visEditorInterval"] + .euiFormErrorText)', 2 attempts left
│ debg getVisibleText: elementId=29C3E81151C86107290DD8F020524333_element_293
└- ✓ pass (1.8s)
```
There is no need to use `Retry` service, `WebDriverWrapper.retryCall`
should handle the issue.
Flaky test runner 100x for Vis Editor config:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2236
Flaky test runner 100x for Cases config:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237
And 2 more 100x:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2239https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237
## Summary
This PR adds `NoSuchSessionError: invalid session id` to the irrelevant
failures, so that we don't open github issues for it.
The issue is often related to test infrastructure (worker) and occurs
when Chrome process is reaching the available memory limits. It doesn't
fail regularly but when it fails multiple github issues are
[reported](https://github.com/elastic/kibana/issues?q=is%3Aissue+NoSuchSessionError+is%3Aopen)
and we usually just close them as CI related.
## Summary
Fixes#75316
* Lens
* [x] Refactored Top nav actions code to be more modular
* [x] Created new Locator object for Lens
* [x] Enabled server side to make Short URL work with it
* [x] Added unit tests for it
* [x] Extended `getEditPath` to support `filters` and `refreshInterval`
* [x] Extended mounting code in Lens to handle a new type of incoming
context
* [x] Add new Share action
* [x] Added new `objectTypeTitle` prop to have custom titles on Share
popover
* [x] Replaced the `Download CSV` action and move it as menu item
* [x] Refactor code into share item provider + (lazy) panel content
* [x] Add debug flag to make CSV download testable
* [x] Add functional tests for CSV download
* [x] Add Permalink action
* [x] Integrate Permalink with Short URL service
* [x] Tweaked Permalink action to work with SO custom URL
* [x] Tweaked Permalink action to handle disabled state
* [x] Updated unit tests with new features
* [x] Added (basic) caching logic to avoid too many snapshot duplicate
Short URLs
* [x] New share function test suite created
* [x] Extended `browser` service with a new method to have a blank tab
in browser
* [x] New helper functions in Lens to test Share feature
<img width="375" alt="Screenshot 2023-01-11 at 12 58 30"
src="https://user-images.githubusercontent.com/924948/211800819-60efe70a-9ebe-4bde-82e0-8fa264e8c4af.png">
<img width="427" alt="Screenshot 2023-01-11 at 12 58 36"
src="https://user-images.githubusercontent.com/924948/211800825-ae7b86d0-0e42-4227-a425-cdcd94ec78cb.png">
<img width="426" alt="Screenshot 2023-01-11 at 12 58 40"
src="https://user-images.githubusercontent.com/924948/211800827-73bfb773-b30e-495c-aa61-f5fd10f35d31.png">
<img width="428" alt="Screenshot 2023-01-11 at 12 58 46"
src="https://user-images.githubusercontent.com/924948/211800830-89539c37-7495-48f0-9de6-b7d6f15b7397.png">
<img width="427" alt="Screenshot 2023-01-11 at 12 59 03"
src="https://user-images.githubusercontent.com/924948/211800833-6f1843b9-ab22-49d9-adbd-8f5f588b52e7.png">
### Notes
#### Short URL requirement
This feature strictly requires the ShortURL service to be enabled to
work, otherwise the permalink feature is disabled for snapshot sharing.
This requirement is not clearly stated in the Share menu (yet) like
other app do as the Sharing flow had to be customised in Lens due to
some other technical challenges.
Would it be ok to workout a UI improvement as follow up?
#### Context tech debt
The way the locator works as injecting the shared state into the context
produced a discrete amount of branching, due to inconsistency of the
`context` type coming from different sources (Discover, Agg-based/TSVB,
Lens itself...). Perhaps it's worth discussing having a refactoring of
the context type here?
#### Missing locator service
Due to the way the sharing logic works in Lens the locator has not been
exported from the `plugin` functions. I thought to add a custom function
for it, but perhaps we could investigate a bit better whether this is
needed and eventually its implementation in a follow up task.
## How is the snapshot URL generated?
```mermaid
sequenceDiagram
actor User
User->>Share Snapshot URL: click
Share Snapshot URL->> Lens ShortUrlService: Lens state
Lens ShortUrlService->> Lens ShortUrlService: Check cache based on state
Lens ShortUrlService->> ShortUrlService: Generate a Short URL
ShortUrlService->> Lens ShortUrlService: new Short URL
Lens ShortUrlService->> Application `getUrlForApp`: Build absolute URL
Application `getUrlForApp`->> Lens ShortUrlService: final URL
Lens ShortUrlService->>Share Snapshot URL: final URL
Share Snapshot URL->>User: final URL copied in clipboard
```
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Andrew Tate <drewctate@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
[Tests] add smoke functional tests for "render" telemetries
This PR blocked by #143734 and #143552
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
* [ts] set allowJs to true by default
* fix scripts/check_ts_projects, original implementation is now wildly inefficient
* produce stats in check_ts_projects to make sure it's actually working
* fix imports
* [ftr] add first-class support for playwrite journeys
* [CI] Auto-commit changed files from 'node scripts/generate codeowners'
* fix jest test
* remove ability to customize kibana server args, if we need it we can add it back
* remove dev dir that doesn't exist
* fix typo
* prevent duplicated array converstion logic by sharing flag reader
* remove destructuring of option
* fix scalability config and config_path import
* fix start_servers args and tests
* include simple readme
* fix jest tests and support build re-use when changes are just to jest tests
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* Removed file-type from package.json deps.
Updated jimp to 0.16.1.
Updated light & dark dashboard baseline screenshots to resolve failure due to resize degradation.
* Replaced jimp with sharp and pixelmatch. Eliminates several deps, including file-type.
Eliminates quality degradation from image resizing.
* Fixed copy paste error, which lead to pass positives and errors.
* Integrated updateBaselines flag into dashboards report testing.
Fixed sharp resize to fit and not truncate the original image.
* Removed accidental '.only'
* Reverted area chart baseline.
* Removed explicit declaration of sharp from package.json.
* Adjusted dashboard screenshot size in attempt to eliminate layout scaling issues in CI testing.
* Reverted dashboard screenshot size. Lowered tolerance in dashboard PNG reporting tests to force failure and collect CI screenshots.
* Changed session save location for dashboard report images for better CI reporting.
* Fixed small dashboard baseline screenshot with correct aspect ratio.
* Updated 2 basline screenshots from CI. Lowered PNG match tolerance for dashboard screenshots to 1% for investigation of remaining screens.
* Update of dashboard controls light mode baseline.
Change of forced container width size in large dashboard layout test.
* Updated dashboard controls dark mode baseline screenshot.
Adjusted forced container width in large dashboard layout test for troubleshooting.
* Lowering tolerance of PNG compare tests from 9% to 1% for troubleshooting.
Replaced references to dedundant image service with existing, more adopted reporting service.
Ideally, the PNG functions of the reporting service would be merged with the screenshot service as an image service.
* Reverting tolerance of PNG compares to 0.09. Will create GH issues for specific teams to investigate.
* Implemented PngService to provide an accessible single reference to PNG compare testing.
Removed redundant implementations and references.
* Updated service reference and call to png compare function left out of last commit.
* Updated another missed PNG reference.
* Removed superfluous parameter.
* Update after bootstrap
* Updated area chart baseline to match new render with panel filters.
* Reverted limits in dashboard snapshot tests to original values.
* Merge and rebuild.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [ftr] automatically determine config run order
* split lens config into two groups
* support ftr configs always running against CI
* Split detection_engine_api_integration rule exception list tests
* Add configs from previous commit
* [ftr] remove testMetadata and maintain a unique lifecycle instance per run
* Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run"
This reverts commit d2b4fdb824.
* Split alerting_api_integration/security_and_spaces tests
* Add groups to yaml
* Revert "Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run""
This reverts commit 56232eea68.
* stop ES more forcefully and fix timeout
* only cleanup lifecycle phases when the cleanup is totally complete
* only use kill when cleaning up an esTestInstance
* fix broken import
* fix runOptions.alwaysUseSource implementation
* fix config access
* fix x-pack/ccs config
* fix ml import file paths
* update kibana build id
* revert array.concat() change
* fix baseConfig usage
* fix pie chart data
* split up maps tests
* pull in all of group5 so that es archives are loaded correctly
* add to ftr configs.yml
* fix pie chart data without breaking legacy version
* fix more pie_chart stuff in new vis lib
* restore normal PR tasks
* bump kibana-buildkite-library
* remove ciGroup validation
* remove the script which is no longer called from checks.sh
* [CI] Auto-commit changed files from 'yarn kbn run build -i @kbn/pm'
* adapt flaky test runner scripts to handle ftrConfig paths
* fix types in alerting_api_integration
* improve flaky config parsing and use non-local var name for passing explicit configs to ftr_configs.sh
* Split xpack dashboard tests
* Add configs
* [flaky] remove key from ftr-config steps
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
* restore cypress builds
* remove ciGroups from FTR config files
* fixup some docs
* add temporary script to hunt for FTR config files
* use config.base.js naming for clarity
* use script to power ftr_configs.yml
* remove usage of removed x-pack/scripts/functional_tests
* fix test names in dashboard snapshots
* bump kibana-buildkite-library
* Try retrying only failed configs
* be a little quieter about trying to get testStats from configs with testRunners defined
* Remove test code
* bump kibana-buildkite-library
* update es_snapshot and on_merge jobs too
* track duration and exit code for each config and print it at the end of the script
* store results in order, rather than by key, in case there are duplicates in $config
* bash is hard
* fix env source and use +e rather than disabling e for whole file
* bash sucks
* print config summary in jest jobs too
* define results in jest_parallel.sh
* simplify config summary print, format times a little better
* fix reference to unbound time variable, use better variable name
* skip the newline between each result
* finish with the nitpicking
* sync changes with ftr_configs.sh
* refuse to execute config files which aren't listed in the .buildkite/ftr_configs.yml
* fix config.edge.js base config import paths
* fix some readmes
* resolve paths from ftr_configs manifest
* fix readConfigFile tests
* just allow __fixtures__ configs
* list a few more cypress config files
* install the main branch of kibana-buildkite-library
* split up lens group1
* move ml data_visualizer tests to their own config
* fix import paths
* fix more imports
* install specific commit of buildkite-pipeline-library
* sort configs in ftr_configs.yml
* bump kibana-buildkite-library
* remove temporary script
* fix env var for limiting config types
* Update docs/developer/contributing/development-functional-tests.asciidoc
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* produce a JUnit report for saved objects field count
* apply standard concurrency limits from flaky test runner
* support customizing FTR concurrency via the env
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* [Solution Toolbar] Fixing button border on non-text color versions
* [Alerts] Removed extra wrappers and use EuiPageHeader
* [Logstash] Basic conversion to template
* [Reporting] Adding bottomBorder to page header
* [ML] Fix display of main navigation tabs
* [Stack Management] Fix side nav not updating when going back to landing page
* [Tags] Add spacing after page header
* [License Management] Full width on file uploader
* [Page Template] Fixed `emptyState` default template for pages with side nav
* [Infra] Removing some page header displays in empty states
* [Enterprise Search] Fix some error layouts
* [Index Patterns] Quick fix for empty state
* snaps
* [Page Template] Remove forced padding when `centeredBody`
* small hack for tab padding for ml
* scroll ML page to fix test
* fix test method type signature
Co-authored-by: Dave Snider <dave.snider@gmail.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co>
This PR adds screenshot comparison tests for the nodes in the graph on the test plugin.
Run the tests using this command:
`yarn test:ftr --config x-pack/test/plugin_functional/config.ts --grep Resolver`
* Rename kibana_breadcrumbs to kibana_chrome
- in anticipation of upcoming refactor where SetPageChrome now handles document title as well as Kibana breadcrumbs
+ cleanup exports
* Add generate_title helpers
- will be used by new set_chrome helper
* Add setDocTitle context + behavior to set_chrome
+ refactor set_chrome.test.tsx:
- add title tests
- add SetWorkplaceSearchChrome test to increase coverage
- clean up inner/outer call in favor of simpler mocks/imports
- simplify isRoot tests
* Update plugins to set product titles
- on mount but before render
* Copy feedback - change pipe separator to hyphen
- to match ' - Elastic'
* Add functional tests confirming that document titles updated
