[8.8] Update ftr services to handle occasional WebDriverError (#157283) (#157326)

# Backport

This will backport the following commits from `main` to `8.8`:
- [Update ftr services to handle occasional WebDriverError
(#157283)](https://github.com/elastic/kibana/pull/157283)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2023-05-10T18:00:44Z","message":"Update
ftr services to handle occasional WebDriverError (#157283)\n\n##
Summary\r\n\r\nStarting with Chrome v113 we noticed that
`_area_chart.ts` suite became\r\nflaky #156821 failing
with\r\n`WebDriverError: unknown error: unhandled inspector
error:\r\n{\"code\":-32000,\"message\":\"No node with given id
found\"}`\r\nUpdating chromedriver to v113 did not solve the issue and
more tests\r\nstarted to fail with the same error.\r\n\r\nIt happens
occasionally when driver returns unhandled error instead
of\r\nStaleElementReferenceException. This PR adds the error to
the\r\n`RETRY_ON_ERRORS` list, so that FTR can search for the element
again and\r\nre-try the action on it:\r\n\r\n```\r\n │ debg
getVisibleText:
elementId=29C3E81151C86107290DD8F020524333_element_290\r\n │ debg
Chromedriver issue #4440, WebElementWrapper.getVisibleText:
WebDriverError: unknown error: unhandled inspector error:
{\"code\":-32000,\"message\":\"No node with given id found\"}\r\n │
(Session info: chrome=113.0.5672.63)\r\n │ debg current
ElementID=29C3E81151C86107290DD8F020524333_element_290\r\n │ debg new
ElementID=29C3E81151C86107290DD8F020524333_element_293\r\n │ debg
Searching again for the element 'By(css selector,
[data-test-subj=\"visEditorInterval\"] + .euiFormErrorText)', 2 attempts
left\r\n │ debg getVisibleText:
elementId=29C3E81151C86107290DD8F020524333_element_293\r\n └- ✓ pass
(1.8s)\r\n```\r\n\r\nThere is no need to use `Retry` service,
`WebDriverWrapper.retryCall`\r\nshould handle the issue.\r\n\r\nFlaky
test runner 100x for Vis Editor
config:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2236\r\nFlaky
test runner 100x for Cases
config:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237\r\n\r\nAnd
2 more
100x:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2239\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237","sha":"7f4ceb1d302cf24411c1b82030b882e87a8616dd","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.7.2","v8.9.0","v8.8.1"],"number":157283,"url":"https://github.com/elastic/kibana/pull/157283","mergeCommit":{"message":"Update
ftr services to handle occasional WebDriverError (#157283)\n\n##
Summary\r\n\r\nStarting with Chrome v113 we noticed that
`_area_chart.ts` suite became\r\nflaky #156821 failing
with\r\n`WebDriverError: unknown error: unhandled inspector
error:\r\n{\"code\":-32000,\"message\":\"No node with given id
found\"}`\r\nUpdating chromedriver to v113 did not solve the issue and
more tests\r\nstarted to fail with the same error.\r\n\r\nIt happens
occasionally when driver returns unhandled error instead
of\r\nStaleElementReferenceException. This PR adds the error to
the\r\n`RETRY_ON_ERRORS` list, so that FTR can search for the element
again and\r\nre-try the action on it:\r\n\r\n```\r\n │ debg
getVisibleText:
elementId=29C3E81151C86107290DD8F020524333_element_290\r\n │ debg
Chromedriver issue #4440, WebElementWrapper.getVisibleText:
WebDriverError: unknown error: unhandled inspector error:
{\"code\":-32000,\"message\":\"No node with given id found\"}\r\n │
(Session info: chrome=113.0.5672.63)\r\n │ debg current
ElementID=29C3E81151C86107290DD8F020524333_element_290\r\n │ debg new
ElementID=29C3E81151C86107290DD8F020524333_element_293\r\n │ debg
Searching again for the element 'By(css selector,
[data-test-subj=\"visEditorInterval\"] + .euiFormErrorText)', 2 attempts
left\r\n │ debg getVisibleText:
elementId=29C3E81151C86107290DD8F020524333_element_293\r\n └- ✓ pass
(1.8s)\r\n```\r\n\r\nThere is no need to use `Retry` service,
`WebDriverWrapper.retryCall`\r\nshould handle the issue.\r\n\r\nFlaky
test runner 100x for Vis Editor
config:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2236\r\nFlaky
test runner 100x for Cases
config:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237\r\n\r\nAnd
2 more
100x:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2239\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237","sha":"7f4ceb1d302cf24411c1b82030b882e87a8616dd"}},"sourceBranch":"main","suggestedTargetBranches":["8.7","8.8"],"targetPullRequestStates":[{"branch":"8.7","label":"v8.7.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/157283","number":157283,"mergeCommit":{"message":"Update
ftr services to handle occasional WebDriverError (#157283)\n\n##
Summary\r\n\r\nStarting with Chrome v113 we noticed that
`_area_chart.ts` suite became\r\nflaky #156821 failing
with\r\n`WebDriverError: unknown error: unhandled inspector
error:\r\n{\"code\":-32000,\"message\":\"No node with given id
found\"}`\r\nUpdating chromedriver to v113 did not solve the issue and
more tests\r\nstarted to fail with the same error.\r\n\r\nIt happens
occasionally when driver returns unhandled error instead
of\r\nStaleElementReferenceException. This PR adds the error to
the\r\n`RETRY_ON_ERRORS` list, so that FTR can search for the element
again and\r\nre-try the action on it:\r\n\r\n```\r\n │ debg
getVisibleText:
elementId=29C3E81151C86107290DD8F020524333_element_290\r\n │ debg
Chromedriver issue #4440, WebElementWrapper.getVisibleText:
WebDriverError: unknown error: unhandled inspector error:
{\"code\":-32000,\"message\":\"No node with given id found\"}\r\n │
(Session info: chrome=113.0.5672.63)\r\n │ debg current
ElementID=29C3E81151C86107290DD8F020524333_element_290\r\n │ debg new
ElementID=29C3E81151C86107290DD8F020524333_element_293\r\n │ debg
Searching again for the element 'By(css selector,
[data-test-subj=\"visEditorInterval\"] + .euiFormErrorText)', 2 attempts
left\r\n │ debg getVisibleText:
elementId=29C3E81151C86107290DD8F020524333_element_293\r\n └- ✓ pass
(1.8s)\r\n```\r\n\r\nThere is no need to use `Retry` service,
`WebDriverWrapper.retryCall`\r\nshould handle the issue.\r\n\r\nFlaky
test runner 100x for Vis Editor
config:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2236\r\nFlaky
test runner 100x for Cases
config:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237\r\n\r\nAnd
2 more
100x:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2239\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2237","sha":"7f4ceb1d302cf24411c1b82030b882e87a8616dd"}},{"branch":"8.8","label":"v8.8.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
This commit is contained in:
Dzmitry Lemechko 2023-05-10 22:50:57 +02:00 committed by GitHub
parent f18eeb0f51
commit 23decee29e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 45 additions and 25 deletions

View file

@ -433,8 +433,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
expect(isFieldErrorMessageExists).to.be(false);
});
// FLAKY: https://github.com/elastic/kibana/issues/156821
describe.skip('interval errors', () => {
describe('interval errors', () => {
before(async () => {
// to trigger displaying of error messages
await testSubjects.clickWhenNotDisabledWithoutRetry('visualizeEditorRenderButton');

View file

@ -117,7 +117,6 @@ export class ComboBoxService extends FtrService {
* @param value option text
*/
public async setCustom(comboBoxSelector: string, value: string): Promise<void> {
this.log.debug(`comboBox.setCustom, comboBoxSelector: ${comboBoxSelector}, value: ${value}`);
const comboBoxElement = await this.testSubjects.find(comboBoxSelector);
await this.setFilterValue(comboBoxElement, value);
await this.common.pressEnterKey();

View file

@ -195,7 +195,7 @@ export class FindService extends FtrService {
): Promise<WebElementWrapper[]> {
this.log.debug(`Find.allDescendantDisplayedByTagName('${tagName}')`);
const allElements = await this.wrapAll(
await parentElement._webElement.findElements(By.tagName(tagName))
await parentElement._webElement.findElements(By.css(tagName))
);
return await this.filterElementIsDisplayed(allElements);
}
@ -372,7 +372,7 @@ export class FindService extends FtrService {
return await this.retry.tryForTime(timeout, async () => {
// tslint:disable-next-line:variable-name
const _element = element instanceof WebElementWrapper ? element._webElement : element;
const allButtons = this.wrapAll(await _element.findElements(By.tagName('button')));
const allButtons = this.wrapAll(await _element.findElements(By.css('button')));
const buttonTexts = await Promise.all(
allButtons.map(async (el) => {
return el.getVisibleText();

View file

@ -25,11 +25,12 @@ interface ClearOptions {
withJS: boolean;
}
const RETRY_CLICK_MAX_ATTEMPTS = 3;
const RETRY_CLICK_RETRY_ON_ERRORS = [
const RETRY_MAX_ATTEMPTS = 3;
const RETRY_ON_ERRORS = [
'ElementClickInterceptedError',
'ElementNotInteractableError',
'StaleElementReferenceError',
'WebDriverError',
];
export class WebElementWrapper {
@ -105,25 +106,34 @@ export class WebElementWrapper {
private async retryCall<T>(
fn: (wrapper: this) => T | Promise<T>,
attemptsRemaining: number = RETRY_CLICK_MAX_ATTEMPTS
attemptsRemaining: number = RETRY_MAX_ATTEMPTS
): Promise<T> {
try {
return await fn(this);
} catch (err) {
if (
!RETRY_CLICK_RETRY_ON_ERRORS.includes(err.name) ||
this.locator === null ||
attemptsRemaining === 0
) {
if (!RETRY_ON_ERRORS.includes(err.name) || this.locator === null || attemptsRemaining === 0) {
throw err;
}
this.logger.warning(`WebElementWrapper.${fn.name}: ${err.message}`);
/*
https://bugs.chromium.org/p/chromedriver/issues/detail?id=4440
Starting with v113 driver occasionally returns WebDriverError instead of StaleElementReferenceError
*/
if (err.message.includes('"code":-32000,"message":"No node with given id found"')) {
// WebDriverError: unknown error: unhandled inspector error: {"code":-32000,"message":"No node with given id found"}
this.logger.debug(`Chromedriver issue #4440, WebElementWrapper.${fn.name}: ${err}`);
} else {
this.logger.warning(`WebElementWrapper.${fn.name}: ${err}`);
}
this.logger.debug(
`finding element '${this.locator.toString()}' again, ${attemptsRemaining - 1} attempts left`
`Searching again for the element '${this.locator?.toString()}', ${
attemptsRemaining - 1
} attempts left`
);
await setTimeoutAsync(200);
// WebElement reference id will be updated if element is no longer in DOM (StaleElementReferenceError)
this._webElement = await this.driver.findElement(this.locator);
return await this.retryCall(fn, attemptsRemaining - 1);
}
@ -466,7 +476,6 @@ export class WebElementWrapper {
* @return {Promise<WebElementWrapper>}
*/
public async findByCssSelector(selector: string) {
this.logger.debug(`WebElementWrapper.findByCssSelector(${selector})`);
return await this.retryCall(async function findByCssSelector(wrapper) {
return wrapper._wrap(
await wrapper._webElement.findElement(wrapper.By.css(selector)),
@ -576,8 +585,8 @@ export class WebElementWrapper {
public async findByTagName(tagName: string): Promise<WebElementWrapper> {
return await this.retryCall(async function findByTagName(wrapper) {
return wrapper._wrap(
await wrapper._webElement.findElement(wrapper.By.tagName(tagName)),
wrapper.By.tagName(tagName)
await wrapper._webElement.findElement(wrapper.By.css(tagName)),
wrapper.By.css(tagName)
);
});
}
@ -602,7 +611,7 @@ export class WebElementWrapper {
return await this.retryCall(async function findAllByTagName(wrapper) {
return wrapper._wrapAll(
await wrapper._findWithCustomTimeout(
async () => await wrapper._webElement.findElements(wrapper.By.tagName(tagName)),
async () => await wrapper._webElement.findElements(wrapper.By.css(tagName)),
timeout
)
);
@ -687,16 +696,29 @@ export class WebElementWrapper {
* @param {string} className
* @return {Promise<void>}
*/
public async waitForDeletedByCssSelector(
selector: string,
implicitTimeout = 1000
): Promise<void> {
public async waitForDeletedByCssSelector(selector: string, implicitTimeout = 200): Promise<void> {
try {
this.logger.debug(
`waitForDeletedByCssSelector: implicitTimeout=${implicitTimeout}, wait=${this.timeout}`
);
await this.driver.manage().setTimeouts({ implicit: implicitTimeout });
await this.driver.wait(
async () => {
const found = await this._webElement.findElements(this.By.css(selector));
return found.length === 0;
if (this.locator) {
try {
const parentElement = await this.driver.findElement(this.locator);
const found = await parentElement.findElements(this.By.css(selector));
return found.length === 0;
} catch (err) {
this.logger.debug(
`waitForDeletedByCssSelector: parent element is no longer in DOM, wait is done`
);
return true;
}
} else {
const found = await this._webElement.findElements(this.By.css(selector));
return found.length === 0;
}
},
this.timeout,
`The element with ${selector} selector was still present after ${this.timeout} sec.`