mirror of
https://github.com/elastic/kibana.git
synced 2025-04-23 09:19:04 -04:00
[FTR] use xpath locator to find particular element in group by index (#161202)
Related to #158972
I was able to reproduce the failure locally on the latest main with a
small code change:
b3d7b71076/test/functional/page_objects/visual_builder_page.ts (L867-L869)
```
| debg comboBox.isOptionSelected, value: machine.os.raw
│ info Taking screenshot "/Users/dmle/github/kibana/test/functional/screenshots/failure/visualize app visual builder Time Series basics Clicking on the chart should cre-a71516dab48cdb296c45e2b439ed3965cfd400204827bba7ce3cf4719afb093b.png"
│ info Current URL is: http://localhost:5620/app/visualize#/create?type=metrics&_g=(filters:!(),refreshInterval:(pause:!t,value:60000),time:(from:%272015-09-19T06:31:44.000Z%27,to:%272015-09-22T18:31:44.000Z%27))&_a=(filters:!(),linked:!f,query:(language:kuery,query:%27%27),uiState:(),vis:(aggs:!(),params:(axis_formatter:number,axis_position:left,axis_scale:normal,drop_last_bucket:1,id:b6ee5181-addb-436f-8190-4f2a178fee7b,index_pattern:(id:%27logstash-*%27),interval:%27%27,isModelInvalid:!f,max_lines_legend:1,series:!((axis_position:right,chart_type:line,color:%2368BC00,fill:0.5,formatter:default,id:%2724f639d7-d16e-44d1-ac0d-f2b590e94e8e%27,line_width:1,metrics:!((id:d4f68ace-b844-4e00-933f-8c3b49e19067,type:count)),override_index_pattern:0,palette:(name:default,type:palette),point_size:1,separate_axis:0,series_drop_last_bucket:0,split_mode:terms,stacked:none,terms_field:!(bytes,machine.os.raw),time_range_mode:entire_time_range)),show_grid:1,show_legend:1,time_field:%27%27,time_range_mode:entire_time_range,tooltip_mode:show_all,truncate_legend:1,type:timeseries,use_kibana_indexes:!t),title:%27%27,type:metrics))
│ info Saving page source to: /Users/dmle/github/kibana/test/functional/failure_debug/html/visualize app visual builder Time Series basics Clicking on the chart should cre-a71516dab48cdb296c45e2b439ed3965cfd400204827bba7ce3cf4719afb093b.html
└- ✖ fail: visualize app visual builder Time Series basics Clicking on the chart should create a filter for series with multiple split by terms fields one of which has formatting
│ StaleElementReferenceError: stale element reference: stale element not found
│ (Session info: chrome=114.0.5735.198)
│ at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:524:15)
│ at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:601:13)
│ at Executor.execute (node_modules/selenium-webdriver/lib/http.js:529:28)
│ at runMicrotasks (<anonymous>)
│ at processTicksAndRejections (node:internal/process/task_queues:96:5)
│ at Task.exec (prevent_parallel_calls.ts:28:20)
```
### Do we have any service to help with StaleElementReferenceError
handling in FTR?
We do. `WebElementWrapper` object have internal mechanism to handle it:
we wrap most of the actions (click, type, getAttribute, etc.) with
`retryCall` function, which execute command up to RETRY_MAX_ATTEMPTS
times (3 by default) and check for errors. So if we try to click element
that is no longer in the DOM, this mechanism will try to find the
element in the DOM again, assuming it is still there but referenceId was
changed due to page updates. And repeat the click.
https://github.com/elastic/kibana/blob/main/test/functional/services/lib/web_element_wrapper/web_element_wrapper.ts#L107-L140
You might notice a warning during timePicker dates selection
```
│ debg Find.findByCssSelector('[data-test-subj="superDatePickerAbsoluteTab"]') with timeout=10000
│ warn WebElementWrapper.click: StaleElementReferenceError: stale element reference: stale element not found
│ (Session info: chrome=114.0.5735.198)
│ debg Searching again for the element 'By(css selector, [data-test-subj="superDatePickerAbsoluteTab"])',
| 2 attempts left
```
It is helping a lot to minimize the flakiness.
### Why does FTR still fails with StaleElementReferenceError in
Visualize tests?
Because most methods use the same pattern for searching elements in the
DOM: get all elements and then pick one by index.
```
const byFields = await this.testSubjects.findAll('fieldSelectItem');
const selectedByField = byFields[byFields.length - 1];
await this.comboBox.setElement(selectedByField, field);
```
The problem is that WebElementWrapper retry mechanism relies on having
`locator` property defined and it is _only_ defined if you search for a
single element, e.g. `testSubjects.find` or `by.byCssSelector`. This
property is set to `null` for each `WebElementWrapper` object in array
you get from `await this.testSubjects.findAll('fieldSelectItem')`;
So when we pass this object in `this.comboBox.setElement` and go through
a list of actions, it will fail on the first StaleElementReferenceError
occurrence.
The devil is in the detail: usually searching for multiple elements and
doing actions on some of it is totally fine. But comboBox selection has
quite many actions with different child elements, that can be updated in
DOM and lead to the error. Wrapping things with classical `retry`
service only hides the issue, but does not actually solve it.
### Proposed solution
Afaik CSS locators does not support searching for elements and picking
up one by index (I looked at `:nth-child()
`, but it is not what we need).
But we can use xpath locators for this purpose.
```
const selectedByField = await this.find.byXPath(`(//*[@data-test-subj='fieldSelectItem'])[last()]`);
await this.comboBox.setElement(selectedByField, field);
```
This way `selectedByField` has locator property defined and internal
mechanism will be able to retry actions.
#### Why don't we store `locator` for multiple elements search.
We can, but it means every element in array has the same locator and
while retrying to find it WebDriver will return the first one in the DOM
ignoring the element index. Using xpath locator is basically the simpler
version of storing element index in WebElementWrapper object.
Flaky test runner: 50x for visualize/group1..5/config.ts
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2554
100x for group5
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2596#0189403c-ccd2-4186-a83e-af21fe88018c
100x flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2794
This commit is contained in:
Dzmitry Lemechko
GitHub
parent
02e56cf139
commit
8e1d66fab6
4 changed files with 83 additions and 69 deletions
|
|
@ -44,6 +44,7 @@ export class CalculationVars extends Component {
|
|||
<EuiFlexItem>
|
||||
<EuiFieldText
|
||||
className="tvbAggs__varName"
|
||||
data-test-subj="tvbAggsVarNameInput"
|
||||
aria-label={i18n.translate('visTypeTimeseries.vars.variableNameAriaLabel', {
|
||||
defaultMessage: 'Variable name',
|
||||
})}
|
||||
|
|
@ -54,7 +55,10 @@ export class CalculationVars extends Component {
|
|||
value={row.name}
|
||||
/>
|
||||
</EuiFlexItem>
|
||||
<EuiFlexItem className="tvbAggs__varMetricWrapper">
|
||||
<EuiFlexItem
|
||||
className="tvbAggs__varMetricWrapper"
|
||||
data-test-subj="tvbAggsVarMetricWrapper"
|
||||
>
|
||||
<MetricSelect
|
||||
onChange={this.handleChange(row, 'field')}
|
||||
metrics={this.props.metrics}
|
||||
|
|
|
|||
|
|
@ -11,16 +11,9 @@ import expect from '@kbn/expect';
|
|||
import { FtrProviderContext } from '../../../ftr_provider_context';
|
||||
|
||||
export default function ({ getPageObjects, getService }: FtrProviderContext) {
|
||||
const { visualize, visualBuilder, timeToVisualize, dashboard, header, common, visChart } =
|
||||
getPageObjects([
|
||||
'visualBuilder',
|
||||
'visualize',
|
||||
'timeToVisualize',
|
||||
'dashboard',
|
||||
'header',
|
||||
'common',
|
||||
'visChart',
|
||||
]);
|
||||
const { visualize, visualBuilder, timeToVisualize, dashboard, common, visChart } = getPageObjects(
|
||||
['visualBuilder', 'visualize', 'timeToVisualize', 'dashboard', 'header', 'common', 'visChart']
|
||||
);
|
||||
const security = getService('security');
|
||||
const testSubjects = getService('testSubjects');
|
||||
const retry = getService('retry');
|
||||
|
|
@ -225,14 +218,11 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
|
|||
const expectedFilterPills = ['0, win 7'];
|
||||
await visualBuilder.setMetricsGroupByTerms('bytes');
|
||||
await visChart.waitForVisualizationRenderingStabilized();
|
||||
await header.waitUntilLoadingHasFinished();
|
||||
await visualBuilder.setAnotherGroupByTermsField('machine.os.raw');
|
||||
await visChart.waitForVisualizationRenderingStabilized();
|
||||
await header.waitUntilLoadingHasFinished();
|
||||
await visualBuilder.clickSeriesOption();
|
||||
await visualBuilder.setChartType('Bar');
|
||||
await visChart.waitForVisualizationRenderingStabilized();
|
||||
await header.waitUntilLoadingHasFinished();
|
||||
await visualBuilder.clickPanelOptions('timeSeries');
|
||||
await visualBuilder.setIntervalValue('1w');
|
||||
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@
|
|||
*/
|
||||
|
||||
import type { DebugState } from '@elastic/charts';
|
||||
import expect from '@kbn/expect';
|
||||
import { FtrService } from '../ftr_provider_context';
|
||||
import { WebElementWrapper } from '../services/lib/web_element_wrapper';
|
||||
|
||||
|
|
@ -260,8 +261,10 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async clickSeriesOption(nth = 0) {
|
||||
const el = await this.testSubjects.findAll('seriesOptions');
|
||||
await el[nth].click();
|
||||
const button = await this.find.byXPath(
|
||||
`(//button[@data-test-subj='seriesOptions'])[${nth + 1}]`
|
||||
);
|
||||
await button.click();
|
||||
}
|
||||
|
||||
public async clearOffsetSeries() {
|
||||
|
|
@ -325,8 +328,7 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
});
|
||||
}
|
||||
if (decimalPlaces) {
|
||||
const decimalPlacesInput = await this.testSubjects.find('dataFormatPickerDurationDecimal');
|
||||
await decimalPlacesInput.type(decimalPlaces);
|
||||
await this.testSubjects.setValue('dataFormatPickerDurationDecimal', decimalPlaces);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -437,22 +439,30 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async selectAggType(value: string, nth = 0) {
|
||||
const elements = await this.testSubjects.findAll('aggSelector');
|
||||
await this.comboBox.setElement(elements[nth], value);
|
||||
const element = await this.find.byXPath(`(//div[@data-test-subj='aggSelector'])[${nth + 1}]`);
|
||||
await this.comboBox.setElement(element, value);
|
||||
return await this.header.waitUntilLoadingHasFinished();
|
||||
}
|
||||
|
||||
public async fillInExpression(expression: string, nth = 0) {
|
||||
const expressions = await this.testSubjects.findAll('mathExpression');
|
||||
await expressions[nth].type(expression);
|
||||
const element = await this.find.byXPath(
|
||||
`(//textarea[@data-test-subj='mathExpression'])[${nth + 1}]`
|
||||
);
|
||||
await element.type(expression);
|
||||
return await this.header.waitUntilLoadingHasFinished();
|
||||
}
|
||||
|
||||
public async fillInVariable(name = 'test', metric = 'Count', nth = 0) {
|
||||
const elements = await this.testSubjects.findAll('varRow');
|
||||
const varNameInput = await elements[nth].findByCssSelector('.tvbAggs__varName');
|
||||
const varNameInput = await this.find.byXPath(
|
||||
`(//div[@data-test-subj="varRow"])[${nth + 1}]//input[@data-test-subj='tvbAggsVarNameInput']`
|
||||
);
|
||||
await varNameInput.type(name);
|
||||
const metricSelectWrapper = await elements[nth].findByCssSelector('.tvbAggs__varMetricWrapper');
|
||||
const metricSelectWrapper = await this.find.byXPath(
|
||||
`(//div[@data-test-subj="varRow"])[${
|
||||
nth + 1
|
||||
}]//div[@data-test-subj='tvbAggsVarMetricWrapper']`
|
||||
);
|
||||
|
||||
await this.comboBox.setElement(metricSelectWrapper, metric);
|
||||
return await this.header.waitUntilLoadingHasFinished();
|
||||
}
|
||||
|
|
@ -510,19 +520,16 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async setAnnotationFilter(query: string) {
|
||||
const annotationQueryBar = await this.testSubjects.find('annotationQueryBar');
|
||||
await annotationQueryBar.type(query);
|
||||
await this.testSubjects.setValue('annotationQueryBar', query);
|
||||
await this.header.waitUntilLoadingHasFinished();
|
||||
}
|
||||
|
||||
public async setAnnotationFields(fields: string) {
|
||||
const annotationFieldsInput = await this.testSubjects.find('annotationFieldsInput');
|
||||
await annotationFieldsInput.type(fields);
|
||||
await this.testSubjects.setValue('annotationFieldsInput', fields);
|
||||
}
|
||||
|
||||
public async setAnnotationRowTemplate(template: string) {
|
||||
const annotationRowTemplateInput = await this.testSubjects.find('annotationRowTemplateInput');
|
||||
await annotationRowTemplateInput.type(template);
|
||||
await this.testSubjects.setValue('annotationRowTemplateInput', template);
|
||||
}
|
||||
|
||||
public async toggleIndexPatternSelectionModePopover(shouldOpen: boolean) {
|
||||
|
|
@ -644,7 +651,7 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async setStaticValue(value: number, nth: number = 0): Promise<void> {
|
||||
const input = (await this.testSubjects.findAll('staticValue'))[nth];
|
||||
const input = await this.find.byXPath(`(//input[@data-test-subj='staticValue'])[${nth + 1}]`);
|
||||
await input.type(value.toString());
|
||||
}
|
||||
|
||||
|
|
@ -691,16 +698,17 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async getFieldForAggregation(aggNth: number = 0): Promise<WebElementWrapper> {
|
||||
const labels = await this.testSubjects.findAll('aggRow');
|
||||
const label = labels[aggNth];
|
||||
|
||||
return (await label.findAllByTestSubject('comboBoxInput'))[1];
|
||||
// Aggregation has 2 comboBox elements: Aggregation Type and Field
|
||||
// Locator picks the aggregation by index (aggNth) and its Field comboBox child by index (2)
|
||||
return await this.find.byXPath(
|
||||
`((//div[@data-test-subj='aggRow'])[${aggNth + 1}]//div[@data-test-subj='comboBoxInput'])[2]`
|
||||
);
|
||||
}
|
||||
|
||||
public async clickColorPicker(nth: number = 0): Promise<void> {
|
||||
const picker = (await this.find.allByCssSelector('[data-test-subj="tvbColorPicker"] button'))[
|
||||
nth
|
||||
];
|
||||
const picker = await this.find.byXPath(
|
||||
`(//button[@data-test-subj='euiColorPickerAnchor'])[${nth + 1}]`
|
||||
);
|
||||
await picker.clickMouseButton();
|
||||
}
|
||||
|
||||
|
|
@ -838,22 +846,27 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
await this.setMetricsGroupBy('terms');
|
||||
await this.common.sleep(1000);
|
||||
const byField = await this.testSubjects.find('groupByField');
|
||||
await this.retry.try(async () => {
|
||||
await this.comboBox.setElement(byField, field);
|
||||
});
|
||||
|
||||
await this.comboBox.setElement(byField, field);
|
||||
const isSelected = await this.comboBox.isOptionSelected(byField, field);
|
||||
expect(isSelected).to.be(true);
|
||||
await this.setMetricsGroupByFiltering(filtering.include, filtering.exclude);
|
||||
}
|
||||
|
||||
public async setAnotherGroupByTermsField(field: string) {
|
||||
const fieldSelectAddButtons = await this.testSubjects.findAll('fieldSelectItemAddBtn');
|
||||
await fieldSelectAddButtons[fieldSelectAddButtons.length - 1].click();
|
||||
// Using xpath locator to find the last element
|
||||
const fieldSelectAddButtonLast = await this.find.byXPath(
|
||||
`(//*[@data-test-subj='fieldSelectItemAddBtn'])[last()]`
|
||||
);
|
||||
// In case of StaleElementReferenceError 'browser' service will try to find element again
|
||||
await fieldSelectAddButtonLast.click();
|
||||
await this.common.sleep(2000);
|
||||
const byFields = await this.testSubjects.findAll('fieldSelectItem');
|
||||
const selectedByField = byFields[byFields.length - 1];
|
||||
await this.retry.try(async () => {
|
||||
await this.comboBox.setElement(selectedByField, field);
|
||||
});
|
||||
const selectedByField = await this.find.byXPath(
|
||||
`(//*[@data-test-subj='fieldSelectItem'])[last()]`
|
||||
);
|
||||
|
||||
await this.comboBox.setElement(selectedByField, field);
|
||||
const isSelected = await this.comboBox.isOptionSelected(selectedByField, field);
|
||||
expect(isSelected).to.be(true);
|
||||
}
|
||||
|
||||
public async setMetricsGroupByFiltering(include?: string, exclude?: string) {
|
||||
|
|
@ -881,34 +894,40 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async setGroupByFilterQuery(query: string, nth: number = 0) {
|
||||
const filterQueryInput = await this.testSubjects.findAll('filterItemsQueryBar');
|
||||
await filterQueryInput[nth].type(query);
|
||||
const filterQueryInput = await this.find.byXPath(
|
||||
`(//textarea[@data-test-subj='filterItemsQueryBar'])[${nth + 1}]`
|
||||
);
|
||||
await filterQueryInput.type(query);
|
||||
}
|
||||
|
||||
public async setGroupByFilterLabel(label: string, nth: number = 0) {
|
||||
const filterLabelInput = await this.testSubjects.findAll('filterItemsLabel');
|
||||
await filterLabelInput[nth].type(label);
|
||||
const filterLabelInput = await this.find.byXPath(
|
||||
`(//input[@data-test-subj='filterItemsLabel'])[${nth + 1}]`
|
||||
);
|
||||
await filterLabelInput.type(label);
|
||||
}
|
||||
|
||||
public async setChartType(type: 'Bar' | 'Line', nth: number = 0) {
|
||||
const seriesChartTypeComboBoxes = await this.testSubjects.findAll('seriesChartTypeComboBox');
|
||||
return await this.comboBox.setElement(seriesChartTypeComboBoxes[nth], type);
|
||||
const seriesChartTypeComboBox = await this.find.byXPath(
|
||||
`(//div[@data-test-subj='seriesChartTypeComboBox'])[${nth + 1}]`
|
||||
);
|
||||
return await this.comboBox.setElement(seriesChartTypeComboBox, type);
|
||||
}
|
||||
|
||||
public async setStackedType(stackedType: string, nth: number = 0) {
|
||||
const seriesChartTypeComboBoxes = await this.testSubjects.findAll('seriesStackedComboBox');
|
||||
return await this.comboBox.setElement(seriesChartTypeComboBoxes[nth], stackedType);
|
||||
const seriesStackedComboBox = await this.find.byXPath(
|
||||
`(//div[@data-test-subj='seriesStackedComboBox'])[${nth + 1}]`
|
||||
);
|
||||
return await this.comboBox.setElement(seriesStackedComboBox, stackedType);
|
||||
}
|
||||
|
||||
public async setSeriesFilter(query: string) {
|
||||
const seriesFilterQueryInput = await this.testSubjects.find('seriesConfigQueryBar');
|
||||
await seriesFilterQueryInput.type(query);
|
||||
await this.testSubjects.setValue('seriesConfigQueryBar', query);
|
||||
await this.header.waitUntilLoadingHasFinished();
|
||||
}
|
||||
|
||||
public async setPanelFilter(query: string) {
|
||||
const panelFilterQueryInput = await this.testSubjects.find('panelFilterQueryBar');
|
||||
await panelFilterQueryInput.type(query);
|
||||
await this.testSubjects.setValue('panelFilterQueryBar', query);
|
||||
await this.header.waitUntilLoadingHasFinished();
|
||||
}
|
||||
|
||||
|
|
@ -938,8 +957,7 @@ export class VisualBuilderPageObject extends FtrService {
|
|||
}
|
||||
|
||||
public async setFilterRatioOption(optionType: 'Numerator' | 'Denominator', query: string) {
|
||||
const optionInput = await this.testSubjects.find(`filterRatio${optionType}Input`);
|
||||
await optionInput.type(query);
|
||||
await this.testSubjects.setValue(`filterRatio${optionType}Input`, query);
|
||||
}
|
||||
|
||||
public async clickSeriesLegendItem(name: string) {
|
||||
|
|
|
|||
|
|
@ -38,8 +38,9 @@ export class ComboBoxService extends FtrService {
|
|||
|
||||
public async setForLastInput(comboBoxSelector: string, value: string): Promise<void> {
|
||||
this.log.debug(`comboBox.set, comboBoxSelector: ${comboBoxSelector}`);
|
||||
const comboBoxes = await this.testSubjects.findAll(comboBoxSelector);
|
||||
const comboBox = comboBoxes[comboBoxes.length - 1];
|
||||
const comboBox = await this.find.byXPath(
|
||||
`(//*[@data-test-subj='${comboBoxSelector}'])[last()]`
|
||||
);
|
||||
await this.setElement(comboBox, value);
|
||||
}
|
||||
|
||||
|
|
@ -354,9 +355,10 @@ export class ComboBoxService extends FtrService {
|
|||
|
||||
public async clearLastInputField(comboBoxSelector: string): Promise<void> {
|
||||
this.log.debug(`comboBox.clearInputField, comboBoxSelector:${comboBoxSelector}`);
|
||||
const comboBoxElements = await this.testSubjects.findAll(comboBoxSelector);
|
||||
const comboBoxElement = comboBoxElements[comboBoxElements.length - 1];
|
||||
const input = await comboBoxElement.findByTagName('input');
|
||||
const comboBox = await this.find.byXPath(
|
||||
`(//*[@data-test-subj='${comboBoxSelector}'])[last()]`
|
||||
);
|
||||
const input = await comboBox.findByTagName('input');
|
||||
await input.clearValueWithKeyboard();
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue