github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-23 17:28:26 -04:00
Code Issues Projects Releases Packages Wiki Activity

[8.16] [Security Solution] Fix timeline dynamic batching (#204034) | [ Security Solution ] Fix Refetch logic with new timeline batching (#205893) (#205674)

Browse source 
# Backport

This will backport the following commits from `main` to `8.16`:
- [[Security Solution] Fix timeline dynamic batching
(#204034)](https://github.com/elastic/kibana/pull/204034)
 - https://github.com/elastic/kibana/pull/205893

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Jatin
Kathuria","email":"jatin.kathuria@elastic.co"},"sourceCommit":{"committedDate":"2025-01-07T06:20:30Z","message":"[Security
Solution] Fix timeline dynamic batching (#204034)\n\n##
Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n-
https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug
where if users fetched multiple batches and then if\r\nuser adds a new
column, the value of this new columns will only be\r\nfetched for the
latest batch and not old batches.\r\n- This PR fixes that ✅ by
cumulatively fetching the data for old batches\r\ntill current batch
`iff a new column has been added`.\r\n- For example, if user has already
fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched
together when a column has been added,\r\notherwise, data will be
fetched incrementally.\r\n\r\n### Issue with Elastic search
limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we
throw error at\r\n10K which should be allowed.\r\n - Error should be
thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that
just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete
code\r\n\r\nBelow files related to old Timeline code are removed as
well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by:
Philippe Oberti
<philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","Team:Threat
Hunting:Investigations","backport:prev-minor","v8.16.3"],"number":204034,"url":"https://github.com/elastic/kibana/pull/204034","mergeCommit":{"message":"[Security
Solution] Fix timeline dynamic batching (#204034)\n\n##
Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n-
https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug
where if users fetched multiple batches and then if\r\nuser adds a new
column, the value of this new columns will only be\r\nfetched for the
latest batch and not old batches.\r\n- This PR fixes that ✅ by
cumulatively fetching the data for old batches\r\ntill current batch
`iff a new column has been added`.\r\n- For example, if user has already
fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched
together when a column has been added,\r\notherwise, data will be
fetched incrementally.\r\n\r\n### Issue with Elastic search
limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we
throw error at\r\n10K which should be allowed.\r\n - Error should be
thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that
just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete
code\r\n\r\nBelow files related to old Timeline code are removed as
well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by:
Philippe Oberti
<philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},"sourceBranch":"main","suggestedTargetBranches":["8.16"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204034","number":204034,"mergeCommit":{"message":"[Security
Solution] Fix timeline dynamic batching (#204034)\n\n##
Summary\r\n\r\nHandles :\r\n\r\n\r\n### Issue with Batches\r\n-
https://github.com/elastic/kibana/issues/201405\r\n- Timeline had a bug
where if users fetched multiple batches and then if\r\nuser adds a new
column, the value of this new columns will only be\r\nfetched for the
latest batch and not old batches.\r\n- This PR fixes that ✅ by
cumulatively fetching the data for old batches\r\ntill current batch
`iff a new column has been added`.\r\n- For example, if user has already
fetched the 3rd batch, data for\r\n1st,2nd and 3rd will be fetched
together when a column has been added,\r\notherwise, data will be
fetched incrementally.\r\n\r\n### Issue with Elastic search
limit\r\n\r\n- Elastic search has a limit of 10K hits at max but we
throw error at\r\n10K which should be allowed.\r\n - Error should be
thrown at anything `>10K`. 10001 for example.\r\n - ✅ This PR fixes that
just for timeline by allowing 10K hits.\r\n\r\n### Removal of obsolete
code\r\n\r\nBelow files related to old Timeline code are removed as
well:\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx\r\n-\r\nx-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx\r\n\r\n---------\r\n\r\nCo-authored-by:
Philippe Oberti
<philippe.oberti@elastic.co>","sha":"088169f446788f9fa8800d77817881524514943e"}},{"branch":"8.16","label":"v8.16.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
This commit is contained in:
Jatin Kathuria 2025-02-06 18:08:47 +01:00 • committed by GitHub
parent 87e8b1406d
commit 476335ab08
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 1003 additions and 1145 deletions
Download patch file Download diff file Expand all files Collapse all files

4
x-pack/plugins/security_solution/common/types/timeline/store.ts
View file

@ -72,8 +72,8 @@ export type OnColumnRemoved = (columnId: ColumnId) => void;
export type OnColumnResized = ({ columnId, delta }: { columnId: ColumnId; delta: number }) => void;
/** Invoked when a user clicks to load more item */
export type OnFetchMoreRecords = (nextPage: number) => void;
/** Invoked when a user clicks to load next batch */
export type OnFetchMoreRecords = VoidFunction;
/** Invoked when a user checks/un-checks a row */
export type OnRowSelected = ({

51
x-pack/plugins/security_solution/public/common/mock/mock_timeline_search_service.ts Normal file
View file

@ -0,0 +1,51 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { mockTimelineData } from './mock_timeline_data';
const mockEvents = structuredClone(mockTimelineData);
/*
* This helps to mock `data.search.search` method to mock the timeline data
* */
export const getMockTimelineSearchSubscription = () => {
const mockSearchWithArgs = jest.fn();
const mockTimelineSearchSubscription = jest.fn().mockImplementation((args) => {
mockSearchWithArgs(args);
return {
subscribe: jest.fn().mockImplementation(({ next }) => {
const start = args.pagination.activePage * args.pagination.querySize;
const end = start + args.pagination.querySize;
const timelineOut = setTimeout(() => {
next({
isRunning: false,
isPartial: false,
inspect: {
dsl: [],
response: [],
},
edges: mockEvents.map((item) => ({ node: item })).slice(start, end),
pageInfo: {
activePage: args.pagination.activePage,
querySize: args.pagination.querySize,
},
rawResponse: {},
totalCount: mockEvents.length,
});
}, 50);
return {
unsubscribe: jest.fn(() => {
clearTimeout(timelineOut);
}),
};
}),
};
});
return { mockTimelineSearchSubscription, mockSearchWithArgs };
};

236
x-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.test.tsx
View file

@ -1,236 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { render, screen, fireEvent } from '@testing-library/react';
import React from 'react';
import { TestProviders } from '../../../../common/mock/test_providers';
import { FooterComponent, PagingControlComponent } from '.';
import { TimelineId } from '../../../../../common/types/timeline';
jest.mock('../../../../common/lib/kibana');
describe('Footer Timeline Component', () => {
const loadMore = jest.fn();
const updatedAt = 1546878704036;
const serverSideEventCount = 15546;
const itemsCount = 2;
describe('rendering', () => {
it('shoult render the default timeline footer', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={false}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
expect(screen.getByTestId('timeline-footer')).toBeInTheDocument();
});
it('should render the loading panel at the beginning ', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={true}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
expect(screen.getByTestId('LoadingPanelTimeline')).toBeInTheDocument();
});
it('should render the loadMore button if it needs to fetch more', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={false}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
expect(screen.getByTestId('timeline-pagination')).toBeInTheDocument();
});
it('should render `Loading...` when fetching new data', () => {
render(
<PagingControlComponent
activePage={0}
totalCount={30}
totalPages={3}
onPageClick={loadMore}
isLoading={true}
/>
);
expect(screen.queryByTestId('LoadingPanelTimeline')).not.toBeInTheDocument();
expect(screen.getByText('Loading...')).toBeInTheDocument();
});
it('should render the Pagination in the more load button when fetching new data', () => {
render(
<PagingControlComponent
activePage={0}
totalCount={30}
totalPages={3}
onPageClick={loadMore}
isLoading={false}
/>
);
expect(screen.getByTestId('timeline-pagination')).toBeInTheDocument();
});
it('should NOT render the loadMore button because there is nothing else to fetch', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={true}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
expect(screen.queryByTestId('timeline-pagination')).not.toBeInTheDocument();
});
it('should render the popover to select new itemsPerPage in timeline', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={false}
itemsCount={itemsCount}
itemsPerPage={1}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
fireEvent.click(screen.getByTestId('local-events-count-button'));
expect(screen.getByTestId('timelinePickSizeRow')).toBeInTheDocument();
});
});
describe('Events', () => {
it('should call loadmore when clicking on the button load more', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={false}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
fireEvent.click(screen.getByTestId('pagination-button-next'));
expect(loadMore).toBeCalled();
});
it('should render the auto-refresh message instead of load more button when stream live is on', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={true}
isLoading={false}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
expect(screen.queryByTestId('timeline-pagination')).not.toBeInTheDocument();
expect(screen.getByTestId('is-live-on-message')).toBeInTheDocument();
});
it('should render the load more button when stream live is off', () => {
render(
<TestProviders>
<FooterComponent
activePage={0}
updatedAt={updatedAt}
height={100}
id={TimelineId.test}
isLive={false}
isLoading={false}
itemsCount={itemsCount}
itemsPerPage={2}
itemsPerPageOptions={[1, 5, 10, 20]}
onChangePage={loadMore}
totalCount={serverSideEventCount}
/>
</TestProviders>
);
expect(screen.getByTestId('timeline-pagination')).toBeInTheDocument();
expect(screen.queryByTestId('is-live-on-message')).not.toBeInTheDocument();
});
});
});

380
x-pack/plugins/security_solution/public/timelines/components/timeline/footer/index.tsx
View file

@ -1,380 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import {
EuiBadge,
EuiButtonEmpty,
EuiContextMenuItem,
EuiContextMenuPanel,
EuiFlexGroup,
EuiFlexItem,
EuiIconTip,
EuiPopover,
EuiText,
EuiToolTip,
EuiPagination,
} from '@elastic/eui';
import { FormattedMessage } from '@kbn/i18n-react';
import React, { useCallback, useEffect, useState, useMemo } from 'react';
import styled from 'styled-components';
import { useDispatch } from 'react-redux';
import type { OnChangePage } from '../events';
import { EVENTS_COUNT_BUTTON_CLASS_NAME } from '../helpers';
import * as i18n from './translations';
import { timelineActions, timelineSelectors } from '../../../store';
import { useDeepEqualSelector } from '../../../../common/hooks/use_selector';
import { useKibana } from '../../../../common/lib/kibana';
import { LastUpdatedContainer } from './last_updated';
interface HeightProp {
height: number;
}
const FooterContainer = styled(EuiFlexGroup).attrs<HeightProp>(({ height }) => ({
style: {
height: `${height}px`,
},
}))<HeightProp>`
flex: 0 0 auto;
`;
FooterContainer.displayName = 'FooterContainer';
const FooterFlexGroup = styled(EuiFlexGroup)`
height: 35px;
width: 100%;
`;
FooterFlexGroup.displayName = 'FooterFlexGroup';
const LoadingPanelContainer = styled.div`
padding-top: 3px;
`;
LoadingPanelContainer.displayName = 'LoadingPanelContainer';
export const ServerSideEventCount = styled.div`
margin: 0 5px 0 5px;
`;
ServerSideEventCount.displayName = 'ServerSideEventCount';
/** The height of the footer, exported for use in height calculations */
export const footerHeight = 40; // px
/** Displays the server-side count of events */
export const EventsCountComponent = ({
closePopover,
documentType,
footerText,
isOpen,
items,
itemsCount,
onClick,
serverSideEventCount,
}: {
closePopover: () => void;
documentType: string;
isOpen: boolean;
items: React.ReactElement[];
itemsCount: number;
onClick: () => void;
serverSideEventCount: number;
footerText: string | React.ReactNode;
}) => {
const totalCount = useMemo(
() => (serverSideEventCount > 0 ? serverSideEventCount : 0),
[serverSideEventCount]
);
return (
<h5>
<EuiPopover
className="footer-popover"
id="customizablePagination"
data-test-subj="timelineSizeRowPopover"
button={
<>
<EuiBadge data-test-subj="local-events-count" color="hollow">
{itemsCount}
<EuiButtonEmpty
className={EVENTS_COUNT_BUTTON_CLASS_NAME}
flush="both"
size="s"
color="text"
iconType="arrowDown"
iconSide="right"
onClick={onClick}
data-test-subj="local-events-count-button"
/>
</EuiBadge>
{` ${i18n.OF} `}
</>
}
isOpen={isOpen}
closePopover={closePopover}
panelPaddingSize="none"
>
<EuiContextMenuPanel items={items} data-test-subj="timelinePickSizeRow" />
</EuiPopover>
<EuiToolTip
content={
<>
{totalCount} {footerText}
</>
}
>
<ServerSideEventCount>
<EuiBadge color="hollow" data-test-subj="server-side-event-count">
{totalCount}
</EuiBadge>{' '}
{documentType}
</ServerSideEventCount>
</EuiToolTip>
</h5>
);
};
EventsCountComponent.displayName = 'EventsCountComponent';
export const EventsCount = React.memo(EventsCountComponent);
EventsCount.displayName = 'EventsCount';
interface PagingControlProps {
activePage: number;
isLoading: boolean;
onPageClick: OnChangePage;
totalCount: number;
totalPages: number;
}
const TimelinePaginationContainer = styled.div<{ hideLastPage: boolean }>`
ul.euiPagination__list {
li.euiPagination__item:last-child {
${({ hideLastPage }) => `${hideLastPage ? 'display:none' : ''}`};
}
}
`;
export const PagingControlComponent: React.FC<PagingControlProps> = ({
activePage,
isLoading,
onPageClick,
totalCount,
totalPages,
}) => {
if (isLoading) {
return <>{`${i18n.LOADING}...`}</>;
}
if (!totalPages) {
return null;
}
return (
<TimelinePaginationContainer hideLastPage={totalCount > 9999}>
<EuiPagination
data-test-subj="timeline-pagination"
pageCount={totalPages}
activePage={activePage}
onPageClick={onPageClick}
/>
</TimelinePaginationContainer>
);
};
PagingControlComponent.displayName = 'PagingControlComponent';
export const PagingControl = React.memo(PagingControlComponent);
PagingControl.displayName = 'PagingControl';
interface FooterProps {
updatedAt: number;
activePage: number;
height: number;
id: string;
isLive: boolean;
isLoading: boolean;
itemsCount: number;
itemsPerPage: number;
itemsPerPageOptions: number[];
onChangePage: OnChangePage;
totalCount: number;
}
/** Renders a loading indicator and paging controls */
export const FooterComponent = ({
activePage,
updatedAt,
height,
id,
isLive,
isLoading,
itemsCount,
itemsPerPage,
itemsPerPageOptions,
onChangePage,
totalCount,
}: FooterProps) => {
const dispatch = useDispatch();
const { timelines } = useKibana().services;
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const [paginationLoading, setPaginationLoading] = useState(false);
const getManageTimeline = useMemo(() => timelineSelectors.getTimelineByIdSelector(), []);
const {
documentType = i18n.TOTAL_COUNT_OF_EVENTS,
loadingText = i18n.LOADING_EVENTS,
footerText = i18n.TOTAL_COUNT_OF_EVENTS,
} = useDeepEqualSelector((state) => getManageTimeline(state, id));
const handleChangePageClick = useCallback(
(nextPage: number) => {
setPaginationLoading(true);
onChangePage(nextPage);
},
[onChangePage]
);
const onButtonClick = useCallback(
() => setIsPopoverOpen(!isPopoverOpen),
[isPopoverOpen, setIsPopoverOpen]
);
const closePopover = useCallback(() => setIsPopoverOpen(false), [setIsPopoverOpen]);
const onChangeItemsPerPage = useCallback(
(itemsChangedPerPage: number) =>
dispatch(timelineActions.updateItemsPerPage({ id, itemsPerPage: itemsChangedPerPage })),
[dispatch, id]
);
const rowItems = useMemo(
() =>
itemsPerPageOptions &&
itemsPerPageOptions.map((item) => (
<EuiContextMenuItem
key={item}
icon={itemsPerPage === item ? 'check' : 'empty'}
data-test-subj={`items-per-page-option-${item}`}
onClick={() => {
closePopover();
onChangeItemsPerPage(item);
}}
>
{`${item} ${i18n.ROWS}`}
</EuiContextMenuItem>
)),
[closePopover, itemsPerPage, itemsPerPageOptions, onChangeItemsPerPage]
);
const totalPages = useMemo(
() => Math.ceil(totalCount / itemsPerPage),
[itemsPerPage, totalCount]
);
useEffect(() => {
if (paginationLoading && !isLoading) {
setPaginationLoading(false);
}
}, [isLoading, paginationLoading]);
if (isLoading && !paginationLoading) {
return (
<LoadingPanelContainer>
{timelines.getLoadingPanel({
dataTestSubj: 'LoadingPanelTimeline',
height: '35px',
showBorder: false,
text: loadingText,
width: '100%',
})}
</LoadingPanelContainer>
);
}
return (
<FooterContainer
data-test-subj="timeline-footer"
direction="column"
gutterSize="none"
height={height}
justifyContent="spaceAround"
>
<FooterFlexGroup
alignItems="center"
data-test-subj="footer-flex-group"
direction="row"
gutterSize="none"
justifyContent="spaceBetween"
>
<EuiFlexItem data-test-subj="event-count-container" grow={false}>
<EuiFlexGroup
alignItems="center"
data-test-subj="events-count"
direction="row"
gutterSize="none"
>
<EventsCount
closePopover={closePopover}
documentType={documentType}
footerText={footerText}
isOpen={isPopoverOpen}
items={rowItems}
itemsCount={itemsCount}
onClick={onButtonClick}
serverSideEventCount={totalCount}
/>
</EuiFlexGroup>
</EuiFlexItem>
<EuiFlexItem data-test-subj="last-updated-container" grow={false}>
<LastUpdatedContainer updatedAt={updatedAt} />
</EuiFlexItem>
<EuiFlexItem grow={false}>
{isLive ? (
<EuiText size="s" data-test-subj="is-live-on-message">
<b>
{i18n.AUTO_REFRESH_ACTIVE}{' '}
<EuiIconTip
color="text"
content={
<FormattedMessage
id="xpack.securitySolution.footer.autoRefreshActiveTooltip"
defaultMessage="While auto-refresh is enabled, timeline will show you the latest {numberOfItems} events that match your query."
values={{
numberOfItems: itemsCount,
}}
/>
}
type="iInCircle"
/>
</b>
</EuiText>
) : (
<PagingControl
totalCount={totalCount}
totalPages={totalPages}
activePage={activePage}
onPageClick={handleChangePageClick}
isLoading={isLoading}
/>
)}
</EuiFlexItem>
</FooterFlexGroup>
</FooterContainer>
);
};
FooterComponent.displayName = 'FooterComponent';
export const Footer = React.memo(FooterComponent);
Footer.displayName = 'Footer';

4
x-pack/plugins/security_solution/public/timelines/components/timeline/tabs/eql/index.tsx
View file

@ -105,7 +105,7 @@ export const EqlTabContentComponent: React.FC<Props> = ({
[end, isBlankTimeline, loadingSourcerer, start]
);
const [dataLoadingState, { events, inspect, totalCount, loadPage, refreshedAt, refetch }] =
const [dataLoadingState, { events, inspect, totalCount, loadNextBatch, refreshedAt, refetch }] =
useTimelineEvents({
dataViewId,
endDate: end,
@ -298,7 +298,7 @@ export const EqlTabContentComponent: React.FC<Props> = ({
refetch={refetch}
dataLoadingState={dataLoadingState}
totalCount={isBlankTimeline ? 0 : totalCount}
onFetchMoreRecords={loadPage}
onFetchMoreRecords={loadNextBatch}
activeTab={activeTab}
updatedAt={refreshedAt}
isTextBasedQuery={false}

4
x-pack/plugins/security_solution/public/timelines/components/timeline/tabs/pinned/index.tsx
View file

@ -138,7 +138,7 @@ export const PinnedTabContentComponent: React.FC<Props> = ({
);
const { augmentedColumnHeaders } = useTimelineColumns(columns);
const [queryLoadingState, { events, totalCount, loadPage, refreshedAt, refetch }] =
const [queryLoadingState, { events, totalCount, loadNextBatch, refreshedAt, refetch }] =
useTimelineEvents({
endDate: '',
id: `pinned-${timelineId}`,
@ -285,7 +285,7 @@ export const PinnedTabContentComponent: React.FC<Props> = ({
refetch={refetch}
dataLoadingState={queryLoadingState}
totalCount={totalCount}
onFetchMoreRecords={loadPage}
onFetchMoreRecords={loadNextBatch}
activeTab={TimelineTabs.pinned}
updatedAt={refreshedAt}
isTextBasedQuery={false}

299
x-pack/plugins/security_solution/public/timelines/components/timeline/tabs/query/index.test.tsx
View file

@ -10,7 +10,6 @@ import React, { useEffect } from 'react';
import QueryTabContent from '.';
import { defaultRowRenderers } from '../../body/renderers';
import { TimelineId } from '../../../../../../common/types/timeline';
import { useTimelineEvents } from '../../../../containers';
import { useTimelineEventsDetails } from '../../../../containers/details';
import { useSourcererDataView } from '../../../../../sourcerer/containers';
import { mockSourcererScope } from '../../../../../sourcerer/containers/mocks';
@ -40,13 +39,21 @@ import { createExpandableFlyoutApiMock } from '../../../../../common/mock/expand
import { OPEN_FLYOUT_BUTTON_TEST_ID } from '../../../../../notes/components/test_ids';
import { userEvent } from '@testing-library/user-event';
import * as notesApi from '../../../../../notes/api/api';
import { getMockTimelineSearchSubscription } from '../../../../../common/mock/mock_timeline_search_service';
import * as useTimelineEventsModule from '../../../../containers';
jest.mock('../../../../../common/utils/route/use_route_spy', () => {
return {
useRouteSpy: jest.fn().mockReturnValue([
{
pageName: 'timeline',
},
]),
};
});
jest.mock('../../../../../common/components/user_privileges');
jest.mock('../../../../containers', () => ({
useTimelineEvents: jest.fn(),
}));
jest.mock('../../../../containers/details');
jest.mock('../../../fields_browser', () => ({
@ -62,8 +69,6 @@ jest.mock('../../../../../sourcerer/containers/use_signal_helpers', () => ({
useSignalHelpers: () => ({ signalIndexNeedsInit: false }),
}));
jest.mock('../../../../../common/lib/kuery');
jest.mock('../../../../../common/hooks/use_experimental_features');
jest.mock('react-router-dom', () => ({
@ -74,6 +79,8 @@ jest.mock('react-router-dom', () => ({
})),
}));
const { mockTimelineSearchSubscription } = getMockTimelineSearchSubscription();
// These tests can take more than standard timeout of 5s
// that is why we are increasing it.
const SPECIAL_TEST_TIMEOUT = 50000;
@ -130,8 +137,32 @@ const customColumnOrder = [
},
];
const mockState = {
...structuredClone(mockGlobalState),
const mockBaseState = structuredClone(mockGlobalState);
const mockState: typeof mockGlobalState = {
...mockBaseState,
timeline: {
...mockBaseState.timeline,
timelineById: {
[TimelineId.test]: {
...mockBaseState.timeline.timelineById[TimelineId.test],
/* 1 record for each page */
itemsPerPage: 1,
itemsPerPageOptions: [1, 2, 3, 4, 5],
/* Returns 1 records in one query */
sampleSize: 1,
kqlQuery: {
filterQuery: {
kuery: {
kind: 'kuery',
expression: '*',
},
serializedQuery: '*',
},
},
},
},
},
};
mockState.timeline.timelineById[TimelineId.test].columns = customColumnOrder;
@ -146,20 +177,18 @@ const renderTestComponents = (props?: Partial<ComponentProps<typeof TestComponen
});
};
const loadPageMock = jest.fn();
const useSourcererDataViewMocked = jest.fn().mockReturnValue({
...mockSourcererScope,
});
const { storage: storageMock } = createSecuritySolutionStorageMock();
let useTimelineEventsMock = jest.fn();
const useTimelineEventsSpy = jest.spyOn(useTimelineEventsModule, 'useTimelineEvents');
describe('query tab with unified timeline', () => {
const fetchNotesMock = jest.spyOn(notesApi, 'fetchNotesByDocumentIds');
const fetchNotesSpy = jest.spyOn(notesApi, 'fetchNotesByDocumentIds');
beforeAll(() => {
fetchNotesMock.mockImplementation(jest.fn());
fetchNotesSpy.mockImplementation(jest.fn());
jest.mocked(useExpandableFlyoutApi).mockImplementation(() => ({
...createExpandableFlyoutApiMock(),
openFlyout: mockOpenFlyout,
@ -173,34 +202,30 @@ describe('query tab with unified timeline', () => {
},
});
});
const baseKibanaServicesMock = createStartServicesMock();
const kibanaServiceMock: StartServices = {
...createStartServicesMock(),
...baseKibanaServicesMock,
storage: storageMock,
data: {
...baseKibanaServicesMock.data,
search: {
...baseKibanaServicesMock.data.search,
search: mockTimelineSearchSubscription,
},
},
};
afterEach(() => {
jest.clearAllMocks();
storageMock.clear();
fetchNotesMock.mockClear();
fetchNotesSpy.mockClear();
cleanup();
localStorage.clear();
});
beforeEach(() => {
useTimelineEventsMock = jest.fn(() => [
false,
{
events: structuredClone(mockTimelineData.slice(0, 1)),
pageInfo: {
activePage: 0,
totalPages: 3,
},
refreshedAt: Date.now(),
totalCount: 3,
loadPage: loadPageMock,
},
]);
HTMLElement.prototype.getBoundingClientRect = jest.fn(() => {
return {
width: 1000,
@ -216,8 +241,6 @@ describe('query tab with unified timeline', () => {
};
});
(useTimelineEvents as jest.Mock).mockImplementation(useTimelineEventsMock);
(useTimelineEventsDetails as jest.Mock).mockImplementation(() => [false, {}]);
(useSourcererDataView as jest.Mock).mockImplementation(useSourcererDataViewMocked);
@ -299,33 +322,24 @@ describe('query tab with unified timeline', () => {
});
describe('pagination', () => {
beforeEach(() => {
// pagination tests need more than 1 record so here
// we return 5 records instead of just 1.
useTimelineEventsMock = jest.fn(() => [
false,
{
events: structuredClone(mockTimelineData.slice(0, 5)),
pageInfo: {
activePage: 0,
totalPages: 5,
const mockStateWithNoteInTimeline = {
...mockState,
timeline: {
...mockState.timeline,
timelineById: {
[TimelineId.test]: {
...mockState.timeline.timelineById[TimelineId.test],
/* 1 record for each page */
itemsPerPage: 1,
itemsPerPageOptions: [1, 2, 3, 4, 5],
savedObjectId: 'timeline-1', // match timelineId in mocked notes data
pinnedEventIds: { '1': true },
/* Returns 3 records */
sampleSize: 3,
},
refreshedAt: Date.now(),
/*
* `totalCount` could be any number w.r.t this test
* and actually means total hits on elastic search
* and not the fecthed number of records.
*
* This helps in testing `sampleSize` and `loadMore`
*/
totalCount: 50,
loadPage: loadPageMock,
},
]);
(useTimelineEvents as jest.Mock).mockImplementation(useTimelineEventsMock);
});
},
};
afterEach(() => {
jest.clearAllMocks();
});
@ -333,23 +347,6 @@ describe('query tab with unified timeline', () => {
it(
'should paginate correctly',
async () => {
const mockStateWithNoteInTimeline = {
...mockGlobalState,
timeline: {
...mockGlobalState.timeline,
timelineById: {
[TimelineId.test]: {
...mockGlobalState.timeline.timelineById[TimelineId.test],
/* 1 record for each page */
itemsPerPage: 1,
itemsPerPageOptions: [1, 2, 3, 4, 5],
savedObjectId: 'timeline-1', // match timelineId in mocked notes data
pinnedEventIds: { '1': true },
},
},
},
};
render(
<TestProviders
store={createMockStore({
@ -368,13 +365,13 @@ describe('query tab with unified timeline', () => {
);
expect(screen.getByTestId('pagination-button-0')).toHaveAttribute('aria-current', 'true');
expect(screen.getByTestId('pagination-button-4')).toBeVisible();
expect(screen.queryByTestId('pagination-button-5')).toBeNull();
expect(screen.getByTestId('pagination-button-2')).toBeVisible();
expect(screen.queryByTestId('pagination-button-3')).toBeNull();
fireEvent.click(screen.getByTestId('pagination-button-4'));
fireEvent.click(screen.getByTestId('pagination-button-2'));
await waitFor(() => {
expect(screen.getByTestId('pagination-button-4')).toHaveAttribute('aria-current', 'true');
expect(screen.getByTestId('pagination-button-2')).toHaveAttribute('aria-current', 'true');
});
},
SPECIAL_TEST_TIMEOUT
@ -383,27 +380,6 @@ describe('query tab with unified timeline', () => {
it(
'should load more records according to sample size correctly',
async () => {
const mockStateWithNoteInTimeline = {
...mockGlobalState,
timeline: {
...mockGlobalState.timeline,
timelineById: {
[TimelineId.test]: {
...mockGlobalState.timeline.timelineById[TimelineId.test],
itemsPerPage: 1,
/*
* `sampleSize` is the max number of records that are fetched from elasticsearch
* in one request. If hits > sampleSize, you can fetch more records ( <= sampleSize)
*/
sampleSize: 5,
itemsPerPageOptions: [1, 2, 3, 4, 5],
savedObjectId: 'timeline-1', // match timelineId in mocked notes data
pinnedEventIds: { '1': true },
},
},
},
};
render(
<TestProviders
store={createMockStore({
@ -418,15 +394,18 @@ describe('query tab with unified timeline', () => {
await waitFor(() => {
expect(screen.getByTestId('pagination-button-0')).toHaveAttribute('aria-current', 'true');
expect(screen.getByTestId('pagination-button-4')).toBeVisible();
expect(screen.getByTestId('pagination-button-2')).toBeVisible();
});
// Go to last page
fireEvent.click(screen.getByTestId('pagination-button-4'));
fireEvent.click(screen.getByTestId('pagination-button-2'));
await waitFor(() => {
expect(screen.getByTestId('dscGridSampleSizeFetchMoreLink')).toBeVisible();
});
fireEvent.click(screen.getByTestId('dscGridSampleSizeFetchMoreLink'));
expect(loadPageMock).toHaveBeenNthCalledWith(1, 1);
await waitFor(() => {
expect(screen.getByTestId('pagination-button-2')).toHaveAttribute('aria-current', 'true');
expect(screen.getByTestId('pagination-button-5')).toBeVisible();
});
},
SPECIAL_TEST_TIMEOUT
);
@ -434,24 +413,6 @@ describe('query tab with unified timeline', () => {
it(
'should load notes for current page only',
async () => {
const mockStateWithNoteInTimeline = {
...mockGlobalState,
timeline: {
...mockGlobalState.timeline,
timelineById: {
[TimelineId.test]: {
...mockGlobalState.timeline.timelineById[TimelineId.test],
/* 1 record for each page */
itemsPerPage: 1,
pageIndex: 0,
itemsPerPageOptions: [1, 2, 3, 4, 5],
savedObjectId: 'timeline-1', // match timelineId in mocked notes data
pinnedEventIds: { '1': true },
},
},
},
};
render(
<TestProviders
store={createMockStore({
@ -467,11 +428,11 @@ describe('query tab with unified timeline', () => {
expect(screen.getByTestId('pagination-button-previous')).toBeVisible();
expect(screen.getByTestId('pagination-button-0')).toHaveAttribute('aria-current', 'true');
expect(fetchNotesMock).toHaveBeenCalledWith(['1']);
expect(fetchNotesSpy).toHaveBeenCalledWith(['1']);
// Page : 2
fetchNotesMock.mockClear();
fetchNotesSpy.mockClear();
expect(screen.getByTestId('pagination-button-1')).toBeVisible();
fireEvent.click(screen.getByTestId('pagination-button-1'));
@ -479,19 +440,19 @@ describe('query tab with unified timeline', () => {
await waitFor(() => {
expect(screen.getByTestId('pagination-button-1')).toHaveAttribute('aria-current', 'true');
expect(fetchNotesMock).toHaveBeenNthCalledWith(1, [mockTimelineData[1]._id]);
expect(fetchNotesSpy).toHaveBeenNthCalledWith(1, [mockTimelineData[1]._id]);
});
// Page : 3
fetchNotesMock.mockClear();
fetchNotesSpy.mockClear();
expect(screen.getByTestId('pagination-button-2')).toBeVisible();
fireEvent.click(screen.getByTestId('pagination-button-2'));
await waitFor(() => {
expect(screen.getByTestId('pagination-button-2')).toHaveAttribute('aria-current', 'true');
expect(fetchNotesMock).toHaveBeenNthCalledWith(1, [mockTimelineData[2]._id]);
expect(fetchNotesSpy).toHaveBeenNthCalledWith(1, [mockTimelineData[2]._id]);
});
},
SPECIAL_TEST_TIMEOUT
@ -500,24 +461,6 @@ describe('query tab with unified timeline', () => {
it(
'should load notes for correct page size',
async () => {
const mockStateWithNoteInTimeline = {
...mockGlobalState,
timeline: {
...mockGlobalState.timeline,
timelineById: {
[TimelineId.test]: {
...mockGlobalState.timeline.timelineById[TimelineId.test],
/* 1 record for each page */
itemsPerPage: 1,
pageIndex: 0,
itemsPerPageOptions: [1, 2, 3, 4, 5],
savedObjectId: 'timeline-1', // match timelineId in mocked notes data
pinnedEventIds: { '1': true },
},
},
},
};
render(
<TestProviders
store={createMockStore({
@ -542,11 +485,11 @@ describe('query tab with unified timeline', () => {
expect(screen.getByTestId('tablePagination-2-rows')).toBeVisible();
});
fetchNotesMock.mockClear();
fetchNotesSpy.mockClear();
fireEvent.click(screen.getByTestId('tablePagination-2-rows'));
await waitFor(() => {
expect(fetchNotesMock).toHaveBeenNthCalledWith(1, [
expect(fetchNotesSpy).toHaveBeenNthCalledWith(1, [
mockTimelineData[0]._id,
mockTimelineData[1]._id,
]);
@ -556,6 +499,53 @@ describe('query tab with unified timeline', () => {
);
});
const openDisplaySettings = async () => {
expect(screen.getByTestId('dataGridDisplaySelectorButton')).toBeVisible();
fireEvent.click(screen.getByTestId('dataGridDisplaySelectorButton'));
await waitFor(() => {
expect(
screen
.getAllByTestId('unifiedDataTableSampleSizeInput')
.find((el) => el.getAttribute('type') === 'number')
).toBeVisible();
});
};
const updateSampleSize = async (sampleSize: number) => {
const sampleSizeInput = screen
.getAllByTestId('unifiedDataTableSampleSizeInput')
.find((el) => el.getAttribute('type') === 'number');
expect(sampleSizeInput).toBeVisible();
fireEvent.change(sampleSizeInput as HTMLElement, {
target: { value: sampleSize },
});
};
describe('controls', () => {
it(
'should reftech on sample size change',
async () => {
renderTestComponents();
await waitFor(() => {
expect(screen.getByTestId('discoverDocTable')).toBeVisible();
});
expect(screen.queryByTestId('pagination-button-1')).not.toBeInTheDocument();
await openDisplaySettings();
await updateSampleSize(2);
await waitFor(() => {
expect(screen.getByTestId('pagination-button-1')).toBeVisible();
});
},
SPECIAL_TEST_TIMEOUT
);
});
describe('columns', () => {
it(
'should move column left/right correctly ',
@ -642,12 +632,11 @@ describe('query tab with unified timeline', () => {
});
expect(screen.getByTitle('Unsort New-Old')).toBeVisible();
useTimelineEventsMock.mockClear();
useTimelineEventsSpy.mockClear();
fireEvent.click(screen.getByTitle('Sort Old-New'));
await waitFor(() => {
expect(useTimelineEventsMock).toHaveBeenNthCalledWith(
expect(useTimelineEventsSpy).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
sort: [
@ -686,12 +675,12 @@ describe('query tab with unified timeline', () => {
expect(screen.getByTitle('Sort A-Z')).toBeVisible();
expect(screen.getByTitle('Sort Z-A')).toBeVisible();
useTimelineEventsMock.mockClear();
useTimelineEventsSpy.mockClear();
fireEvent.click(screen.getByTitle('Sort A-Z'));
await waitFor(() => {
expect(useTimelineEventsMock).toHaveBeenNthCalledWith(
expect(useTimelineEventsSpy).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
sort: [
@ -741,12 +730,12 @@ describe('query tab with unified timeline', () => {
expect(screen.getByTitle('Sort Low-High')).toBeVisible();
expect(screen.getByTitle('Sort High-Low')).toBeVisible();
useTimelineEventsMock.mockClear();
useTimelineEventsSpy.mockClear();
fireEvent.click(screen.getByTitle('Sort Low-High'));
await waitFor(() => {
expect(useTimelineEventsMock).toHaveBeenNthCalledWith(
expect(useTimelineEventsSpy).toHaveBeenNthCalledWith(
1,
expect.objectContaining({
sort: [
@ -1214,12 +1203,12 @@ describe('query tab with unified timeline', () => {
'should disable pinning when event has notes attached in timeline',
async () => {
const mockStateWithNoteInTimeline = {
...mockGlobalState,
...mockState,
timeline: {
...mockGlobalState.timeline,
...mockState.timeline,
timelineById: {
[TimelineId.test]: {
...mockGlobalState.timeline.timelineById[TimelineId.test],
...mockState.timeline.timelineById[TimelineId.test],
savedObjectId: 'timeline-1', // match timelineId in mocked notes data
pinnedEventIds: { '1': true },
},

36
x-pack/plugins/security_solution/public/timelines/components/timeline/tabs/query/index.tsx
View file

@ -173,24 +173,22 @@ export const QueryTabContentComponent: React.FC<Props> = ({
const { augmentedColumnHeaders, defaultColumns, timelineQueryFieldsFromColumns } =
useTimelineColumns(columns);
const [
dataLoadingState,
{ events, inspect, totalCount, loadPage: loadNextEventBatch, refreshedAt, refetch },
] = useTimelineEvents({
dataViewId,
endDate: end,
fields: timelineQueryFieldsFromColumns,
filterQuery: combinedQueries?.filterQuery,
id: timelineId,
indexNames: selectedPatterns,
language: kqlQuery.language,
limit: sampleSize,
runtimeMappings: sourcererDataView?.runtimeFieldMap as RunTimeMappings,
skip: !canQueryTimeline,
sort: timelineQuerySortField,
startDate: start,
timerangeKind,
});
const [dataLoadingState, { events, inspect, totalCount, loadNextBatch, refreshedAt, refetch }] =
useTimelineEvents({
dataViewId,
endDate: end,
fields: timelineQueryFieldsFromColumns,
filterQuery: combinedQueries?.filterQuery,
id: timelineId,
indexNames: selectedPatterns,
language: kqlQuery.language,
limit: sampleSize,
runtimeMappings: sourcererDataView?.runtimeFieldMap as RunTimeMappings,
skip: !canQueryTimeline,
sort: timelineQuerySortField,
startDate: start,
timerangeKind,
});
const { onLoad: loadNotesOnEventsLoad } = useFetchNotes();
@ -392,7 +390,7 @@ export const QueryTabContentComponent: React.FC<Props> = ({
dataLoadingState={dataLoadingState}
totalCount={isBlankTimeline ? 0 : totalCount}
leadingControlColumns={leadingControlColumns as EuiDataGridControlColumn[]}
onFetchMoreRecords={loadNextEventBatch}
onFetchMoreRecords={loadNextBatch}
activeTab={activeTab}
updatedAt={refreshedAt}
isTextBasedQuery={false}

18
x-pack/plugins/security_solution/public/timelines/components/timeline/unified_components/data_table/index.test.tsx
View file

@ -16,7 +16,7 @@ import { useSourcererDataView } from '../../../../../sourcerer/containers';
import type { ComponentProps } from 'react';
import { getColumnHeaders } from '../../body/column_headers/helpers';
import { mockSourcererScope } from '../../../../../sourcerer/containers/mocks';
import { timelineActions } from '../../../../store';
import * as timelineActions from '../../../../store/actions';
import { useExpandableFlyoutApi } from '@kbn/expandable-flyout';
jest.mock('../../../../../sourcerer/containers');
@ -31,10 +31,12 @@ jest.mock('react-router-dom', () => ({
const onFieldEditedMock = jest.fn();
const refetchMock = jest.fn();
const onChangePageMock = jest.fn();
const onFetchMoreRecordsMock = jest.fn();
const openFlyoutMock = jest.fn();
const updateSampleSizeSpy = jest.spyOn(timelineActions, 'updateSampleSize');
jest.mock('@kbn/expandable-flyout');
const initialEnrichedColumns = getColumnHeaders(
@ -72,7 +74,7 @@ const TestComponent = (props: TestComponentProps) => {
refetch={refetchMock}
dataLoadingState={DataLoadingState.loaded}
totalCount={mockTimelineData.length}
onFetchMoreRecords={onChangePageMock}
onFetchMoreRecords={onFetchMoreRecordsMock}
updatedAt={Date.now()}
onSetColumns={jest.fn()}
onFilter={jest.fn()}
@ -97,6 +99,7 @@ describe('unified data table', () => {
});
});
afterEach(() => {
updateSampleSizeSpy.mockClear();
jest.clearAllMocks();
});
@ -199,7 +202,7 @@ describe('unified data table', () => {
});
it(
'should refetch on sample size change',
'should update sample size correctly',
async () => {
render(<TestComponent />);
@ -217,8 +220,11 @@ describe('unified data table', () => {
target: { value: '10' },
});
updateSampleSizeSpy.mockClear();
await waitFor(() => {
expect(refetchMock).toHaveBeenCalledTimes(1);
expect(updateSampleSizeSpy).toHaveBeenCalledTimes(1);
expect(updateSampleSizeSpy).toHaveBeenCalledWith({ id: TimelineId.test, sampleSize: 10 });
});
},
SPECIAL_TEST_TIMEOUT
@ -315,7 +321,7 @@ describe('unified data table', () => {
expect(screen.getByTestId('dscGridSampleSizeFetchMoreLink')).toBeVisible();
fireEvent.click(screen.getByTestId('dscGridSampleSizeFetchMoreLink'));
await waitFor(() => {
expect(onChangePageMock).toHaveBeenNthCalledWith(1, 1);
expect(onFetchMoreRecordsMock).toHaveBeenCalledTimes(1);
});
},
SPECIAL_TEST_TIMEOUT

9
x-pack/plugins/security_solution/public/timelines/components/timeline/unified_components/data_table/index.tsx
View file

@ -135,7 +135,6 @@ export const TimelineDataTableComponent: React.FC<DataTableProps> = memo(
} = useKibana();
const [expandedDoc, setExpandedDoc] = useState<DataTableRecord & TimelineItem>();
const [fetchedPage, setFechedPage] = useState<number>(0);
const onCloseExpandableFlyout = useCallback((id: string) => {
setExpandedDoc((prev) => (!prev ? prev : undefined));
@ -236,9 +235,8 @@ export const TimelineDataTableComponent: React.FC<DataTableProps> = memo(
);
const handleFetchMoreRecords = useCallback(() => {
onFetchMoreRecords(fetchedPage + 1);
setFechedPage(fetchedPage + 1);
}, [fetchedPage, onFetchMoreRecords]);
onFetchMoreRecords();
}, [onFetchMoreRecords]);
const additionalControls = useMemo(
() => <ToolbarAdditionalControls timelineId={timelineId} updatedAt={updatedAt} />,
@ -251,10 +249,9 @@ export const TimelineDataTableComponent: React.FC<DataTableProps> = memo(
(newSampleSize: number) => {
if (newSampleSize !== sampleSize) {
dispatch(timelineActions.updateSampleSize({ id: timelineId, sampleSize: newSampleSize }));
refetch();
}
},
[dispatch, sampleSize, timelineId, refetch]
[dispatch, sampleSize, timelineId]
);
const onUpdateRowHeight = useCallback(

891
x-pack/plugins/security_solution/public/timelines/containers/index.test.tsx
View file

@ -6,17 +6,20 @@
*/
import { DataLoadingState } from '@kbn/unified-data-table';
import { renderHook, act } from '@testing-library/react-hooks';
import { act, renderHook } from '@testing-library/react-hooks';
import type { TimelineArgs, UseTimelineEventsProps } from '.';
import { initSortDefault, useTimelineEvents } from '.';
import * as useTimelineEventsModule from '.';
import { SecurityPageName } from '../../../common/constants';
import { TimelineId } from '../../../common/types/timeline';
import { useIsExperimentalFeatureEnabled } from '../../common/hooks/use_experimental_features';
import { mockTimelineData } from '../../common/mock';
import { useRouteSpy } from '../../common/utils/route/use_route_spy';
import { useFetchNotes } from '../../notes/hooks/use_fetch_notes';
import { useKibana } from '../../common/lib/kibana';
import { getMockTimelineSearchSubscription } from '../../common/mock/mock_timeline_search_service';
import { waitFor } from '@testing-library/dom';
const { initSortDefault, useTimelineEvents } = useTimelineEventsModule;
const mockDispatch = jest.fn();
jest.mock('react-redux', () => {
const original = jest.requireActual('react-redux');
@ -31,10 +34,6 @@ jest.mock('../../notes/hooks/use_fetch_notes');
const onLoadMock = jest.fn();
const useFetchNotesMock = useFetchNotes as jest.Mock;
const mockEvents = mockTimelineData.slice(0, 10);
const mockSearch = jest.fn();
jest.mock('../../common/lib/apm/use_track_http_request');
jest.mock('../../common/hooks/use_experimental_features');
const useIsExperimentalFeatureEnabledMock = useIsExperimentalFeatureEnabled as jest.Mock;
@ -46,55 +45,7 @@ jest.mock('../../common/lib/kibana', () => ({
addWarning: jest.fn(),
remove: jest.fn(),
}),
useKibana: jest.fn().mockReturnValue({
services: {
application: {
capabilities: {
siem: {
crud: true,
},
},
},
data: {
search: {
search: jest.fn().mockImplementation((args) => {
mockSearch();
return {
subscribe: jest.fn().mockImplementation(({ next }) => {
const timeoutHandler = setTimeout(() => {
next({
isRunning: false,
isPartial: false,
inspect: {
dsl: [],
response: [],
},
edges: mockEvents.map((item) => ({ node: item })),
pageInfo: {
activePage: args.pagination.activePage,
totalPages: 10,
},
rawResponse: {},
totalCount: mockTimelineData.length,
});
}, 50);
return {
unsubscribe: jest.fn(() => {
clearTimeout(timeoutHandler);
}),
};
}),
};
}),
},
},
notifications: {
toasts: {
addWarning: jest.fn(),
},
},
},
}),
useKibana: jest.fn(),
}));
const mockUseRouteSpy: jest.Mock = useRouteSpy as jest.Mock;
@ -112,7 +63,40 @@ mockUseRouteSpy.mockReturnValue([
},
]);
describe('useTimelineEvents', () => {
const startDate: string = '2020-07-07T08:20:18.966Z';
const endDate: string = '3000-01-01T00:00:00.000Z';
const props: UseTimelineEventsProps = {
dataViewId: 'data-view-id',
endDate,
id: TimelineId.active,
indexNames: ['filebeat-*'],
fields: ['@timestamp', 'event.kind'],
filterQuery: '*',
startDate,
limit: 25,
runtimeMappings: {},
sort: initSortDefault,
skip: false,
};
const { mockTimelineSearchSubscription: mockSearchSubscription, mockSearchWithArgs: mockSearch } =
getMockTimelineSearchSubscription();
const loadNextBatch = async (result: { current: [DataLoadingState, TimelineArgs] }) => {
act(() => {
result.current[1].loadNextBatch();
});
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loadingMore);
});
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
});
};
describe('useTimelineEventsHandler', () => {
useIsExperimentalFeatureEnabledMock.mockReturnValue(false);
beforeEach(() => {
@ -123,155 +107,136 @@ describe('useTimelineEvents', () => {
useFetchNotesMock.mockReturnValue({
onLoad: onLoadMock,
});
(useKibana as jest.Mock).mockReturnValue({
services: {
application: {
capabilities: {
siem: {
crud: true,
},
},
},
data: {
search: {
search: mockSearchSubscription,
},
},
notifications: {
toasts: {
addWarning: jest.fn(),
},
},
},
});
});
const startDate: string = '2020-07-07T08:20:18.966Z';
const endDate: string = '3000-01-01T00:00:00.000Z';
const props: UseTimelineEventsProps = {
dataViewId: 'data-view-id',
endDate,
id: TimelineId.active,
indexNames: ['filebeat-*'],
fields: ['@timestamp', 'event.kind'],
filterQuery: '',
startDate,
limit: 25,
runtimeMappings: {},
sort: initSortDefault,
skip: false,
};
test('should init empty response', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: props,
});
test('init', async () => {
await act(async () => {
const { result, waitForNextUpdate } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
expect(result.current).toEqual([
DataLoadingState.loading,
{
events: [],
id: TimelineId.active,
inspect: expect.objectContaining({ dsl: [], response: [] }),
loadNextBatch: expect.any(Function),
pageInfo: expect.objectContaining({
activePage: 0,
querySize: 0,
}),
refetch: expect.any(Function),
totalCount: -1,
refreshedAt: 0,
},
]);
});
// useEffect on params request
await waitForNextUpdate();
test('should make events search request correctly', async () => {
const { result } = renderHook<UseTimelineEventsProps, [DataLoadingState, TimelineArgs]>(
(args) => useTimelineEvents(args),
{
initialProps: props,
}
);
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledTimes(1);
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
expect(result.current[1].events).toHaveLength(25);
expect(result.current).toEqual([
DataLoadingState.loaded,
{
events: [],
events: expect.any(Array),
id: TimelineId.active,
inspect: result.current[1].inspect,
loadPage: result.current[1].loadPage,
pageInfo: result.current[1].pageInfo,
loadNextBatch: result.current[1].loadNextBatch,
pageInfo: {
activePage: 0,
querySize: 25,
},
refetch: result.current[1].refetch,
totalCount: -1,
refreshedAt: 0,
totalCount: 32,
refreshedAt: result.current[1].refreshedAt,
},
]);
});
});
test('happy path query', async () => {
await act(async () => {
const { result, waitForNextUpdate, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: { ...props, startDate: '', endDate: '' },
});
// useEffect on params request
await waitForNextUpdate();
rerender({ ...props, startDate, endDate });
// useEffect on params request
await waitForNextUpdate();
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledTimes(2);
expect(result.current).toEqual([
DataLoadingState.loaded,
{
events: mockEvents,
id: TimelineId.active,
inspect: result.current[1].inspect,
loadPage: result.current[1].loadPage,
pageInfo: result.current[1].pageInfo,
refetch: result.current[1].refetch,
totalCount: 32,
refreshedAt: result.current[1].refreshedAt,
},
]);
});
test('should mock cache for active timeline when switching page', async () => {
const { result, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: props,
});
});
test('Mock cache for active timeline when switching page', async () => {
await act(async () => {
const { result, waitForNextUpdate, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: { ...props, startDate: '', endDate: '' },
});
mockUseRouteSpy.mockReturnValue([
{
pageName: SecurityPageName.timelines,
detailName: undefined,
tabName: undefined,
search: '',
pathName: '/timelines',
},
]);
// useEffect on params request
await waitForNextUpdate();
rerender({ ...props, startDate, endDate });
// useEffect on params request
await waitForNextUpdate();
rerender({ ...props, startDate, endDate });
mockUseRouteSpy.mockReturnValue([
{
pageName: SecurityPageName.timelines,
detailName: undefined,
tabName: undefined,
search: '',
pathName: '/timelines',
},
]);
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledTimes(2);
expect(result.current).toEqual([
DataLoadingState.loaded,
{
events: mockEvents,
id: TimelineId.active,
inspect: result.current[1].inspect,
loadPage: result.current[1].loadPage,
pageInfo: result.current[1].pageInfo,
refetch: result.current[1].refetch,
totalCount: 32,
refreshedAt: result.current[1].refreshedAt,
},
]);
});
await waitFor(() => {
expect(result.current[0]).toEqual(DataLoadingState.loaded);
});
expect(mockSearch).toHaveBeenCalledTimes(1);
expect(result.current[1].events).toHaveLength(25);
expect(result.current).toEqual([
DataLoadingState.loaded,
{
events: expect.any(Array),
id: TimelineId.active,
inspect: result.current[1].inspect,
loadNextBatch: result.current[1].loadNextBatch,
pageInfo: result.current[1].pageInfo,
refetch: result.current[1].refetch,
totalCount: 32,
refreshedAt: result.current[1].refreshedAt,
},
]);
});
test('Correlation pagination is calling search strategy when switching page', async () => {
await act(async () => {
const { result, waitForNextUpdate, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: {
...props,
language: 'eql',
eqlOptions: {
eventCategoryField: 'category',
tiebreakerField: '',
timestampField: '@timestamp',
query: 'find it EQL',
size: 100,
},
},
});
// useEffect on params request
await waitForNextUpdate();
rerender({
const { result, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: {
...props,
startDate,
endDate,
language: 'eql',
eqlOptions: {
eventCategoryField: 'category',
@ -280,32 +245,102 @@ describe('useTimelineEvents', () => {
query: 'find it EQL',
size: 100,
},
},
});
// useEffect on params request
await waitFor(() => new Promise((resolve) => resolve(null)));
rerender({
...props,
startDate,
endDate,
language: 'eql',
eqlOptions: {
eventCategoryField: 'category',
tiebreakerField: '',
timestampField: '@timestamp',
query: 'find it EQL',
size: 100,
},
});
// useEffect on params request
await waitFor(() => new Promise((resolve) => resolve(null)));
mockSearch.mockReset();
act(() => {
result.current[1].loadNextBatch();
});
await waitFor(() => expect(mockSearch).toHaveBeenCalledTimes(1));
});
describe('error/invalid states', () => {
const uniqueError = 'UNIQUE_ERROR';
const onError = jest.fn();
const mockSubscribeWithError = jest.fn(({ error }) => {
error(uniqueError);
});
beforeEach(() => {
onError.mockClear();
mockSubscribeWithError.mockClear();
(useKibana as jest.Mock).mockReturnValue({
services: {
data: {
search: {
search: () => ({
subscribe: jest.fn().mockImplementation(({ error }) => {
const requestTimeout = setTimeout(() => {
mockSubscribeWithError({ error });
}, 100);
return {
unsubscribe: () => {
clearTimeout(requestTimeout);
},
};
}),
}),
showError: onError,
},
},
},
});
});
test('should broadcast correct loading state when request throws error', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
expect(result.current[0]).toBe(DataLoadingState.loading);
await waitFor(() => {
expect(onError).toHaveBeenCalledWith(uniqueError);
expect(result.current[0]).toBe(DataLoadingState.loaded);
});
});
test('should should not fire any request when indexName is empty', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props, indexNames: [] },
});
await waitFor(() => {
expect(mockSearch).not.toHaveBeenCalled();
expect(result.current[0]).toBe(DataLoadingState.loaded);
});
// useEffect on params request
await waitForNextUpdate();
mockSearch.mockReset();
result.current[1].loadPage(4);
await waitForNextUpdate();
expect(mockSearch).toHaveBeenCalledTimes(1);
});
});
test('should query again when a new field is added', async () => {
await act(async () => {
const { waitForNextUpdate, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: { ...props, startDate: '', endDate: '' },
describe('fields', () => {
test('should query again when a new field is added', async () => {
const { rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: props,
});
// useEffect on params request
await waitForNextUpdate();
rerender({ ...props, startDate, endDate });
// useEffect on params request
await waitForNextUpdate();
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledTimes(1);
});
expect(mockSearch).toHaveBeenCalledTimes(2);
mockSearch.mockClear();
rerender({
@ -315,83 +350,437 @@ describe('useTimelineEvents', () => {
fields: ['@timestamp', 'event.kind', 'event.category'],
});
await waitFor(() => expect(mockSearch).toHaveBeenCalledTimes(1));
});
test('should not query again when a field is removed', async () => {
const { rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: props,
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledTimes(1);
});
});
});
test('should not query again when a field is removed', async () => {
await act(async () => {
const { waitForNextUpdate, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: { ...props, startDate: '', endDate: '' },
});
// useEffect on params request
await waitForNextUpdate();
rerender({ ...props, startDate, endDate });
// useEffect on params request
await waitForNextUpdate();
expect(mockSearch).toHaveBeenCalledTimes(2);
mockSearch.mockClear();
rerender({ ...props, startDate, endDate, fields: ['@timestamp'] });
rerender({ ...props, fields: ['@timestamp'] });
expect(mockSearch).toHaveBeenCalledTimes(0);
await waitFor(() => expect(mockSearch).toHaveBeenCalledTimes(0));
});
});
test('should not query again when a removed field is added back', async () => {
await act(async () => {
const { waitForNextUpdate, rerender } = renderHook<
UseTimelineEventsProps,
[DataLoadingState, TimelineArgs]
>((args) => useTimelineEvents(args), {
initialProps: { ...props, startDate: '', endDate: '' },
test('should not query again when a removed field is added back', async () => {
const { rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: props,
});
// useEffect on params request
await waitForNextUpdate();
rerender({ ...props, startDate, endDate });
// useEffect on params request
await waitForNextUpdate();
expect(mockSearch).toHaveBeenCalledTimes(2);
expect(mockSearch).toHaveBeenCalledTimes(1);
mockSearch.mockClear();
// remove `event.kind` from default fields
rerender({ ...props, startDate, endDate, fields: ['@timestamp'] });
rerender({ ...props, fields: ['@timestamp'] });
expect(mockSearch).toHaveBeenCalledTimes(0);
await waitFor(() => expect(mockSearch).toHaveBeenCalledTimes(0));
// request default Fields
rerender({ ...props, startDate, endDate });
rerender({ ...props });
expect(mockSearch).toHaveBeenCalledTimes(0);
await waitFor(() => expect(mockSearch).toHaveBeenCalledTimes(0));
});
});
test('should return the combined list of events for all the pages when multiple pages are queried', async () => {
await act(async () => {
describe('batching', () => {
test('should broadcast correct loading state based on the batch being fetched', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
await waitFor(() => {
expect(result.current[1].events).toHaveLength(10);
});
result.current[1].loadPage(1);
await waitFor(() => {
expect(result.current[0]).toEqual(DataLoadingState.loadingMore);
expect(result.current[0]).toBe(DataLoadingState.loading);
});
await waitFor(() => {
expect(result.current[1].events).toHaveLength(20);
expect(result.current[0]).toBe(DataLoadingState.loaded);
});
act(() => {
result.current[1].loadNextBatch();
});
expect(result.current[0]).toBe(DataLoadingState.loadingMore);
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
});
});
test('should request incremental batches when next batch has been requested', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
expect(mockSearch).toHaveBeenCalledTimes(1);
mockSearch.mockClear();
await loadNextBatch(result);
await waitFor(() => {
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 1, querySize: 25 } })
);
});
mockSearch.mockClear();
await loadNextBatch(result);
await waitFor(() => {
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 2, querySize: 25 } })
);
});
});
test('should fetch new columns data for the all the batches ', async () => {
const { result, rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
});
////////
// fetch 2 more batches before requesting new column
////////
await loadNextBatch(result);
await loadNextBatch(result);
///////
rerender({ ...props, fields: [...props.fields, 'new_column'] });
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({
fields: ['@timestamp', 'event.kind', 'new_column'],
pagination: { activePage: 0, querySize: 75 },
})
);
});
});
describe('refetching', () => {
/*
* Below are some use cases where refetch is triggered :
*
* - When user triggers a manual refresh of the data
* - When user updates an event, which triggers a refresh of the data
* - For example, when alert status is updated.
* - When user adds a new column
*
*/
test('should fetch first batch again when refetch is triggered', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props, timerangeKind: 'absolute' } as UseTimelineEventsProps,
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
mockSearch.mockClear();
act(() => {
result.current[1].refetch();
});
await waitFor(() => {
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
});
test('should fetch first batch again when refetch is triggered with relative timerange', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props, timerangeKind: 'relative' } as UseTimelineEventsProps,
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
mockSearch.mockClear();
act(() => {
result.current[1].refetch();
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledTimes(1);
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
});
test('should fetch first batch again when refetch is triggered when user has already fetched multiple batches', async () => {
const { result } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
mockSearch.mockClear();
await loadNextBatch(result);
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 1, querySize: 25 } })
);
});
mockSearch.mockClear();
act(() => {
result.current[1].refetch();
});
await waitFor(() => {
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
});
});
describe('sort', () => {
test('should fetch first batch again when sort is updated', async () => {
const { result, rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props } as UseTimelineEventsProps,
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
act(() => {
result.current[1].loadNextBatch();
});
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
mockSearch.mockClear();
act(() => {
rerender({
...props,
sort: [...initSortDefault, { ...initSortDefault[0], field: 'event.kind' }],
});
});
await waitFor(() => {
expect(mockSearch).toHaveBeenNthCalledWith(
1,
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
});
});
test('should query all batches when new column is added', async () => {
const { result, rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props },
});
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 25 } })
);
});
mockSearch.mockClear();
await loadNextBatch(result);
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 1, querySize: 25 } })
);
});
mockSearch.mockClear();
rerender({ ...props, fields: [...props.fields, 'new_column'] });
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 50 } })
);
});
mockSearch.mockClear();
await loadNextBatch(result);
await waitFor(() => {
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 2, querySize: 25 } })
);
});
});
test('should combine batches correctly when new column is added', async () => {
const { result, rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props, limit: 5 },
});
await waitFor(() => {
expect(result.current[1].events.length).toBe(5);
});
//////////////////////
// Batch 2
await loadNextBatch(result);
await waitFor(() => {
expect(result.current[1].events.length).toBe(10);
});
//////////////////////
//////////////////////
// Batch 3
await loadNextBatch(result);
await waitFor(() => {
expect(result.current[1].events.length).toBe(15);
});
//////////////////////
///////////////////////////////////////////
// add new column
// Fetch all 3 batches together
rerender({ ...props, limit: 5, fields: [...props.fields, 'new_column'] });
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loadingMore);
});
// should fetch all the records together
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
expect(result.current[1].events.length).toBe(15);
expect(result.current[1].pageInfo).toMatchObject({
activePage: 0,
querySize: 15,
});
});
///////////////////////////////////////////
//////////////////////
// subsequent batch should be fetched incrementally
// Batch 4
await loadNextBatch(result);
await waitFor(() => {
expect(result.current[1].events.length).toBe(20);
expect(result.current[1].pageInfo).toMatchObject({
activePage: 3,
querySize: 5,
});
});
//////////////////////
//////////////////////
// Batch 5
await loadNextBatch(result);
await waitFor(() => {
expect(result.current[1].events.length).toBe(25);
expect(result.current[1].pageInfo).toMatchObject({
activePage: 4,
querySize: 5,
});
});
//////////////////////
});
test('should request 0th batch when batchSize is changed', async () => {
const { result, rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props, limit: 5 },
});
//////////////////////
// Batch 2
await loadNextBatch(result);
//////////////////////
// Batch 3
await loadNextBatch(result);
mockSearch.mockClear();
// change the batch size
rerender({ ...props, limit: 10 });
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
expect(mockSearch).toHaveBeenCalledWith(
expect.objectContaining({ pagination: { activePage: 0, querySize: 10 } })
);
});
});
test('should return correct list of events ( 0th batch ) when batchSize is changed', async () => {
const { result, rerender } = renderHook((args) => useTimelineEvents(args), {
initialProps: { ...props, limit: 5 },
});
//////////////////////
// Batch 2
await loadNextBatch(result);
//////////////////////
// Batch 3
await loadNextBatch(result);
// change the batch size
rerender({ ...props, limit: 10 });
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loading);
});
await waitFor(() => {
expect(result.current[0]).toBe(DataLoadingState.loaded);
expect(result.current[1].events.length).toBe(10);
});
});
});

209
x-pack/plugins/security_solution/public/timelines/containers/index.tsx
View file

@ -6,7 +6,7 @@
*/
import deepEqual from 'fast-deep-equal';
import { isEmpty, noop } from 'lodash/fp';
import { isEmpty } from 'lodash/fp';
import { useCallback, useEffect, useRef, useState, useMemo } from 'react';
import { useDispatch } from 'react-redux';
import { Subscription } from 'rxjs';
@ -53,7 +53,7 @@ export interface TimelineArgs {
inspect: InspectResponse;
/**
* `loadPage` loads the next page/batch of records.
* `loadNextBatch` loads the next page/batch of records.
* This is different from the data grid pages. Data grid pagination is only
* client side and changing data grid pages does not impact this function.
*
@ -61,7 +61,7 @@ export interface TimelineArgs {
* irrespective of where user is in Data grid pagination.
*
*/
loadPage: LoadPage;
loadNextBatch: LoadPage;
pageInfo: Pick<PaginationInputPaginated, 'activePage' | 'querySize'>;
refetch: inputsModel.Refetch;
totalCount: number;
@ -72,7 +72,7 @@ type OnNextResponseHandler = (response: TimelineArgs) => Promise<void> | void;
type TimelineEventsSearchHandler = (onNextResponse?: OnNextResponseHandler) => void;
type LoadPage = (newActivePage: number) => void;
type LoadPage = () => void;
type TimelineRequest<T extends KueryFilterQueryKind> = T extends 'kuery'
? TimelineEventsAllOptionsInput
@ -163,11 +163,10 @@ export const useTimelineEventsHandler = ({
const [{ pageName }] = useRouteSpy();
const dispatch = useDispatch();
const { data } = useKibana().services;
const refetch = useRef<inputsModel.Refetch>(noop);
const abortCtrl = useRef(new AbortController());
const searchSubscription$ = useRef(new Subscription());
const [loading, setLoading] = useState<DataLoadingState>(DataLoadingState.loaded);
const [activePage, setActivePage] = useState(
const [activeBatch, setActiveBatch] = useState(
id === TimelineId.active ? activeTimeline.getActivePage() : 0
);
const [timelineRequest, setTimelineRequest] = useState<TimelineRequest<typeof language> | null>(
@ -184,7 +183,7 @@ export const useTimelineEventsHandler = ({
}, [dispatch, id]);
/**
* `wrappedLoadPage` loads the next page/batch of records.
* `loadBatchHandler` loads the next batch of records.
* This is different from the data grid pages. Data grid pagination is only
* client side and changing data grid pages does not impact this function.
*
@ -192,30 +191,33 @@ export const useTimelineEventsHandler = ({
* irrespective of where user is in Data grid pagination.
*
*/
const wrappedLoadPage = useCallback(
(newActivePage: number) => {
const loadBatchHandler = useCallback(
(newActiveBatch: number) => {
clearSignalsState();
if (id === TimelineId.active) {
activeTimeline.setActivePage(newActivePage);
activeTimeline.setActivePage(newActiveBatch);
}
setActivePage(newActivePage);
setActiveBatch(newActiveBatch);
},
[clearSignalsState, id]
);
const loadNextBatch = useCallback(() => {
loadBatchHandler(activeBatch + 1);
}, [activeBatch, loadBatchHandler]);
useEffect(() => {
return () => {
searchSubscription$.current?.unsubscribe();
};
}, []);
const refetchGrid = useCallback(() => {
if (refetch.current != null) {
refetch.current();
}
wrappedLoadPage(0);
}, [wrappedLoadPage]);
useEffect(() => {
// when batch size changes, refetch DataGrid
setActiveBatch(0);
}, [limit]);
const [timelineResponse, setTimelineResponse] = useState<TimelineArgs>({
id,
@ -223,14 +225,14 @@ export const useTimelineEventsHandler = ({
dsl: [],
response: [],
},
refetch: refetchGrid,
refetch: () => {},
totalCount: -1,
pageInfo: {
activePage: 0,
querySize: 0,
},
events: [],
loadPage: wrappedLoadPage,
loadNextBatch,
refreshedAt: 0,
});
@ -246,7 +248,8 @@ export const useTimelineEventsHandler = ({
const asyncSearch = async () => {
prevTimelineRequest.current = request;
abortCtrl.current = new AbortController();
if (activePage === 0) {
if (activeBatch === 0) {
setLoading(DataLoadingState.loading);
} else {
setLoading(DataLoadingState.loadingMore);
@ -317,7 +320,6 @@ export const useTimelineEventsHandler = ({
} else {
prevTimelineRequest.current = activeTimeline.getRequest();
}
refetch.current = asyncSearch;
setTimelineResponse((prevResp) => {
const resp =
@ -325,11 +327,7 @@ export const useTimelineEventsHandler = ({
? activeTimeline.getEqlResponse()
: activeTimeline.getResponse();
if (resp != null) {
return {
...resp,
refetch: refetchGrid,
loadPage: wrappedLoadPage,
};
return resp;
}
return prevResp;
});
@ -343,21 +341,35 @@ export const useTimelineEventsHandler = ({
searchSubscription$.current.unsubscribe();
abortCtrl.current.abort();
await asyncSearch();
refetch.current = asyncSearch;
},
[
pageName,
skip,
id,
activePage,
startTracking,
data.search,
dataViewId,
refetchGrid,
wrappedLoadPage,
]
[pageName, skip, id, activeBatch, startTracking, data.search, dataViewId]
);
const refetchGrid = useCallback(() => {
/*
*
* Trigger search with a new request object to fetch the latest data.
*
*/
const newTimelineRequest: typeof timelineRequest = {
...timelineRequest,
factoryQueryType: TimelineEventsQueries.all,
language,
sort,
fieldRequested: timelineRequest?.fieldRequested ?? fields,
fields: timelineRequest?.fieldRequested ?? fields,
pagination: {
activePage: 0,
querySize: limit,
},
};
setTimelineRequest(newTimelineRequest);
timelineSearch(newTimelineRequest);
setActiveBatch(0);
}, [timelineRequest, timelineSearch, limit, language, sort, fields]);
useEffect(() => {
if (indexNames.length === 0) {
return;
@ -368,7 +380,6 @@ export const useTimelineEventsHandler = ({
const prevSearchParameters = {
defaultIndex: prevRequest?.defaultIndex ?? [],
filterQuery: prevRequest?.filterQuery ?? '',
querySize: prevRequest?.pagination?.querySize ?? 0,
sort: prevRequest?.sort ?? initSortDefault,
timerange: prevRequest?.timerange ?? {},
runtimeMappings: (prevRequest?.runtimeMappings ?? {}) as unknown as RunTimeMappings,
@ -382,16 +393,15 @@ export const useTimelineEventsHandler = ({
const currentSearchParameters = {
defaultIndex: indexNames,
filterQuery: createFilter(filterQuery),
querySize: limit,
sort,
runtimeMappings,
runtimeMappings: runtimeMappings ?? {},
...timerange,
...deStructureEqlOptions(eqlOptions),
};
const newActivePage = deepEqual(prevSearchParameters, currentSearchParameters)
? activePage
: 0;
const areSearchParamsSame = deepEqual(prevSearchParameters, currentSearchParameters);
const newActiveBatch = !areSearchParamsSame ? 0 : activeBatch;
/*
* optimization to avoid unnecessary network request when a field
@ -410,16 +420,32 @@ export const useTimelineEventsHandler = ({
finalFieldRequest = prevRequest?.fieldRequested ?? [];
}
let newPagination = {
/*
*
* fetches data cumulatively for the batches upto the activeBatch
* This is needed because, we want to get incremental data as well for the old batches
* For example, newly requested fields
*
* */
activePage: newActiveBatch,
querySize: limit,
};
if (newFieldsRequested.length > 0) {
newPagination = {
activePage: 0,
querySize: (newActiveBatch + 1) * limit,
};
}
const currentRequest = {
defaultIndex: indexNames,
factoryQueryType: TimelineEventsQueries.all,
fieldRequested: finalFieldRequest,
fields: finalFieldRequest,
filterQuery: createFilter(filterQuery),
pagination: {
activePage: newActivePage,
querySize: limit,
},
pagination: newPagination,
language,
runtimeMappings,
sort,
@ -427,10 +453,10 @@ export const useTimelineEventsHandler = ({
...(eqlOptions ? eqlOptions : {}),
} as const;
if (activePage !== newActivePage) {
setActivePage(newActivePage);
if (activeBatch !== newActiveBatch) {
setActiveBatch(newActiveBatch);
if (id === TimelineId.active) {
activeTimeline.setActivePage(newActivePage);
activeTimeline.setActivePage(newActiveBatch);
}
}
if (!deepEqual(prevRequest, currentRequest)) {
@ -441,7 +467,7 @@ export const useTimelineEventsHandler = ({
}, [
dispatch,
indexNames,
activePage,
activeBatch,
endDate,
eqlOptions,
filterQuery,
@ -454,6 +480,31 @@ export const useTimelineEventsHandler = ({
runtimeMappings,
]);
/*
cleanup timeline events response when the filters were removed completely
to avoid displaying previous query results
*/
useEffect(() => {
if (isEmpty(filterQuery)) {
setTimelineResponse({
id,
inspect: {
dsl: [],
response: [],
},
refetch: () => {},
totalCount: -1,
pageInfo: {
activePage: 0,
querySize: 0,
},
events: [],
loadNextBatch,
refreshedAt: 0,
});
}
}, [filterQuery, id, loadNextBatch]);
const timelineSearchHandler = useCallback(
async (onNextHandler?: OnNextResponseHandler) => {
if (
@ -467,32 +518,15 @@ export const useTimelineEventsHandler = ({
[id, timelineRequest, timelineSearch, timerangeKind]
);
/*
cleanup timeline events response when the filters were removed completely
to avoid displaying previous query results
*/
useEffect(() => {
if (isEmpty(filterQuery)) {
setTimelineResponse({
id,
inspect: {
dsl: [],
response: [],
},
refetch: refetchGrid,
totalCount: -1,
pageInfo: {
activePage: 0,
querySize: 0,
},
events: [],
loadPage: wrappedLoadPage,
refreshedAt: 0,
});
}
}, [filterQuery, id, refetchGrid, wrappedLoadPage]);
const finalTimelineLineResponse = useMemo(() => {
return {
...timelineResponse,
loadNextBatch,
refetch: refetchGrid,
};
}, [timelineResponse, loadNextBatch, refetchGrid]);
return [loading, timelineResponse, timelineSearchHandler];
return [loading, finalTimelineLineResponse, timelineSearchHandler];
};
export const useTimelineEvents = ({
@ -536,19 +570,32 @@ export const useTimelineEvents = ({
* the combined list of events can be supplied to DataGrid.
*
* */
if (dataLoadingState !== DataLoadingState.loaded) return;
const { activePage, querySize } = timelineResponse.pageInfo;
setEventsPerPage((prev) => {
const result = [...prev];
result[timelineResponse.pageInfo.activePage] = timelineResponse.events;
let result = [...prev];
if (querySize === limit && activePage > 0) {
result[activePage] = timelineResponse.events;
} else {
result = [timelineResponse.events];
}
return result;
});
}, [timelineResponse.events, timelineResponse.pageInfo.activePage]);
}, [timelineResponse.events, timelineResponse.pageInfo, dataLoadingState, limit]);
useEffect(() => {
if (!timelineSearchHandler) return;
timelineSearchHandler();
}, [timelineSearchHandler]);
const combinedEvents = useMemo(() => eventsPerPage.flat(), [eventsPerPage]);
const combinedEvents = useMemo(
// exclude undefined values / empty slots
() => eventsPerPage.filter(Boolean).flat(),
[eventsPerPage]
);
const combinedResponse = useMemo(
() => ({

2
x-pack/plugins/timelines/server/search_strategy/timeline/eql/helpers.ts
View file

@ -20,7 +20,7 @@ import { inspectStringifyObject } from '../../../utils/build_query';
import { formatTimelineData } from '../factory/helpers/format_timeline_data';
export const buildEqlDsl = (options: TimelineEqlRequestOptions): Record<string, unknown> => {
if (options.pagination && options.pagination.querySize >= DEFAULT_MAX_TABLE_QUERY_SIZE) {
if (options.pagination && options.pagination.querySize > DEFAULT_MAX_TABLE_QUERY_SIZE) {
throw new Error(`No query size above ${DEFAULT_MAX_TABLE_QUERY_SIZE}`);
}

2
x-pack/plugins/timelines/server/search_strategy/timeline/factory/events/all/index.ts
View file

@ -22,7 +22,7 @@ import { formatTimelineData } from '../../helpers/format_timeline_data';
export const timelineEventsAll: TimelineFactory<TimelineEventsQueries.all> = {
buildDsl: ({ authFilter, ...options }) => {
if (options.pagination && options.pagination.querySize >= DEFAULT_MAX_TABLE_QUERY_SIZE) {
if (options.pagination && options.pagination.querySize > DEFAULT_MAX_TABLE_QUERY_SIZE) {
throw new Error(`No query size above ${DEFAULT_MAX_TABLE_QUERY_SIZE}`);
}
const { fieldRequested, ...queryOptions } = cloneDeep(options);

1
x-pack/plugins/translations/translations/fr-FR.json
View file

@ -41347,7 +41347,6 @@
"xpack.securitySolution.flyout.user.closeButton": "fermer",
"xpack.securitySolution.flyout.user.preview.viewDetailsLabel": "Afficher tous les détails de l'utilisateur",
"xpack.securitySolution.footer.autoRefreshActiveDescription": "Actualisation automatique active",
"xpack.securitySolution.footer.autoRefreshActiveTooltip": "Lorsque l'actualisation automatique est activée, la chronologie vous montrera les {numberOfItems} derniers événements correspondant à votre recherche.",
"xpack.securitySolution.footer.cancel": "Annuler",
"xpack.securitySolution.footer.data": "données",
"xpack.securitySolution.footer.events": "Événements",

1
x-pack/plugins/translations/translations/ja-JP.json
View file

@ -41313,7 +41313,6 @@
"xpack.securitySolution.flyout.user.closeButton": "閉じる",
"xpack.securitySolution.flyout.user.preview.viewDetailsLabel": "すべてのユーザー詳細を表示",
"xpack.securitySolution.footer.autoRefreshActiveDescription": "自動更新アクション",
"xpack.securitySolution.footer.autoRefreshActiveTooltip": "自動更新が有効な間、タイムラインはクエリーに一致する最新の {numberOfItems} 件のイベントを表示します。",
"xpack.securitySolution.footer.cancel": "キャンセル",
"xpack.securitySolution.footer.data": "データ",
"xpack.securitySolution.footer.events": "イベント",

1
x-pack/plugins/translations/translations/zh-CN.json
View file

@ -41383,7 +41383,6 @@
"xpack.securitySolution.flyout.user.closeButton": "关闭",
"xpack.securitySolution.flyout.user.preview.viewDetailsLabel": "显示全部用户详情",
"xpack.securitySolution.footer.autoRefreshActiveDescription": "自动刷新已启用",
"xpack.securitySolution.footer.autoRefreshActiveTooltip": "自动刷新已启用时，时间线将显示匹配查询的最近 {numberOfItems} 个事件。",
"xpack.securitySolution.footer.cancel": "取消",
"xpack.securitySolution.footer.data": "数据",
"xpack.securitySolution.footer.events": "事件",