Commit graph

41 commits

Author SHA1 Message Date
Quynh Nguyen
d5a9f0b375
[ML] Fix runtime field validations not supporting special script types (#122469) (#125300)
* Fix validations not handling extra things like params or lang

* Add translations

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit f8d932f70a)
2022-02-10 14:07:37 -07:00
Spencer
7d3d79e6a1
[8.0] [eslint] enable type-specific lint rules (#114184) (#117444)
* [eslint] enable type-specific lint rules (#114184)

* [eslint] enable type-specific lint rules

* autofix violations

* duplicate eslint-disable to new export statement

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Conflicts:
#	src/plugins/share/common/index.ts

* autofix more types

Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-11-04 03:02:56 +00:00
Kibana Machine
bd75d8047a
[ML] Fix Transforms not retaining _meta on clone (#116206) (#116598)
* [ML] Retain _meta on clone

* [ML] Fix validation on schema to only check it it's defined/not null

* [ML] Remove validation because es should handle the validation already

* Change type to unknown

Co-authored-by: Quynh Nguyen <43350163+qn895@users.noreply.github.com>
2021-11-01 18:05:35 -05:00
Mikhail Shustov
3c8fa527a7
[ES] Upgrade client to v8.0 (#113950)
* bump to a pre-8.0 version

* export KibanaClient from /lib sub-folder

* workaround the problem of the absence of estypes

* update es client usage in pacakges

* export estypes from another path

* import errors from root

* import errors from root 2

* update transport import

* update import path for /api/types

* update import path for /api/types

* import errors from top export

* use TransportResult instead if ApiResponse

* fix errors in client_config

* fix src/core/server/saved_objects/migrationsv2/actions/integration_tests/actions.test.ts

* use KibanaClient in mock. we dont export the original Client

* fix client mocks

* fix errors on SO

* fix remaining core errors

* update estype import path

* fix errors in data plugin

* fix data_views

* fix es_ui_shared

* fix errors in interactive_setup

* fix errors in ./test folder

* add @elastic/transport to the runtime deps

* fix errors in packages

* fix erros in src/core

* fix errors in test/

* fix an error in actions plugin

* woraround and fix errors in APM plugin

* fix errors in canvas

* fix errors in event_log

* fix errors in fleet

* fix errors in ILM

* fix errors in infra

* fix errors in ingest_pipeline

* fix errors in lens

* fix errors in license_management

* fix errors in licensing

* fix errors in logstash

* fix errors in ml

* fix errors in monitoring

* fix errors in observability

* fix errors in rule_registry

* fix errors in reporting

* fix errors in rule_registry

* fix errors in security

* fix errors in security_solution

* fix errors in snapshot_restore

* fix errors in transform

* fix errors in UA

* fix errors in uptime

* fix errors in x-pack/test

* fix eslint errors

* fix new errors

* use default HTTP Connection. Undici does not support agent config options keepAlive and maxSockets

* create does not accept require_alias option

* update deps

* use transport types exported from ES client package

* fix ErrorCause | string errors

* do not use enum

* fix errors in data plugin

* update x-pack code

* fix transport

* fix apm search request

* do not crash on reporting

* fix kbn-test build

* mute reporting error to start

* fix ftr build

* another attempt

* update import path

* address or mute new errors

* REMOVE me. pin transport version temporarily.

* remove deep imports from transport package

* fix jest crash

* fix product check tests

* remove unnecessary ts-expect-error

* fix a few failed unit tests

* bump to canary 24

* remove unnecessary ts-expect-error

* remove dependency on transport

* fix types in tests

* mute errors in xpack tests

* product check doesn;t  spam in logs anymore

* filterPath --> filter_path

* ignoreUnavailable --> ignore_unavailable

* ignoreUnavailable --> ignore_unavailable

* trackScores --> track_scores

* trackTotalHits --> track_total_hits

* fix es-arcives

* fix data plugin crashes

* fix watcher test utils

* rollback unnecessary changes

* fix another problem in es-archiver

* fix scroll. for whatever reason scroll fails when request scroll_id in body

* add meta: true in kbn-securitysolution-es-utils

* bump client to canary 25

* fix errors in accordance with the es client spec

* update securityscolution-es-utils

* unify scroll api in reporting and fix tests

* fix unit tests in watcher

* refactor APM to abort request with AbortController API

* fix missing es client calls in tests

* fix missing meta in detection engine FTR tests

* fix another bunch of errors in js tests

* fix wrong coercion

* remove test-grep pattern

* fix apm unit test

* rename terminateAfter to terminate_after in infra plugin

* rename terminateAfter to terminate_after in uptime plugin

* rename terminateAfter to terminate_after in apm plugin

* fix security roles FTR tests

* fix reference

* fix post_privilidges test

* fix post_privilidges

* bump client to 26

* add meta for index_management test helpers

* remove ts-expect-error caused by bad type in reason

* bump client to 27

* REMOVE me. workaround until fixed in the es client

* fix incorrect type casting

* swtich from camelCase params

* use `HttpConnection` for FTR-related clients

* bump client to 29

* Revert "REMOVE me. workaround until fixed in the es client"

This reverts commit c038850c09.

* fix new util

* revert repository changes

* do not crash if cannot store event_loop data

* fix new estypes imports

* fix more types

* fix security test types and add ts-ignore for custom ES client

* fix more estypes imports

* yet more ts violations

* line by line fixing is hard

* adapt `evaluateAlert` from infra as it's also used from FTR tests

* use convertToKibanaClient in FTR test instead of meta:true in plugin code

* migrate from deprecated API in fleet

* fix intergration tests

* fix fleet tests

* fix another fleet test

* fix more tests

* let's call it a day

* Removes custom header check on 404 responses, includes es client ProductNotSupportedError in EsUnavailableError conditional (#116029)

* Removes custom header check on 404 responses, includes es client ProductNotSupportedError in EsUnavailableError conditional

* Updates proxy response integration test

* disable APM until compatible with client v8

* skip async_search FTR test

* use kbnClient in integration tests

* bump version to 29

* bump to 30

* have configureClient return a KibanaClient instead of Client, remove resolved violations.

* bump to 31

* bump to 31

* Revert "bump to 31"

This reverts commit 5ac713e640.

* trigger stop to unusubscribe

* update generated docs

* remove obsolete test

* put "as" back

* cleanup

* skip test

* remove new type errors in apm package

* remove ErrorCause casting

* update a comment

* bump version to 32

* remove unnecessary ts-expect-error in apm code

* update comments

* update to client v33

* remove outdated type definition

* bump to 34 without params mutation

* unskip the test that should not fail anymore

* remove unnecessary ts-expect-error comments

* update to v35. body can be string

* move `sort` to body and use body friendly syntax

* fix a failing test. maps register the same SO that has been already registered by home

Co-authored-by: pgayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
2021-10-26 14:08:22 +02:00
Matthew Kime
64f1dddcb0
[feature branch] index pattern => data view for user facing content (#109821)
* [index pattern management] index pattern => data view for user facing content (#109577)
2021-10-20 18:53:39 -05:00
Dima Arnautov
2aaa515bbc
[Transform] Add alerting rules management to Transform UI (#115363)
* transform alert flyout

* fetch alerting rules

* show alerting rules indicators

* filter continuous transforms

* add alert rules to the expanded row

* edit alert rule from the list

* fix ts issues

* fix types

* update texts

* refactor using context, wip create alert from the list

* update unit test

* fix ts issue

* privilege check
2021-10-19 17:15:52 +02:00
Dima Arnautov
6da1323ff5
[Transform] Transforms health alerting rule type (#112277) 2021-10-06 18:27:24 +02:00
Walter Rafelsberger
ae4e7ccc51
[ML] Transforms: Align privileges checks with ML plugin. (#112970)
To check the available node count, the ML plugin has an additional privileges check before returning the result. This PR uses the same approach for the corresponding transforms node endpoint.
2021-09-27 15:35:27 +02:00
Sébastien Loix
b24d44d165
[Index pattern field editor] Add preview for runtime fields (#100198) 2021-08-13 18:27:23 -04:00
Mikhail Shustov
d920682e4e
Update @elastic/elasticsearch to 8.0.0-canary13 (#98266)
* bump @elastic/elasticsearch to canary.7

* address errors in core

* address errors in data plugin

* address errors in Alerting team plugins

* remove outdated messages in Lens

* remove unnecessary comments in ML

* address errors in Observability plugin

* address errors in reporting plugin

* address errors in Rule registry plugin

* fix errors in Security plugins

* fix errors in ES-UI plugin

* remove unnecessary union.

* update core tests

* fix kbn-es-archiver

* update to canary 8

* bump to v9

* use new typings

* fix new errors in core

* fix errors in core typeings

* fix type errors in data plugin

* fix type errors in telemetray plugin

* fix data plugin tests

* fix search examples type error

* fix errors in discover plugin

* fix errors in index_pattern_management

* fix type errors in vis_type_*

* fix errors in typings/elasticsearch

* fix type errors in actions plugin

* fix type errors in alerting and apm plugins

* fix type errors in canvas and cases

* fix errors in event_log

* fix type errors in ILM and ingest_pipelines

* fix errors in lens plugin

* fix errors in lists plugin

* fix errors in logstash

* fix errors in metrics_entities

* fix errors in o11y

* fix errors in watcher

* fix errors in uptime

* fix errors in upgrade_assistant

* fix errors in task_manager

* fix errors in stack_alerts

* fix errors in security_solution

* fix errors in rule_registry

* fix errors in snapshot_restore

* fix remaining errors

* fix search intergration tests

* adjust assetion

* bump version to canary.10

* adapt code to new naming schema

* use mapping types provided by the client library

* Revert "adjust assetion"

This reverts commit 19b8fe0464.

* fix so intergration tests

* fix http integration tests

* bump version to canary 11

* fix login test

* fix http integration test

* fix apm test

* update docs

* fixing some ml types

* fix new errors in data plugin

* fix new errors in alerting plugin

* fix new errors in lists plugin

* fix new errors in reporting

* fix or mute errors in rule_registry plugin

* more ML type fixes

* bump to canary 12

* fix errors after merge conflict

* additional ML fixes

* bump to canary 13

* fix errors in apm plugin

* fix errors in fleet plugin

* fix errors in infra plugin

* fix errors in monitoring plugin

* fix errors in osquery plugin

* fix errors in security solution plugins

* fix errors in transform plugin

* Update type imports for ES

* fix errors in x-pack plugins

* fix errors in tests

* update docs

* fix errors in x-pack/test

* update error description

* fix errors after master merge

* update comment in infra plugin

* fix new errors on xpack tests/

Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
2021-06-08 15:06:06 +02:00
Dima Arnautov
9810a72720
[Transform] Support for the top_metrics aggregation (#101152)
* [ML] init top_metrics agg

* [ML] support sort

* [ML] support _score sorting

* [ML] support sort mode

* [ML] support numeric type sorting

* [ML] update field label, hide additional sorting controls

* [ML] preserve advanced config

* [ML] update agg fields after runtime fields edit

* [ML] fix TS issue with EuiButtonGroup

* [ML] fix Field label

* [ML] refactor setUiConfig

* [ML] update unit tests

* [ML] wrap advanced sorting settings with accordion

* [ML] config validation with tests

* [ML] fix preserving of the unsupported config

* [ML] update translation message

* [ML] fix level of the custom config

* [ML] preserve unsupported config for sorting
2021-06-04 16:04:53 +02:00
Walter Rafelsberger
dfcb1794d4
[ML] Transforms: Fix handling of fields with keyword mapping available (#98882)
- For groupby/agg configs, removes the .keyword postfix for the agg name and field name being displayed. The config itself will still use the field name including .keyword.
- For histogram charts, if available, query data using the .keyword field. This enables support for charts for terms when there's both a text and keyword variant.
- Fixes isKeywordDuplicate check for field names with multiple dots in them.
2021-05-04 16:21:27 +02:00
Walter Rafelsberger
42d361c644
[ML] Adds waiting state for transforms. (#98592)
When no transform nodes are available, existing continuous transform end up in a waiting state. This PR adds support for this state in the transforms UI. Without the fix, transforms in a waiting state would fail to show up in the transform list.
2021-04-29 10:17:22 +02:00
Quynh Nguyen
71e8118984
[ML] Fix runtime mapping texts to runtime fields, add transform switch modal (#97008) 2021-04-19 13:46:18 -05:00
Walter Rafelsberger
a1c36e7a06
[ML] Refactor imports using 'elasticsearch' to '@elastic/elasticsearch'. Extend 'isPopulatedOjbect()'. (#95651)
- Gets rid of imports from 'elasticsearch' and replaces them with '@elastic/elasticsearch'.
- Extends isPopulatedObject() to allow an optional array of attributes to check if they exist. Allows us to get rid of the manual and inconsistent usages of hasOwnProperty().
2021-04-01 11:46:17 +02:00
Quynh Nguyen
f061303282
[ML] Fix Transform runtime mappings retain original definition when cloning (#95436)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-03-26 10:25:39 -05:00
Walter Rafelsberger
f3b74b457c
[ML] Transforms: Fixes missing number of transform nodes and error reporting in stats bar. (#93956)
- Adds a Kibana API endpoint transforms/_nodes
- Adds number of nodes to the stats bar in the transforms list.
- Shows a callout when no transform nodes are available.
- Disable all actions except delete when no transform nodes are available.
- Disables the create button when no transform nodes are available.
2021-03-16 11:41:48 +01:00
Walter Rafelsberger
066e47e9ea
[ML] Use unknown instead of any for type guards. (#94090) 2021-03-10 12:40:49 +01:00
Walter Rafelsberger
326dfa43ca
[ML] Improves error handling for transform wizard when Kibana index pattern or saved search fails to load. (#93915)
Improves error handling for the transform wizard when Kibana index pattern or saved search fails to load.
Previously a non-existing index pattern or saved search or corrupt saved object could cause the page to end up blank. Improved error reporting will catch the problem and display an error callout.
2021-03-09 15:29:45 +01:00
Walter Rafelsberger
8201d4fd01
[ML] Transforms: Fixes chart histograms for runtime fields. (#93028)
Fixes chart histograms for runtime fields. The runtime field configurations were not passed on to the endpoint to fetch the charts data, so charts ended up being empty with a 0 documents legend.
2021-03-02 13:37:29 +01:00
Walter Rafelsberger
45155f089d
[ML] Show mini histograms by default if row count below threshold. (#92021)
Show mini histograms by default if row count below threshold of 10000 docs.
2021-02-23 17:34:59 +01:00
Quynh Nguyen
7a75e19c3e
[ML] Fix runtime mappings for runtimeField with optional script field (#91634) 2021-02-17 09:40:49 -06:00
Quynh Nguyen
63ac0f74be
[ML] Add better UI support for runtime fields Transforms (#90363)
* [ML] Add RT support for transforms from index pattern

* [ML] Add support for cloned transform from api

* [ML] Add support for runtime pivot

* [ML] Add support for api created runtime

* [ML] Add preview for expanded row

* [ML] Add runtime fields to dropdown options

* [ML] Add runtime fields to latest

* [ML] Fix duplicate columns

* [ML] Update types and test

* [ML] Add runtime mappings to index pattern on creation

* [ML] Add callout to show unsupported fields in dfa

* [ML] Update types to RuntimeField

* [ML] Fix runtime fields, remove runtime mappings, fix copy to console

* [ML] Fix incompatible kbn field type

* [ML] Add advanced mappings editor

* [ML] Add support for filter terms agg control

* [ML] Fix jest tests hanging

* [ML] Fix translations

* [ML] Fix over-sized buttons for filter range

* [ML] Update runtime mappings schema

* [ML] Update runtime mappings schema

* [ML] Use isRecord for object checks

* [ML] Fix and more message

* [ML] Update schema to correctly match types

* [ML] Update schema to correctly match types

* [ML] Fix pivot duplicates

* [ML] Rename isRecord to isPopulatedObject

* [ML] Remove fit-content

* [ML] Update runtime field type to prevent potential conflicts

* Revert "[ML] Remove fit-content"

This reverts commit 76c9c799

* [ML] Remove misc comment

* [ML] Fix missing typeof

* [ML] Add sorts and constants

* [ML] Add i18n to includedFields description

* [ML] fix imports

* [ML] Only pass runtime mappings if it's latest

* [ML] Fix functional tests
2021-02-16 20:00:02 -06:00
Walter Rafelsberger
073cd4d508
[ML] Transforms: Adds retention policy options to transform UI. (#91162)
Adds retention policy options to the transform UI.
2021-02-16 21:29:56 +01:00
Brandon Kobel
4584a8b570
Elastic License 2.0 (#90099)
* Updating everything except the license headers themselves

* Applying ESLint rules

* Manually replacing the stragglers
2021-02-03 18:12:39 -08:00
Dima Arnautov
1d49166203
[Transform] Fix transform preview for the latest function (#87168)
* [Transform] retrieve mappings from the source index

* [Transform] fix preview for nested props

* [Transform] exclude meta fields

* [Transform] use agg config to only suggest term agg supported fields

* [Transform] refactor

* [Transform] remove incorrect data mock

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-01-07 00:28:05 +01:00
Dima Arnautov
e17cd65196
[Transform] Add support for latest function (#85784)
* [Transform] add latest to the schema definition

* [ML] update interfaces, add guards

* [Transform] WIP support latest function

* [Transform] fix request with missing_bucket after merge

* [Transform] fix error in useDeleteTransforms

* [Transform] fix types and fields sorting for pivot

* [Transform] fix types and jest tests

* [Transform] fix error shape

* [Transform] fixed card width, change description

* [Transform] fixed API integration tests

* [Transform] fix config mapper

* [Transform] improve wizard performance
2020-12-15 22:40:13 +01:00
Walter Rafelsberger
a6e6b62658
[ML] Transforms: Support for missing_bucket in transform advanced pivot editor (#85758)
Adds support for missing_bucket with group-by configurations in the advanced editor for pivot configurations.
Previously, the editor would allow you to add the setting, but it would be stripped from the config once the transform gets created.
2020-12-14 17:51:18 +01:00
Walter Rafelsberger
61aac716b3
[ML] Limit exposing shared static code through ml/public/index.ts. (#77745)
- Clean up lodash imports.
- Import types only where applicable.
- Reduce page load bundle size by fetching more code asynchronously.
2020-09-29 11:57:53 +02:00
Pierre Gayvallet
0d09cea436
Remove legacy plugins support (#77599)
* remove ALL the things.

* adapt some types and tests

* restore ensureValidConfiguration

* fix legacy service tests

* adapt uiRender mixin

* remove legacy types

* update generated doc

* restore legacy plugin schema

* update generated doc

* remove remaining code of x-pack/legacy

* adapt imports due to merge

* cleanup CODEOWNERS

* cleanup gitignore & i18nrc

* cleanup tsconfig.json

* remove unused i18n keys

* add back `"legacy/plugins/**/*",` to tsconfig until legacy space plugin is deleted

* fix create_jest_config

* remove references from eslintrc

* more eslint cleanup

* remove `x-pack/index.js`

* fix xpack gulp scripts

* fix bug with default + named imports from boom

* remove rules from eslintrc

* remove LegacyInternals

* review comments

* update generated doc

* cleanup legacy metadatas

* revert changes to eslintrc

* update generated doc
2020-09-23 09:52:51 +02:00
Walter Rafelsberger
4e7b7bf65f
[ML] Transforms: Extend editing and creation options. (#77370)
Extends the available options in the edit transform flyout and the wizard details step.
2020-09-21 16:55:40 +02:00
Walter Rafelsberger
dd1822047c
[ML] Transforms: API schemas and integration tests (#75164)
- Adds schema definitions to transform API endpoints and adds API integration tests.
- The type definitions based on the schema definitions can be used on the client side too.
- Adds apidoc documentation.
2020-09-14 16:31:23 +02:00
James Gowdy
ea8086b3f2
[ML] Improving client side error handling (#76743)
* [ML] Improving client side error handling

* adding stacktrace to request errors

* copying error parsing to transforms

* update snapshot

* fixing jest tests

* adding test and removing debug log output

* updating translations

* rewriting error extracting code

* fixing tests

* removing message bar service

* removing test code

* updating translations

* combining job creation error handling

* refactoring error files

* updating test

* fixing bug in DFA deletion

* improving mml warning

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-11 08:53:34 +01:00
Quynh Nguyen
22d09a3bbd
[ML] Transform: Enable force delete if one of the transforms failed (#69472)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-06-23 15:05:44 -05:00
Quynh Nguyen
1158be9264
[ML] Transform: Table enhancements (#69307)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-06-23 15:04:24 -05:00
Quynh Nguyen
5f3c3c0354
[ML] Transforms - Updated: Add ability to delete dest index & index pattern when deleting transform job (#68896) 2020-06-11 10:39:31 -05:00
Quynh Nguyen
3197a00e79
Revert "[ML] Transforms: Add ability to delete dest index & index pattern when deleting transform job" (#68793) 2020-06-10 12:02:52 -05:00
Quynh Nguyen
9bc0936311
[ML] Transforms: Add ability to delete dest index & index pattern when deleting transform job (#67922) 2020-06-10 10:11:46 -05:00
restrry
bf04235dae apply prettier styles 2020-05-22 09:08:58 +02:00
Walter Rafelsberger
ec1f46bdbd
[ML] Transforms: Data grid fixes. (#59538)
- Fixes data grid schemas to avoid crashing the page.
- Fixes date formatting.
- Uses data grid for preview table in transform list.
2020-03-10 15:52:17 +01:00
Walter Rafelsberger
f4f956dfeb
[ML] Transforms: Migrate client plugin to NP. (#59443)
Migrates the client side plugin of transforms to NP.
- Gets rid of the last parts of the shim (http, documentation links)
- Moves the plugin from x-pack/legacy/plugins/transform/public to x-pack/plugins/transform
- Creates a custom mock for appDependencies based on NP services
- Fixes jest tests to get rid of all act() related warnings
2020-03-06 17:44:35 +01:00