Commit graph

160 commits

Author SHA1 Message Date
Jeramy Soucy
2e256ddd27
[8.3] Update simple-git 1.116.0 -> 3.10.0 (#137232) (#137603)
* Update simple-git 1.116.0 -> 3.10.0 (#137232)

* Updates simple-git to 3.10.0

* Replaces deprecated simple-git imports. Adds needed null check for git status.

* Updated simple-git package dep with caret.

* Fixes call to deprecated simple-git function prototype.

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-08-02 09:08:17 -04:00
Martijn Laarman
9ebb269f13
Allow default arguments to yarn es to be overwritten. (#130864)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Spencer <spencer@elastic.co>
2022-05-18 15:57:21 -07:00
Spencer
f3d69b8197
[@kbn/dev-utils] break out more pieces (#132292)
* [@kbn/dev-utils] break out more pieces

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-05-17 11:19:20 -05:00
Spencer
c6108ba076
[ftr] handle unexpected Kibana/ES shutdowns better (#131767) 2022-05-09 08:08:53 -07:00
Spencer
542b381fa5
[ftr] automatically determine config run order (#130983)
* [ftr] automatically determine config run order

* split lens config into two groups

* support ftr configs always running against CI

* Split detection_engine_api_integration rule exception list tests

* Add configs from previous commit

* [ftr] remove testMetadata and maintain a unique lifecycle instance per run

* Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run"

This reverts commit d2b4fdb824.

* Split alerting_api_integration/security_and_spaces tests

* Add groups to yaml

* Revert "Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run""

This reverts commit 56232eea68.

* stop ES more forcefully and fix timeout

* only cleanup lifecycle phases when the cleanup is totally complete

* only use kill when cleaning up an esTestInstance

* fix broken import

* fix runOptions.alwaysUseSource implementation

* fix config access

* fix x-pack/ccs config

* fix ml import file paths

* update kibana build id

* revert array.concat() change

* fix baseConfig usage

* fix pie chart data

* split up maps tests

* pull in all of group5 so that es archives are loaded correctly

* add to ftr configs.yml

* fix pie chart data without breaking legacy version

* fix more pie_chart stuff in new vis lib

* restore normal PR tasks

* bump kibana-buildkite-library

* remove ciGroup validation

* remove the script which is no longer called from checks.sh

* [CI] Auto-commit changed files from 'yarn kbn run build -i @kbn/pm'

* adapt flaky test runner scripts to handle ftrConfig paths

* fix types in alerting_api_integration

* improve flaky config parsing and use non-local var name for passing explicit configs to ftr_configs.sh

* Split xpack dashboard tests

* Add configs

* [flaky] remove key from ftr-config steps

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* restore cypress builds

* remove ciGroups from FTR config files

* fixup some docs

* add temporary script to hunt for FTR config files

* use config.base.js naming for clarity

* use script to power ftr_configs.yml

* remove usage of removed x-pack/scripts/functional_tests

* fix test names in dashboard snapshots

* bump kibana-buildkite-library

* Try retrying only failed configs

* be a little quieter about trying to get testStats from configs with testRunners defined

* Remove test code

* bump kibana-buildkite-library

* update es_snapshot and on_merge jobs too

* track duration and exit code for each config and print it at the end of the script

* store results in order, rather than by key, in case there are duplicates in $config

* bash is hard

* fix env source and use +e rather than disabling e for whole file

* bash sucks

* print config summary in jest jobs too

* define results in jest_parallel.sh

* simplify config summary print, format times a little better

* fix reference to unbound time variable, use better variable name

* skip the newline between each result

* finish with the nitpicking

* sync changes with ftr_configs.sh

* refuse to execute config files which aren't listed in the .buildkite/ftr_configs.yml

* fix config.edge.js base config import paths

* fix some readmes

* resolve paths from ftr_configs manifest

* fix readConfigFile tests

* just allow __fixtures__ configs

* list a few more cypress config files

* install the main branch of kibana-buildkite-library

* split up lens group1

* move ml data_visualizer tests to their own config

* fix import paths

* fix more imports

* install specific commit of buildkite-pipeline-library

* sort configs in ftr_configs.yml

* bump kibana-buildkite-library

* remove temporary script

* fix env var for limiting config types

* Update docs/developer/contributing/development-functional-tests.asciidoc

Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>

* produce a JUnit report for saved objects field count

* apply standard concurrency limits from flaky test runner

* support customizing FTR concurrency via the env

Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
2022-05-04 17:05:58 -05:00
Spencer
2a78f350e1
break out parts of @kbn/dev-utils (#130509)
* break out parts of @kbn/dev-utils

* autofix imports and kbn/pm dist

* update readme for @kbn/stdio-dev-helpers

* finish renames
2022-04-19 12:24:58 -05:00
Martijn Laarman
f74c894c5d
Extend yarn es: plugin support + adding secure files to es keystore (#126938)
Co-authored-by: Spencer <email@spalger.com>
2022-03-09 11:11:34 +01:00
Spencer
4f2321d34c
[ToolingLog] remove some manual indent management (#126636)
* [ToolingLog] remove some manual indent management

* improve log message

* [dev/build] write the newline after a failed step

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-03-03 08:15:46 -07:00
Tyler Smalley
49ad5846ae
[@kbn/es] Disable ES disk usage threshold (#123674) 2022-03-01 08:06:39 -08:00
Spencer
a16c20b7b4
[kbn/es] add support for --ready-timeout (#126217) 2022-02-24 15:44:37 -07:00
John Dorlus
68ab355dc3
CCS Fail Test Mode (#125997)
* Added the search.check_ccs_compatibility flag

* Moved up the level on the logging statements for ES Args so we can debug locally.

* Updated snapshots to reflect new flag that was added.
2022-02-24 13:10:55 -05:00
Spencer
1a1a191eb7
[jest] add *_node presets (#126192) 2022-02-23 09:36:30 -06:00
Tyler Smalley
435b772786
[ci] Splits Jest integration tests (#125454) 2022-02-14 17:12:42 +01:00
Tyler Smalley
a80acbb871
Resolves errors preventing ES snapshot promotion (#123649) 2022-01-26 10:11:36 -06:00
Joe Portner
f2e2add9ae
Use unencrypted P12 keystore for ES (#123650) 2022-01-24 16:37:27 -05:00
Tyler Smalley
23739f5f4d
Support for superuser not having write access (#123337)
Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-01-24 09:39:08 -08:00
Brian Seeders
17e79c150a
[kbn-es] Allow ES snapshot version mismatch with Kibana (#120942) 2021-12-15 13:17:02 -05:00
Tiago Costa
905a4472ca
chore(NA): use internal pkg_npm on @kbn/es (#120637)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-12-07 15:03:50 -07:00
Spencer
760374f5a0
[kbn-es/installers] convert to TypeScript (#120560)
* [kbn-es/installers] convert to TypeScript

* catch invalid urls passed to installArchive()
2021-12-07 09:28:19 -07:00
Esteban Beltran
a1e441401d
kibana es add --use-cached cli flag (#119149) 2021-11-19 12:39:56 -07:00
Mikhail Shustov
3c8fa527a7
[ES] Upgrade client to v8.0 (#113950)
* bump to a pre-8.0 version

* export KibanaClient from /lib sub-folder

* workaround the problem of the absence of estypes

* update es client usage in pacakges

* export estypes from another path

* import errors from root

* import errors from root 2

* update transport import

* update import path for /api/types

* update import path for /api/types

* import errors from top export

* use TransportResult instead if ApiResponse

* fix errors in client_config

* fix src/core/server/saved_objects/migrationsv2/actions/integration_tests/actions.test.ts

* use KibanaClient in mock. we dont export the original Client

* fix client mocks

* fix errors on SO

* fix remaining core errors

* update estype import path

* fix errors in data plugin

* fix data_views

* fix es_ui_shared

* fix errors in interactive_setup

* fix errors in ./test folder

* add @elastic/transport to the runtime deps

* fix errors in packages

* fix erros in src/core

* fix errors in test/

* fix an error in actions plugin

* woraround and fix errors in APM plugin

* fix errors in canvas

* fix errors in event_log

* fix errors in fleet

* fix errors in ILM

* fix errors in infra

* fix errors in ingest_pipeline

* fix errors in lens

* fix errors in license_management

* fix errors in licensing

* fix errors in logstash

* fix errors in ml

* fix errors in monitoring

* fix errors in observability

* fix errors in rule_registry

* fix errors in reporting

* fix errors in rule_registry

* fix errors in security

* fix errors in security_solution

* fix errors in snapshot_restore

* fix errors in transform

* fix errors in UA

* fix errors in uptime

* fix errors in x-pack/test

* fix eslint errors

* fix new errors

* use default HTTP Connection. Undici does not support agent config options keepAlive and maxSockets

* create does not accept require_alias option

* update deps

* use transport types exported from ES client package

* fix ErrorCause | string errors

* do not use enum

* fix errors in data plugin

* update x-pack code

* fix transport

* fix apm search request

* do not crash on reporting

* fix kbn-test build

* mute reporting error to start

* fix ftr build

* another attempt

* update import path

* address or mute new errors

* REMOVE me. pin transport version temporarily.

* remove deep imports from transport package

* fix jest crash

* fix product check tests

* remove unnecessary ts-expect-error

* fix a few failed unit tests

* bump to canary 24

* remove unnecessary ts-expect-error

* remove dependency on transport

* fix types in tests

* mute errors in xpack tests

* product check doesn;t  spam in logs anymore

* filterPath --> filter_path

* ignoreUnavailable --> ignore_unavailable

* ignoreUnavailable --> ignore_unavailable

* trackScores --> track_scores

* trackTotalHits --> track_total_hits

* fix es-arcives

* fix data plugin crashes

* fix watcher test utils

* rollback unnecessary changes

* fix another problem in es-archiver

* fix scroll. for whatever reason scroll fails when request scroll_id in body

* add meta: true in kbn-securitysolution-es-utils

* bump client to canary 25

* fix errors in accordance with the es client spec

* update securityscolution-es-utils

* unify scroll api in reporting and fix tests

* fix unit tests in watcher

* refactor APM to abort request with AbortController API

* fix missing es client calls in tests

* fix missing meta in detection engine FTR tests

* fix another bunch of errors in js tests

* fix wrong coercion

* remove test-grep pattern

* fix apm unit test

* rename terminateAfter to terminate_after in infra plugin

* rename terminateAfter to terminate_after in uptime plugin

* rename terminateAfter to terminate_after in apm plugin

* fix security roles FTR tests

* fix reference

* fix post_privilidges test

* fix post_privilidges

* bump client to 26

* add meta for index_management test helpers

* remove ts-expect-error caused by bad type in reason

* bump client to 27

* REMOVE me. workaround until fixed in the es client

* fix incorrect type casting

* swtich from camelCase params

* use `HttpConnection` for FTR-related clients

* bump client to 29

* Revert "REMOVE me. workaround until fixed in the es client"

This reverts commit c038850c09.

* fix new util

* revert repository changes

* do not crash if cannot store event_loop data

* fix new estypes imports

* fix more types

* fix security test types and add ts-ignore for custom ES client

* fix more estypes imports

* yet more ts violations

* line by line fixing is hard

* adapt `evaluateAlert` from infra as it's also used from FTR tests

* use convertToKibanaClient in FTR test instead of meta:true in plugin code

* migrate from deprecated API in fleet

* fix intergration tests

* fix fleet tests

* fix another fleet test

* fix more tests

* let's call it a day

* Removes custom header check on 404 responses, includes es client ProductNotSupportedError in EsUnavailableError conditional (#116029)

* Removes custom header check on 404 responses, includes es client ProductNotSupportedError in EsUnavailableError conditional

* Updates proxy response integration test

* disable APM until compatible with client v8

* skip async_search FTR test

* use kbnClient in integration tests

* bump version to 29

* bump to 30

* have configureClient return a KibanaClient instead of Client, remove resolved violations.

* bump to 31

* bump to 31

* Revert "bump to 31"

This reverts commit 5ac713e640.

* trigger stop to unusubscribe

* update generated docs

* remove obsolete test

* put "as" back

* cleanup

* skip test

* remove new type errors in apm package

* remove ErrorCause casting

* update a comment

* bump version to 32

* remove unnecessary ts-expect-error in apm code

* update comments

* update to client v33

* remove outdated type definition

* bump to 34 without params mutation

* unskip the test that should not fail anymore

* remove unnecessary ts-expect-error comments

* update to v35. body can be string

* move `sort` to body and use body friendly syntax

* fix a failing test. maps register the same SO that has been already registered by home

Co-authored-by: pgayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
2021-10-26 14:08:22 +02:00
Tyler Smalley
0a3ede8fd1
[ci] Explicitly define tasks for ES snapshot build (#115074)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2021-10-14 15:59:02 -07:00
Aleh Zasypkin
025861c189
Add API integration tests for Interactive Setup. (#111879) 2021-10-14 20:14:01 +02:00
Spencer
eeed2ca6ae
[ci] filter out proc-runner logs from stdout on CI (#114568)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-10-13 11:03:04 -05:00
Liza Katz
b62566da33
[ci-stats] Collects additional timings for cli tools (#113030)
* [ci-stats] Collects additional metrics about bootstrap

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* test reporting

* [ci-stats] Collects additional metrics about bootstrap

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Move ts timing to build_ts_refs_cli script

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Add timings to run

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* info debug level

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* fix build

* Move report function to dev-utils
align name of tests
report from functional test runner

* report snapshot install \ ready times
fix event names

* Report memory usage and branch hash

* fix eslint

* fix integration test

* build

* mergy merge

* mergy merge 2

* eslint

* eslint

* ready events

* Update packages/kbn-es/src/cli_commands/snapshot.js

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update packages/kbn-es/src/cluster.js

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update packages/kbn-es/src/cli_commands/snapshot.js

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update packages/kbn-test/src/functional_tests/cli/start_servers/cli.js

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update src/dev/run_check_published_api_changes.ts

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update packages/kbn-test/src/jest/run.ts

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update src/dev/run_i18n_check.ts

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* Update packages/kbn-test/src/functional_test_runner/cli.ts

Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>

* code review

* pm

* merge

* dist

* jest fix

Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tyler Smalley <tylersmalley@gmail.com>
2021-10-05 12:57:40 +03:00
Spencer
de2cfd2e65
[ci] bump Jenkins worker memory and es JVM size (#111760)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-09-09 14:26:08 -04:00
Tiago Costa
7c4e4f507b
chore(NA): replace babel config files by inline presets by default in the jsts_transpiler rule (#110620)
* chore(NA): replace babel config files by inline presets by default in the js_ts_transpiler rule

* chore(NA): update @kbn/ace build to exclude worker file

* chore(NA): remove config file support

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-09-03 21:17:46 +01:00
Alejandro Fernández Haro
bb8ee0ce05
Enable Product check from @elastic/elasticsearch-js (#107663)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-08-26 15:36:46 +01:00
Spencer
c0395c9ef6
[build_ts_refs] improve caches, allow building a subset of projects (#107981)
* [build_ts_refs] improve caches, allow building a subset of projects

* cleanup project def script and update refs in type check script

* rename browser_bazel config to avoid kebab-case

* remove execInProjects() helper

* list references for tsconfig.types.json for api-extractor workload

* disable composite features of tsconfig.types.json for api-extractor

* set declaration: true to avoid weird debug error

* fix jest tests

Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-08-10 22:12:45 -07:00
Alejandro Fernández Haro
74107a1fbd
Disable Product check in @elastic/elasticsearch-js (#107642)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-08-05 15:55:41 -04:00
Tiago Costa
4351d7c954
chore(NA): moving @kbn/es to babel transpiler (#107577) 2021-08-03 17:02:56 -04:00
Spencer
ec160d5c47
[kbn/dev-utils] pull in extract() helper (#106277)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-07-20 13:13:48 -07:00
Luke Elmers
d2690d8ac8
[migrations v2] Integration test for multi-node cluster. (#100957) 2021-06-28 12:49:38 -04:00
Tiago Costa
f69d63e8be
fix(NA): windows ts_project outside sandbox compilation (#100947)
* fix(NA): windows ts_project outside sandbox compilation adding tsconfig paths for packages

* chore(NA): missing @kbn paths for node_modules so types can work

* chore(NA): missing @kbn paths for node_modules so types can work

* chore(NA): organizing deps on non ts_project packages

* chore(NA): change order to find @kbn packages on node_modules first

* chore(NA): add @kbn/expect typings setting on package.json

* chore(NA): fix typechecking

* chore(NA): add missing change on tsconfig file

* chore(NA): unblock windows build by not depending on the pkg_npm rule symlink in the package.json

* chore(NA): add missing depedencies on BUILD.bazel file for io-ts-list-types

* chore(NA): remove rootDirs configs

* chore(NA): change kbn/monaco targets order

* chore(NA): update kbn-monaco build

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-06-03 17:53:39 +01:00
Tiago Costa
50616aa0f9
chore(NA): correctly setup dep order on js_library to fix windows build (#100420) 2021-05-21 00:03:54 +01:00
Tyler Smalley
9da9b8d4b9
[kbn-es] Allow overriding of Java heap (#99517)
* [kbn-es] Allow overriding of Java heap

Fixes #99494

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2021-05-11 07:36:20 -07:00
Tiago Costa
43d3a97f5b
chore(NA): moving @kbn/es into bazel (#99371)
* chore(NA): moving @kbn/es into bazel

* chore(NA): remove build scripts from pkg json

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-05-05 20:52:01 +01:00
Tiago Costa
f690c60517
chore(NA): moving @kbn/dev-utils into bazel (#98496)
* chore(NA): moving @kbn/dev-utils into bazel

* docs(NA): updated generated plugin list
2021-04-28 23:46:33 +01:00
Spencer
31660419ea
[kbn/es] disable geoip downloader at the root (#98372)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-04-27 11:31:07 -07:00
Tyler Smalley
c59d0ddeef
[kbn-es] Increase ES heap (#97905)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2021-04-21 18:37:44 -04:00
Tiago Costa
b6fd28884b
chore(NA): moving @kbn/babel-preset into bazel (#97271)
* chore(NA): moving @kbn/babel-preset into bazel

* chore(NA): remove unused deps from xpack package.json
2021-04-19 12:37:56 +02:00
Spencer
df4eb0f726
[kbn/es] avoid splitting string esArgs into separate args (#90253)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-02-03 21:58:28 -07:00
Brandon Kobel
4584a8b570
Elastic License 2.0 (#90099)
* Updating everything except the license headers themselves

* Applying ESLint rules

* Manually replacing the stragglers
2021-02-03 18:12:39 -08:00
Spencer
d07ae11da6
[es] disable wildcards in destructive actions (#88986)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-02-03 09:29:37 -07:00
Patrick Mueller
e4c344ada6
[data] change KQL node builder to not generate recursive and/or clauses (#89345)
resolves https://github.com/elastic/kibana/issues/88367

Prior to this PR, the KQL node_builder code was using recursion to generate
"and" & "or" expressions.  Eg, `and(foo1=bar1, foo2=bar2, foo3=bar3)` would
be generated as if was specified as `and(foo1=bar1, and(foo2=bar2, foo3=bar3))`.

Calls to the builder with long lists of expressions would generate nested JSON
as deep as the lists are long.  This is problematic, as Elasticsearch is
changing the default limit on nested bools to 20 levels, and alerting already
generates nested bools greater than that limit.
See: https://github.com/elastic/elasticsearch/issues/55303

This PR changes the generated shape of above, so that all the nodes are at the
same level, instead of the previous "recursive" treatment.
2021-02-01 12:22:29 -05:00
Brian Seeders
5ce916b308
[kbn-es] Always use bundled JDK when starting Elasticsearch (#89437) 2021-01-27 16:26:45 -05:00
Spencer
3c604438b8
[kbn/pm] throw an error if package doesn't have a script (#89438)
* [kbn/pm] throw an error if package doesn't have a script

* actually add the kbn/es build script 🤦‍♂️

Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-01-27 11:22:26 -07:00
Brandon Kobel
170a2956c8
Updating the License (#88343)
* Updating the Licenses, except for applying eslint, building

* Applying ESLint rules,building @kbn/pm, regenerating api docs
2021-01-19 17:52:56 -08:00
spalger
2fc6214627 [kbn/es] start es with raised indices.query.bool.max_nested_depth
(cherry picked from commit e914a60ba8c335a187e8a269596e5072ed4502ee)
(cherry picked from commit 28beb60d79)
2021-01-13 22:06:27 -07:00
Brian Seeders
646a34043a
Explicitly set Elasticsearch heap size during CI and local development (#86513) 2020-12-18 17:32:58 -05:00