Fixes https://github.com/elastic/kibana/issues/158133
Sources where not getting removed because `return` was used instead of
`continue` in `for...in` loop. This caused the function to return
instead of processing remaining sources.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Makes version headers required for internal endpoints. We also require
version headers for public endpoints when in dev mode.
### Note to reviewers
This PR is a re-revert of the original
https://github.com/elastic/kibana/pull/158667 with some minor additions
(see comments).
The original was reverted due to failing Cypress tests blocking Kibana
promotion for 8.8.1 (CC @stephmilovic,
https://github.com/elastic/kibana/pull/158961)
Not sending headers to versioned, internal endpoints will return 400!
Due to the somewhat sensitive nature of this change, I went through all
of the existing `.versioned` endpoints and tried to ensure that for
_internal_ endpoints we send through a version as this is now
**required**.
I would greatly appreciate it if code owners could check their code,
think of any existing consumers of your versioned endpoints and ensure
they are sending a version.
Closes https://github.com/elastic/kibana/issues/158722
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Pitch: https://github.com/elastic/apm-dev/issues/872 (_internal_)
Introduces the Diagnostics tool under a hidden page:
`/app/apm/diagnostics`. The Diagnostics tool can help find problems with
missing index templates, data streams with incorrect index templates and
indices with incorrect mappings.
It will be released in 8.9 but customers can take advantage of it as
soon as this PR is merged. They can export a diagnostics bundle from
their system by running this script:
```
node x-pack/plugins/apm/scripts/diagnostics_bundle \
--kbHost https://foo.kb.europe-west2.gcp.elastic-cloud.com:9243 \
--esHost https://foo.es.europe-west2.gcp.elastic-cloud.com:9243 \
--username elastic
--password changeme
```
---------
Co-authored-by: Yngrid Coello <yngrdyn@gmail.com>
## Summary
Fixes: #155154 (introduced in #149878), builds on #155436 .
- Adds tests to ensure the configuration merging order, check those for
reference.
- Updates the README to explain the intention
For the tests, I needed to output something to the logs. I hope it's not
a big issue to log it. If needed, I might hide that behind a verbose- or
feature flag.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## [Security Solution] Elastic Security Assistant
The _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.
Bring your alerts, events, rules, and data quality checks into the conversation.
<31d65c78-5692-4817-b726-820c5df0801c>
This PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<https://github.com/elastic/kibana/pull/157228>) , developed by @stephmilovic . This PR includes:
- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`
- See the `How to embed the Assistant in other parts of Kibana` for details
- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)
### An assistant trained on the Elastic stack and Elastic Security
The [Generative AI Connector](<https://github.com/elastic/kibana/pull/157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:
- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)
- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)
- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)
- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)
This training enables the assistant to offer fully interactive chat experiences that include:
- alert summarization
- interactive query generation
- workflow suggestions
- generating ingestion configurations that conform to the Elastic Common Schema
- your imagination
using context from Elastic Security.
### Take action from your conversations
The Actions (from assistant response):
- Send KQL to Timeline
- Send EQL to Timeline
- Send Elasticsearch DSL to Timeline
- Send Note to timeline
- Create new case
- Add to existing case
- Copy to clipboard
### Components architecture diagram
### How to embed the Assistant in other parts of Kibana
Follow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.
#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component
```ts
// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically
// happens in the root of your app. Optionally provide a custom title for the assistant:
/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */
export { AssistantProvider } from './impl/assistant_context';
```
#### Step 2: Add the `AssistantOverlay` component to your app
```ts
// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant
// overlay in a modal, bound to a shortcut key:
/** modal overlay for Elastic Assistant conversations */
export { AssistantOverlay } from './impl/assistant/assistant_overlay';
// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component
// to display the assistant without the modal overlay:
/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */
export { Assistant } from './impl/assistant';
```
#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following
```ts
// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following
// components and hooks:
// - `NewChat` component
// - `NewChatById` component
// - `useAssistantOverlay` hook
/**
* `NewChat` displays a _New chat_ icon button, providing all the context
* necessary to start a new chat. You may optionally style the button icon,
* or override the default _New chat_ text with custom content, like `🪄✨`
*
* USE THIS WHEN: All the data necessary to start a new chat is available
* in the same part of the React tree as the _New chat_ button.
*/
export { NewChat } from './impl/new_chat';
/**
* `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`
* of a context that was (already) registered by the `useAssistantOverlay` hook. You may
* optionally style the button icon, or override the default _New chat_ text with custom
* content, like {'🪄✨'}
*
* USE THIS WHEN: all the data necessary to start a new chat is NOT available
* in the same part of the React tree as the _New chat_ button. When paired
* with the `useAssistantOverlay` hook, this option enables context to be be
* registered where the data is available, and then the _New chat_ button can be displayed
* in another part of the tree.
*/
export { NewChatById } from './impl/new_chat_by_id';
/**
* `useAssistantOverlay` is a hook that registers context with the assistant overlay, and
* returns an optional `showAssistantOverlay` function to display the assistant overlay.
* As an alterative to using the `showAssistantOverlay` returned from this hook, you may
* use the `NewChatById` component and pass it the `promptContextId` returned by this hook.
*
* USE THIS WHEN: You want to register context in one part of the tree, and then show
* a _New chat_ button in another part of the tree without passing around the data, or when
* you want to build a custom `New chat` button with features not not provided by the
* `NewChat` component.
*/
export { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';
```
Co-authored-by: Garrett Spong <garrett.spong@elastic.co>
Co-authored-by: Andrew Macri <andrew.macri@elastic.co>
## Summary
issue: https://github.com/elastic/kibana/issues/158810
follow-up of: https://github.com/elastic/kibana/pull/158179
Improves the Security AppFeatures architecture:
- SubFeatures now preserve always the same order in the Security Kibana
config, despite the order of processing of enabled appFeatures.
- Change the `productTypes` config format
- Update `getProductAppFeatures` to:
- process the new `productTypes` format.
- include _essentials_ tiers PLIs inside _complete_ tiers automatically.
- AppFeatures module now receives an array of PLIs instead of an object
- AppFeatures config now uses only SubFeature IDS instead of
`subActions` config objects directly
- Upselling components updated and `useProductTypeByPLI` implemented to
display the Product Type required
## Summary
Original issue: https://github.com/elastic/kibana/issues/154842
Collaboration with @cnasikas
In dashboard, when clicking on `...` of each chart, we should be able
to:
- [x] - Add to New case
- [x] - Add to existing case
- [x] - Create case from existing case modal
- [x] - Create case from existing case modal when no existing cases
512b32f9-10df-4f8b-a456-08f28a987826
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>
## Summary
As part of the Serverless work in
https://github.com/elastic/kibana/issues/153726, we validate all of our
responses returned by the Cases APIs. The GET case users internal API
validates the schema of the `avatar` persisted in the `data` of the user
profile. It may be possible to set the `imageUrl` to `null` from within
the user profile page. This PR fixes this bug. It also updates the type
of the `imageUrl` in `UserProfileAvatarData`.
## Testing
1. Go to your user profile and press save
2. Go to a case
3. You should see an error toaster. The fix in this RP should eliminate
the error toaster and the Case view page should be shown properly
4. Go to your user profile and change the initials the color and set a
profile image. Go again to a case and verify that the avatar is shown as
expected
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
[Documentation](https://docs.google.com/document/d/1Ms8d8d_fbTTRHlBroEAKGNMNk3jFFgOAkVDRhqLxAPQ/edit?pli=1#)
issue: https://github.com/elastic/kibana/issues/158810
## Summary
This PR is a cleanup to make [this
POC](https://github.com/elastic/kibana/pull/155420) production ready
- Serverless PLI features splitting in Security Solution, to allow/deny
access to configured functionalities, using the current Kibana RBAC
service.
- Create the Upselling service to display Serveless-specific prompts in
the application when features are not available
- Create a `SecurityRoutePageWrapper` component that wraps Pages and
displays the upsell when necessary.
- We will refactor the code base to use `SecurityRoutePageWrapper`
everywhere on another PR.
- Create an Upsell page and section for entity analytics
bd8db822-2f4b-4545-9da7-bedc07d93f90
### test:
Serverless: `yarn serverless-security`.
* To change the product line you have to update
`xpack.serverless.security.productLineIds` on
`config/serverless.security.yml`.
ESS: `yarn start`
### Glossary
* PLI - Product Line Item (`Alert Triage`, `Osquery`, `Cases` , ... )
* Product Line - The product that the user is subscribed to (Security
Essentials, Security Complete, ...)
* essSecurity - New plugin with code that only runs for ESS offer
(non-serverless).
* App Feature - A security solution feature or group of features that
can be disabled for a product line. It can be mapped to PLIs (`Alert
Triage`, `Osquery`, `Cases` , ... ).
* Capability - A string that when present represents that the user can
access a given feature. A capability could be of the type UI or API
(`read_cases`, `crud_cases`, ...).
### Current architecture
### New architecture
### How does it work?
Every serverless product line (endpointEssentials, cloud essentials) can
define which features are enabled:
69d0fc15f4/x-pack/plugins/serverless_security/common/pli/pli_config.ts (L12-L19)
For ESS (non-serverless) offer we enable all features by default.
69d0fc15f4/x-pack/plugins/ess_security/server/constants.ts (L10-L13)
A feature can define privileges:
69d0fc15f4/x-pack/plugins/security_solution/server/lib/app_features/security_kibana_features.ts (L177-L185)
When the feature is enabled the privileges get merged into the base
config and injected into kibana features.
69d0fc15f4/x-pack/plugins/security_solution/server/lib/app_features/app_features.ts (L61-L70)
### TODO
- [x] lazy load these components
- [x] Add unit test to:
- ~SecurityRoutePageWrapper
x-pack/plugins/security_solution/public/common/components/security_route_page_wrapper/index.tsx~
-
~x-pack/plugins/security_solution/public/common/hooks/use_upselling.ts~
-
~x-pack/plugins/security_solution/public/common/lib/capabilities/has_capabilities.ts~
-
~x-pack/plugins/security_solution/public/common/lib/upsellings/upselling_service.ts~
- ~x-pack/plugins/serverless_security/common/pli/pli_features.ts~
-
~x-pack/plugins/serverless_security/public/components/upselling/register_upsellings.tsx~
-
~x-pack/plugins/security_solution/server/lib/app_features/app_features.ts~
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Addresses the following feedback:
https://github.com/elastic/kibana/pull/154151#discussion_r1158470566
Similar to what has been done for ZDT, the goal of this PR is to extract
the logic of the `runV2Migration()` from the `KibanaMigrator` into a
separate file.
The PR also fixes some incomplete / incorrect UTs and adds a few missing
ones.
The existing `build_api_docs` script takes about ~10m to run and
generate the API docs. That's not too bad for the daily CI task that
generates the docs. However, even if you're just interested in getting
some stats on a single plugin/package it runs the full script. We often
have to run this script locally to check the stats in detail to improve
JSDoc and missing exports which is quite cumbersome if it takes this
long to run after each code change.
This PR updates the script in the following way: If both the options
`--stats` and `--plugin` (with a single plugin/package) are set, it will
not generate a TS project for the whole Kibana codebase but just the
code related to that plugin/package. API docs will then not be written
to avoid inconsistencies and just the stats will be logged.
Depending on the size of the package/plugin this can reduce the time
needed to run the script down to a few seconds.
## Summary
Add validation to ensure that mappings added via a `mappings_addition`
type change are also present in the type's full mappings.
(just an extra layer of protection for when the teams will switch to
using the new API)
## Summary
Fixes an issue with the field browser where all types currently display
as unkown, this was because in a code path where a type cast happens, we
were using the wrong type. To see this, remove the as unknown from the
cast, and the typescript compiler will show the problem:
```
'BrowserField' is deprecated.ts(6385)
index.ts(70, 4): The declaration was marked as deprecated here.
Conversion of type 'DataViewField' to type 'BrowserField' may be a mistake because neither type sufficiently overlaps with the other. If this was intentional, convert the expression to 'unknown' first.
Type 'DataViewField' is missing the following properties from type 'BrowserField': category, description, example, fields, and 2 more.ts(2352)
```
DataViewField actually only has spec and kbnFieldType properties, spec
is of type FieldSpec which is basically the same type as BrowserField,
and has sufficient overlap for the (still unsafe, but more safe than as
unknown) cast to occur.
Before:
<img width="338" alt="image"
src="f31c1f9e-25f0-41ee-9e1c-a70171e41d29">
After:
<img width="555" alt="image"
src="8b462477-2dce-41bb-9592-f34b20634b84">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
`@elastic/eui@80.0.0` ⏩ `@elastic/eui@81.0.0`
---
## [`81.0.0`](https://github.com/elastic/eui/tree/v81.0.0)
- Added ability to set `options.checked` to "mixed" in `EuiSelectable`
([#6774](https://github.com/elastic/eui/pull/6774))
**Bug fixes**
- Portalled components (e.g. `EuiPopover`, `EuiModal`, `EuiFlyout`) will
correctly inherit text color from its nearest `EuiThemeProvider` parent.
`<EuiText color="default">` is no longer needed.
([#6775](https://github.com/elastic/eui/pull/6775))
**Breaking changes**
- `EuiSelectable` no longer renders a `data-test-selected` attribute on
its list items. Use the `aria-checked` property instead
([#6774](https://github.com/elastic/eui/pull/6774))
- Nested `EuiThemeProvider`s now render a wrapping `<span>` element in
order to correctly set the inherited text `color` of all descendants.
`<EuiText color="default">` is no longer needed.
([#6775](https://github.com/elastic/eui/pull/6775))
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Constance Chen <constance.chen@elastic.co>
closes https://github.com/elastic/kibana/issues/154147
## Summary
This PR introduces `io-ts` runtime validation to asset manager
endpoints.
The parameters will be validated as follows:
- `maxDistance`: range from 1 to 5
- `size`: range from 1 to 100
- `type`: `k8s.pod` or `k8s.cluster` or `k8s.node`
- `relation`: `ancestors` or `descendants` or `references`
- any date parameter: Date ISO or DateMath formats
### How to test this PR
- enable `xpack.assetManager.alphaEnabled` in kibana.dev.yml
- Call the endpoints below and try to pass invalid parameters where
they're applicable:
- `GET
kbn:/api/asset-manager/assets?from=2023-03-25T17:44:44.000Z&to=2023-03-25T18:44:44.000Z`
- `GET
kbn:/api/asset-manager/assets/diff?aFrom=2022-02-07T00:00:00.000Z&aTo=2022-02-07T01:30:00.000Z&bFrom=2022-02-07T01:00:00.000Z&bTo=2022-02-07T02:00:00.000Z`
- `GET
kbn:/api/asset-manager/assets/related?ean=k8s.node:node-101&relation=ancestors&maxDistance=1&from=2023-04-18T13:10:13.111Z&to=2023-04-18T15:10:13.111Z`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Tackles https://github.com/elastic/kibana/issues/155136
When an upgrade fails, a cluster might be on a partially migrated state
(with some indices already updated to the newer version). When rolling
back to the previous version, in ESS, this can cause these indices to be
`write_blocked`.
This PR aims at detecting this situation and failing early, effectively
preventing to `write_block` any indices.
## Summary
Closes https://github.com/elastic/kibana/issues/154334
This PR:
- make the text based languages to work with adhoc dataviews and not
permanent dataviews. The text based languages will not be related with
dataviews
- enables the timepicker always for text based languages
- the timepicker is disabled if the index pattern doesn't have an
@timestamp field
- the timepicker is enabled if the index pattern has an @timestamp field
- the timepicker is enabled if the index pattern doesn't have an
@timestamp field but there is a dirty state (user is writing the query
and hasn't hit the update button. We do that to give the user the
ability to change the timepicker with the query
- An info text has been added to the editor footer to inform the users
about the @timestamp existence
The timepicker in the disabled state needs to have a disabled status
text (All time) but this is not possible atm. I have created an issue to
eui https://github.com/elastic/eui/issues/6814 to add this property.
This is going to be tackled before the 8.9 FF but we don't want to block
this PR
<img width="1839" alt="image"
src="8fc0a492-1f00-41b6-a4a6-b0527725931f">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Related to https://github.com/elastic/kibana/issues/150296
Use a `range` clause for version filtering for the ZDT outdated doc
query, similar to what is already done for the `v2` algo, to avoid
eventually trying to migrate documents from higher versions (which would
result in a migration failure)
This PR addresses Cypress parallelisation in Security Solution scope. It
is a first step in improving e2e tests reliability and gives us a solid
foundation for further work on flakiness and run times. With this PR
each CI job starts 3 instances of ES, kibana and Cypress runner in
isolation.
Other issues addressed in this PR:
- Use click() instead of trigger('click') in all Cypress tests.
- Use testIsolation in all Cypress tests.
- login before each test to ensure that the correct user is always
logged in. Also, with test isolation enabled, login should be required
before each test.
- use visit() instead of cy.visit() in some cases so that tests wait for
the page to fully load before executing.
- Cypress e2e tests are no longer compatible with Firefox (or any
non-Chromium based browser) due to using cypress-real-events
- Removes uses of cypress-pipe. Tests that used cypress-pipe in order to
retry click events will no longer retry click events.
- Fixes an error related to test files being overwritten by Webpack file
processing
---------
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Migrations read 1000 documents by default which works well for most
deployments. But if any batch happens to be > ~512MB we hit NodeJS' max
string length limit and cannot process that batch. This forces users to
reduce the batch size to a smaller number which could severely slow down
migrations.
This PR reduces the impact of large batches by catching
elasticsearch-js' `RequestAbortedError` and reducing the batch size in
half. When subsequent batches are successful the batchSize increases by
20%. This means we'll have a sequence like:
1. Read 1000 docs ✅ (small batch)
2. Read 1000 docs 🔴 (too large batch)
3. Read 500 docs ✅
4. Read 600 docs ✅
5. Read 720 docs ✅
6. Read 864 docs ✅
7. Read 1000 docs ✅ (small batch)
This assumes that most clusters just have a few large batches exceeding
the limit. If all batches exceed the limit we'd have 1 failure for every
4 successful reads so we pay a 20% throughput penalty. In such a case it
would be better to configure a lower `migrations.batchSize`.
Tested this manually:
1. Start ES with more heap than the default, otherwise reading large
batches will cause it to run out of memory
`ES_JAVA_OPTS=' -Xms6g -Xmx6g' yarn es snapshot
--data-archive=/Users/rudolf/dev/kibana/src/core/server/integration_tests/saved_objects/migrations/archives/8.4.0_with_sample_data_logs.zip`
2. Ingest lots of large documents of ~5mb
```
curl -XPUT
"elastic:changeme@localhost:9200/_security/role/grant_kibana_system_indices"
-H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
"indices": [
{
"names": [
".kibana*"
],
"privileges": [
"all"
],
"allow_restricted_indices": true
}
]
}'
curl -XPOST "elastic:changeme@localhost:9200/_security/user/superuser"
-H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
"password" : "changeme",
"roles" : [ "superuser", "grant_kibana_system_indices" ]
}'
curl -XPUT
"superuser:changeme@localhost:9200/.kibana_8.4.0_001/_mappings" -H
"kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
"dynamic": false,
"properties": {
}
}'
set -B # enable brace expansion
for i in {1..400}; do
curl -k --data-binary
"@/Users/rudolf/dev/kibana/src/core/server/integration_tests/saved_objects/migrations/group3/body.json"
-X PUT
"http://superuser:changeme@localhost:9200/.kibana_8.4.0_001/_doc/cases-comments:"{$i}"?&pretty=true"
-H "Content-Type: application/json"
done
```
3. Start Kibana with a modest batchSize otherwise we could OOM ES `node
scripts/kibana --dev --migrations.batchSize=120`
<details><summary>Example logs. Note the "Processed x documents" only
logs when the next batch is successfull read, so the order seems wrong.
To improve it we'd need to log progress after a batch is successfully
written instead 🤷 </summary>
```
[.kibana] Processed 120 documents out of 542.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_TRANSFORM. took: 3667ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_TRANSFORM -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1740ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1376ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1402ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1311ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1388ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_READ. took: 900ms.
[.kibana] Read a batch that exceeded the NodeJS maximum string length, retrying by reducing the batch size in half to 60.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_READ. took: 1538ms.
[.kibana] Processed 240 documents out of 542.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_TRANSFORM. took: 2054ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_TRANSFORM -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1042ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1310ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1388ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_READ. took: 1130ms.
[.kibana] Processed 300 documents out of 542.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_TRANSFORM. took: 2610ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_TRANSFORM -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1262ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1299ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1363ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1341ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_READ. took: 572ms.
[.kibana] Processed 372 documents out of 542.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_TRANSFORM. took: 3330ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_TRANSFORM -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1488ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1349ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1312ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1380ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1310ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_READ. took: 139ms.
[.kibana] Processed 458 documents out of 542.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_TRANSFORM. took: 3278ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_TRANSFORM -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1460ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1370ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1303ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_INDEX_BULK. took: 1384ms.
[.kibana] REINDEX_SOURCE_TO_TEMP_INDEX_BULK -> REINDEX_SOURCE_TO_TEMP_READ. took: 1298ms.
[.kibana] Processed 542 documents out of 542.
[.kibana] REINDEX_SOURCE_TO_TEMP_READ -> REINDEX_SOURCE_TO_TEMP_CLOSE_PIT. took: 4ms.
```
</details>
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risks
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Gerard Soldevila <gerard.soldevila@elastic.co>
## Summary
Fix https://github.com/elastic/kibana/issues/158372
Change the logic of the `zdt` algorithm to be able to re-use an existing
"v2 managed" cluster state (a kibana cluster previously running the v2
migration algorithm) if compatible.
Technically the v2 index state is compatible with zdt after 8.8 (when we
introduced `indexTypeMap` and restricted to compatible mapping changes
only).
See https://github.com/elastic/kibana/issues/158372 for more detailed
explanations.
## Summary
Adds build date to `GET kbn:api/status` similar to ES. Example output
running locally:
```json
{
"name": "ES-DMVD5M3",
"uuid": "545ba70c-063e-449b-af21-6c8e7b30f77e",
"version": {
"number": "8.9.0",
"build_hash": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"build_number": 9007199254740991,
"build_snapshot": false,
"build_date": "2023-05-15T23:12:09.000Z"
},
...rest
}
```
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Release Note
The status endpoint now returns the build date alongside other build
information.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#154937
This PR is an extension of
[#157802](https://github.com/elastic/kibana/pull/157802) which was
reverted because public Api endpoint versioning problems.
---------
Co-authored-by: Oliver Gupte <oliver.gupte@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
