# Backport
This will backport the following commits from `main` to `8.13`:
- [[Search] Fix native connector search results
(#178003)](https://github.com/elastic/kibana/pull/178003)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Navarone
Feekery","email":"13634519+navarone-feekery@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-03-06T08:34:12Z","message":"[Search]
Fix native connector search results (#178003)\n\n## Summary\r\n\r\nThis
fixes the search result to redirect to native connector pages if\r\nthe
user is on cloud and the connector supports native. Otherwise
the\r\nconnector page will be for a connector client.\r\n\r\nAlso
consolidate the reference for connector integrations used
for\r\nsearching the top search bar and the integrations
page.\r\nThere's a lot of repetitiveness here, and because they are very
similar\r\nin both use and design, it is ideal to have a single source
of
truth.","sha":"9262d96788dbf23c20f29b8f778accd862d07ca6","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:EnterpriseSearch","v8.13.0","v8.14.0"],"title":"[Search]
Fix native connector search
results","number":178003,"url":"https://github.com/elastic/kibana/pull/178003","mergeCommit":{"message":"[Search]
Fix native connector search results (#178003)\n\n## Summary\r\n\r\nThis
fixes the search result to redirect to native connector pages if\r\nthe
user is on cloud and the connector supports native. Otherwise
the\r\nconnector page will be for a connector client.\r\n\r\nAlso
consolidate the reference for connector integrations used
for\r\nsearching the top search bar and the integrations
page.\r\nThere's a lot of repetitiveness here, and because they are very
similar\r\nin both use and design, it is ideal to have a single source
of
truth.","sha":"9262d96788dbf23c20f29b8f778accd862d07ca6"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/178003","number":178003,"mergeCommit":{"message":"[Search]
Fix native connector search results (#178003)\n\n## Summary\r\n\r\nThis
fixes the search result to redirect to native connector pages if\r\nthe
user is on cloud and the connector supports native. Otherwise
the\r\nconnector page will be for a connector client.\r\n\r\nAlso
consolidate the reference for connector integrations used
for\r\nsearching the top search bar and the integrations
page.\r\nThere's a lot of repetitiveness here, and because they are very
similar\r\nin both use and design, it is ideal to have a single source
of truth.","sha":"9262d96788dbf23c20f29b8f778accd862d07ca6"}}]}]
BACKPORT-->
Co-authored-by: Navarone Feekery <13634519+navarone-feekery@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `8.13`:
- [Add tiles and integrations for Jira Data Center and Confluence Data
Center (#177436)](https://github.com/elastic/kibana/pull/177436)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Artem
Shelkovnikov","email":"artem.shelkovnikov@elastic.co"},"sourceCommit":{"committedDate":"2024-02-22T00:49:21Z","message":"Add
tiles and integrations for Jira Data Center and Confluence Data Center
(#177436)\n\n## Summary\r\n\r\nThis PR adds tiles for two
connectors:\r\n\r\n- Jira Data Center\r\n- Confluence Data
Center\r\n\r\nThese connectors internally use same service types: `jira`
and\r\n`confluence`. They re-use documentation for these connectors - it
can be\r\nfound in the same pages as for original Jira/Confluence
connectors.\r\n\r\nThe tiles are added to Enterprise Search connectors
page + into\r\nintegrations page. See screenshots.\r\n\r\n<img
width=\"1217\" alt=\"Screenshot 2024-02-21 at 14 10
20\"\r\nsrc=\"da944de9-828b-453b-ad38-1695ae3d557a\">\r\n\r\n<img
width=\"1217\" alt=\"Screenshot 2024-02-21 at 14 11
09\"\r\nsrc=\"aaac975e-e423-42a3-846b-628e9e673b5a\">\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [x] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\\\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c2dedc957a029c7b51a2abad4e4d79922d5d77ea","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["backport","release_note:feature","backport:prev-minor","v8.13.0","v8.14.0"],"title":"Add
tiles and integrations for Jira Data Center and Confluence Data
Center","number":177436,"url":"https://github.com/elastic/kibana/pull/177436","mergeCommit":{"message":"Add
tiles and integrations for Jira Data Center and Confluence Data Center
(#177436)\n\n## Summary\r\n\r\nThis PR adds tiles for two
connectors:\r\n\r\n- Jira Data Center\r\n- Confluence Data
Center\r\n\r\nThese connectors internally use same service types: `jira`
and\r\n`confluence`. They re-use documentation for these connectors - it
can be\r\nfound in the same pages as for original Jira/Confluence
connectors.\r\n\r\nThe tiles are added to Enterprise Search connectors
page + into\r\nintegrations page. See screenshots.\r\n\r\n<img
width=\"1217\" alt=\"Screenshot 2024-02-21 at 14 10
20\"\r\nsrc=\"da944de9-828b-453b-ad38-1695ae3d557a\">\r\n\r\n<img
width=\"1217\" alt=\"Screenshot 2024-02-21 at 14 11
09\"\r\nsrc=\"aaac975e-e423-42a3-846b-628e9e673b5a\">\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [x] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\\\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c2dedc957a029c7b51a2abad4e4d79922d5d77ea"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/177436","number":177436,"mergeCommit":{"message":"Add
tiles and integrations for Jira Data Center and Confluence Data Center
(#177436)\n\n## Summary\r\n\r\nThis PR adds tiles for two
connectors:\r\n\r\n- Jira Data Center\r\n- Confluence Data
Center\r\n\r\nThese connectors internally use same service types: `jira`
and\r\n`confluence`. They re-use documentation for these connectors - it
can be\r\nfound in the same pages as for original Jira/Confluence
connectors.\r\n\r\nThe tiles are added to Enterprise Search connectors
page + into\r\nintegrations page. See screenshots.\r\n\r\n<img
width=\"1217\" alt=\"Screenshot 2024-02-21 at 14 10
20\"\r\nsrc=\"da944de9-828b-453b-ad38-1695ae3d557a\">\r\n\r\n<img
width=\"1217\" alt=\"Screenshot 2024-02-21 at 14 11
09\"\r\nsrc=\"aaac975e-e423-42a3-846b-628e9e673b5a\">\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [x] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\\\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"c2dedc957a029c7b51a2abad4e4d79922d5d77ea"}}]}]
BACKPORT-->
Co-authored-by: Artem Shelkovnikov <artem.shelkovnikov@elastic.co>
# Backport
This will backport the following commits from `main` to `8.13`:
- [Add tiles for Notion and Redis connector
(#177306)](https://github.com/elastic/kibana/pull/177306)
<!--- Backport version: 9.4.3 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Artem
Shelkovnikov","email":"artem.shelkovnikov@elastic.co"},"sourceCommit":{"committedDate":"2024-02-21T17:50:08Z","message":"Add
tiles for Notion and Redis connector (#177306)\n\n## Summary\r\n\r\nThis
PR adds tiles for two new connectors:\r\n\r\n- Redis\r\n-
Notion\r\n\r\nThe tiles are added to Enterprise Search connectors page +
into\r\nintegrations page. See screenshots.\r\n\r\n<img width=\"1217\"
alt=\"image\"\r\nsrc=\"17bac917-f636-477e-91ae-facc48b3c06c\">\r\n\r\n<img
width=\"1217\"
alt=\"image\"\r\nsrc=\"f56f44bf-25a9-464c-b84b-f71947d7a3c3\">\r\n\r\n<img
width=\"1217\"
alt=\"image\"\r\nsrc=\"329a3a7e-da1b-49b3-902f-75421fb9e289\">\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [x] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"81bccc5a8130f8d0c5641ecc6f5a4076281332d4","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:feature","v8.13.0","v8.14.0"],"title":"Add
tiles for Notion and Redis
connector","number":177306,"url":"https://github.com/elastic/kibana/pull/177306","mergeCommit":{"message":"Add
tiles for Notion and Redis connector (#177306)\n\n## Summary\r\n\r\nThis
PR adds tiles for two new connectors:\r\n\r\n- Redis\r\n-
Notion\r\n\r\nThe tiles are added to Enterprise Search connectors page +
into\r\nintegrations page. See screenshots.\r\n\r\n<img width=\"1217\"
alt=\"image\"\r\nsrc=\"17bac917-f636-477e-91ae-facc48b3c06c\">\r\n\r\n<img
width=\"1217\"
alt=\"image\"\r\nsrc=\"f56f44bf-25a9-464c-b84b-f71947d7a3c3\">\r\n\r\n<img
width=\"1217\"
alt=\"image\"\r\nsrc=\"329a3a7e-da1b-49b3-902f-75421fb9e289\">\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [x] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"81bccc5a8130f8d0c5641ecc6f5a4076281332d4"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/177306","number":177306,"mergeCommit":{"message":"Add
tiles for Notion and Redis connector (#177306)\n\n## Summary\r\n\r\nThis
PR adds tiles for two new connectors:\r\n\r\n- Redis\r\n-
Notion\r\n\r\nThe tiles are added to Enterprise Search connectors page +
into\r\nintegrations page. See screenshots.\r\n\r\n<img width=\"1217\"
alt=\"image\"\r\nsrc=\"17bac917-f636-477e-91ae-facc48b3c06c\">\r\n\r\n<img
width=\"1217\"
alt=\"image\"\r\nsrc=\"f56f44bf-25a9-464c-b84b-f71947d7a3c3\">\r\n\r\n<img
width=\"1217\"
alt=\"image\"\r\nsrc=\"329a3a7e-da1b-49b3-902f-75421fb9e289\">\r\n\r\n\r\n###
Checklist\r\n\r\nDelete any items that are not applicable to this
PR.\r\n\r\n- [x] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"81bccc5a8130f8d0c5641ecc6f5a4076281332d4"}}]}]
BACKPORT-->
Co-authored-by: Artem Shelkovnikov <artem.shelkovnikov@elastic.co>
## Summary
This PR introduces a number of changes and improvements to saved query
management:
- Add server side pagination (5 queries per page) and search
functionality to the "Load query" list, which improves UX and
performance by no longer requesting all queries at once.
- Redesign the "Load query" list to improve the UX and a11y, making it
possible for keyboard users to effectively navigate the list and
load/delete queries.
- Add an "Active" badge to the "Load query" list to indicate which list
entry represents the currently loaded query, and hoist the entry to the
top of the first page for better visibility when no search term exists.
- Deprecate the saved query `/_all` endpoint and update it to return
only the first 100 queries instead of loading them all into memory at
once.
- Add a new `titleKeyword` field to the saved query SO, which allows
sorting queries alphabetically by title when displaying them in the
"Load query" list.
- Improve the performance of the "has saved queries" check when Unified
Search is mounted to no longer request actual queries, and instead just
request the count.
- Update the saved query duplicate title check to no longer rely on
fetching all queries at once, and instead asynchronously check for
duplicates by title on save.
- Add server side duplicate title validation to the create and update
saved query endpoints.
- Various small fixes and cleanups throughout saved query management.
43328aea-0f7b-4b7a-a5fb-e33ed822f317
Resolves#172044.
Resolves#176427.
## Testing
To generate saved queries for testing, run the script below and replace
`{KIBANA_REQUEST_COOKIE}` with the cookie header value from an API
request of a Kibana user with an active session:
```shell
for i in {1..100}; do curl 'http://localhost:5601/internal/saved_query/_create' \
-H 'Accept: */*' \
-H 'Accept-Language: en-US,en;q=0.9,az;q=0.8,es;q=0.7' \
-H 'Cache-Control: no-cache' \
-H 'Connection: keep-alive' \
-H 'Content-Type: application/json' \
-H 'Cookie: {KIBANA_REQUEST_COOKIE}' \
-H 'elastic-api-version: 1' \
-H 'kbn-build-number: 9007199254740991' \
-H 'kbn-version: 8.13.0' \
-H 'x-elastic-internal-origin: Kibana' \
--data-raw '{"title":"query '"$(echo $(($i - 1)) | tr '[0-9]' '[a-j]')"'","description":"","query":{"query":"bytes > 500","language":"kuery"},"filters":[]}' \
--compressed; done
```
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
Fixes https://github.com/elastic/kibana/issues/176227
As not everyone is using expressions and the `?drop_null_columns`
significantly changing the output, I am making this configurable. So for
the expressions users it will always be true but for anyone else it
would be false unless they specifically configure it.
This PR is fixing it but Nathan is going to follow up:
- maps are setting the drop_null_columns to true (as it makes the
response smaller, which is always a benefit)
- add some FTs to test the output
---------
Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
## Summary
Closes: https://github.com/elastic/kibana/issues/169622
Field caps request caching - implement `stale-while-revalidate` cache
headers and etags with 304 responses as appropriate.
This PR accomplishes
- Adds 304 Not Modified to http server
- Adds refresh button to data view management that force refreshes field
list
- Adds `/internal/data_views/fields` endpoint which supports caching.
- This is necessary since `fields_for_wildcard` doesn't support caching
due to POST requests
- Adds `stale-while-revalidate` header directive UNLESS field list is
empty.
- Uses Vary header with hash of user id to force requests when user has
changed.
- Unchanged field list responses won't recreate data view field list
### How to test
1. Pop open the dev tools to the network tab and make sure 'Disable
cache' is unchecked. filter for 'fields' requests
2. Load more than one data set (sample data is fine)
3. Go to discover
4. Switch selected data views. Notice status code on first load vs
subsequent loads
5. Open a new window, notice loading from cache
6. Push document that changes field list. You'll need to wait for cache
to expire for it to load. The default is 5s.
7. Notice that field list loads after cache is old properly result in
304 response.
8. Set `data_views:cache_max_age`, shift reload. No field requests will
be cached.
#### Note on Safari
Safari doesn't support the `stale-while-revalidate` directive.
Additionally, the Safari dev console shows 304 responses as 200's. I
figured this out by adding console statements to the fields endpoint. As
best I can tell, Safari won't be performant on slow clusters but its
also no slower than it was before this PR. Safari does respect the
disabling of cache headers.
## Release note
Data View field list requests are now cached with a
`stale-while-revalidate` strategy. This means that after the initial
field list request, all subsequent requests return the cached response
which is very fast. If the cache is determined to be stale then the
cache will update in the background and new data will be available on
the next request.
This behavior can be modified via the `data_views:cache_max_age` Kibana
advanced setting. Setting it to zero will disable the cache. All other
values (in seconds) will be used to determine whether the cache is
stale. The default value is 5 seconds.
The field list can be manually updated via the refresh button in data
view management or a hard refresh with your browser.
Note for Safari: The `stale-while-revalidate` cache directive is
unsupported, therefore it makes additional requests. If this is
impacting Kibana performance then try Chrome or Firefox.
Data View Management
<img width="1064" alt="Screenshot 2024-01-09 at 1 36 20 PM"
src="f272c19f-81b4-4697-9303-b1f8f150e2b9">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: amyjtechwriter <61687663+amyjtechwriter@users.noreply.github.com>
Co-authored-by: Julia Rechkunova <julia.rechkunova@elastic.co>
Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
Fixes https://github.com/elastic/kibana/issues/168732
## Summary
This PR changes the default value of the `defaultIndex` setting to an
empty string. Previously, the default value was set to `null` and this
was causing a wrong behaviour in the Advanced Settings UI as it expects
a string value for the settings of type `string`.
**Before - the "Reset to default" link did not clear the user value:**
2c47306b-feac-4e08-bbae-3db7b0a3e5fd
**With these changes:**
dc46caa8-9860-4b66-87ca-9c184c5a8391
Closes https://github.com/elastic/kibana/issues/45931
PR updates data plugin `search` and `bsearch` endpoints to return
method, path, and querystring request params from elasticsearch-js
client requests. This provides inspector with the exact details used to
fetch data from elasticsearch, ensuring inspector displays request
exactly as used by elasticsearch-js client.
**ESQL** This PR makes it possible to open ESQL searches in console.
<img width="500" alt="Screen Shot 2023-09-16 at 4 19 58 PM"
src="56019fb5-ca88-46cf-a42f-86f5f51edfcc">
### background
If you are thinking to yourself, "haven't I reviewed this before?", you
are right. This functionality has been through several iterations.
1) Original PR https://github.com/elastic/kibana/pull/166565 was
reverted for exposing `headers`.
2) [Fix to only expose method, path, and querystring keys from request
parameters](https://github.com/elastic/kibana/pull/167544) was rejected
because it applied changes to
`kibana_utils/server/report_server_error.ts`, which is used extensively
throughout Kibana.
3) This iteration moves logic into the data plugin to be as narrow as
possible.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/167099
#### Problem
`/bsearch` and `/search` APIs only return `error` key from elasticsearch
error response. This is problematic because Inspector needs
`rawResponse` to populate "Clusters and shards"
While working on this issue, I discovered another problem with how error
responses are added to inspector requestResponder. The `Error` instance
is added as `json` key. This is a little awkward since the response tab
just stringifies the contents of `json`, thus stringifing the Error
object instead of just the error body returned from API. This PR address
this problem by setting `json` to either `attributes` or `{ message }`.
#### Solution
PR updates `/bsearch` and `/search` APIs to return `{ attributes: {
error: ErrorCause, rawResponse }}` for failed responses. Solution
avoided changing KbnServerError and reportServerError since these
methods are used extensivly throughout Kibana (see
https://github.com/elastic/kibana/pull/167544#discussion_r1342460941 for
more details). Instead, KbnSearchError and reportSearchError are created
to report search error messages.
#### Test
1) install web logs sample data set
2) open discover
3) add filter
```
{
"error_query": {
"indices": [
{
"error_type": "exception",
"message": "local shard failure message 123",
"name": "kibana_sample_data_logs",
"shard_ids": [
0
]
}
]
}
}
```
4) Open inspector. Verify "Clusters and shards" tab is visible and
populated. Verify "Response" tab shows "error" and "rawResponse" keys.
<img width="500" alt="Screenshot 2023-10-09 at 9 29 16 AM"
src="461b0eb0-0502-4d48-a487-68025ef24d35">
<img width="500" alt="Screenshot 2023-10-09 at 9 29 06 AM"
src="9aff41eb-f771-48e3-a66d-1447689c2c6a">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Gloria Hornero <gloria.hornero@elastic.co>
Closes https://github.com/elastic/kibana/issues/45931
PR updates bsearch service to return request params from
elasticsearch-js client requests. This provides inspector with the exact
details used to fetch data from elasticsearch, ensuring inspector
displays request exactly as used by elasticsearch-js client.
**ESQL** This PR makes it possible to open ESQL searches in console.
<img width="500" alt="Screen Shot 2023-09-16 at 4 19 58 PM"
src="56019fb5-ca88-46cf-a42f-86f5f51edfcc">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR removes the `systemIndicesSuperuser` auth in the `esSupertest`
service for serverless test runs, adds `certificateAuthorities` to the
Elasticsearch server config in serverless and adds a tiny test to verify
functionality of this fix.
### Details
Issues before this PR when using the `esSupertest` service in serverless
tests (can be reproduced by running the added `elasticsearch_api` test
without the supertest and config changes):
1. Running against a local `functional_tests_server`, `esSupertest`
produces an error: `Error: self-signed certificate in certificate chain`
2. Running against an MKI project produces an error: `unable to
authenticate user [system_indices_superuser] for REST request [/]`,
because the `system_indices_superuser` doesn't exist in MKI.
How this PR addresses the issues:
1. Add `certificateAuthorities` to `servers.elasticsearch` in
`x-pack/test_serverless/shared/config.base.ts` in the same way we
already have it for `servers.kibana`
2. Modify the `esSUpertest` service to not override the default ES auth
when running in serverless, instead go with the default auth (regular
superuser), which is the best we can get.
### Additional information
It has been considered to add a serverless specific version of
`esSupertest` in order to avoid the `config.get('serverless') ?` code.
The fact that a number of stateful services are planned to be re-used in
serverless and rely on `esSupertest` in combination with the very small
change in a central service made it seem worth to make an exception
here.
Note that service methods or tests that use `esSupertest` can still run
into issues on serverless if they actually try to modify system indices.
This should be avoided anyways and particularly for serverless tests.
When in data view management an exact match index pattern is entered (without wildcard), there's now a status code of 404 instead of 500 returned, and no more error message logged.
## Summary
I'm waiting for this to fail CI - this means that a new ES snapshot is
being used. Then we update the test snapshot and should be good to
merge.
The lastest ES snapshot is returning field types in a different order
which really doesn't matter to this endpoint so I'll just update the
snapshot.
Closes https://github.com/elastic/kibana/issues/164753
## Summary
Resolves https://github.com/elastic/kibana/issues/153497.
Updates the saved query service to properly handle & return errors from
the saved object client. Instead of displaying "internal server error"
and returning 500, specific error messages occur for corresponding saved
object client errors.
After:
### To do
- [x] API integration tests
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Some simple dev UX improvements to the swap_references data views api -
```
POST /api/data_views/swap_references/_preview
{
"fromId" : "abcd-efg",
"toId" : "xyz-123"
}
returns
{
result: [{ id: "123", type: "visualization" }],
}
```
```
POST /api/data_views/swap_references
{
"fromId" : "abcd-efg",
"toId" : "xyz-123",
"delete" : true // optional, removes data view which is no longer referenced
}
returns
{
result: [{ id: "123", type: "visualization" }],
deleteStatus: {
remainingRefs: 0,
deletePerformed: true
}
```
Additional params -
```
fromType: string - specify the saved object type. Default is `index-pattern` for data view
forId: string | string[] - limit the affected saved objects to one or more by id
forType: string - limit the affected saved objects by type
```
Improves upon https://github.com/elastic/kibana/pull/157665
Docs will be created in follow up PR
Currently, if you try loading archive with index mappings not having
replica set into stateless ES, it won't work properly: you will get 503
error on calling `GET <index_name>/_stats`:
```
{
"error": {
"root_cause": [
{
"type": "no_shard_available_action_exception",
"reason": null
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "indices-stats",
"node": null,
"reason": {
"type": "no_shard_available_action_exception",
"reason": null
}
}
]
},
"status": 503
}
```
In stateless replica is
[required](https://elastic.slack.com/archives/C037J0RKRAN/p1690218904855299)
in order to perform search requests (the "search shard").
This PR updates index mappings in es_archives with
`"auto_expand_replicas": "0-1"`, in order to be compatible with
stateless ES and so that we can re-use existing data sets rather than
creating new ones.
I checked with Core Team that we should fine to just adapt all mapping
files, but let me know if that doesn't work for you.
The same value is used to create the "real" SO
[indices](c79c09c3d0/packages/core/saved-objects/core-saved-objects-migration-server-internal/src/actions/constants.ts (L21))
in Kibana.
## Summary
Part of https://github.com/elastic/kibana/issues/161882
Updates all archives of the `encrypted_saved_objects_api_integration`
test suite:
* Removing SO index definitions.
* Updating documents defined in `data.json`.
## Summary
Uses the versioned router for the remaining routes in the data plugin:
KQL telemetry (opt-in stats), and scripting languages list.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Part of https://github.com/elastic/kibana/issues/161882
Updates the
`test/api_integration/fixtures/es_archiver/saved_objects/delete_unknown_types`
archive:
* deleting `mappings.json`
* updating documents in `data.json`
This way, _esArchiver_ will not delete SO indices and recreate them,
eliminating the odds of the related tests being flaky.
## Summary
Managing large number of saved objects can be cumbersome. This api
endpoint allows the management of references without clicking through a
lot of different UIs.
For example -
This swaps all data view id `abcd-efg` references to `xyz-123`
```
POST /api/data_views/swap_references
{
"from_id" : "abcd-efg",
"to_id" : "xyz-123",
"preview" : false, // optional, necessary to save changes
"delete" : true // optional, removes data view which is no longer referenced
}
returns
{
preview: false,
result: [{ id: "123", type: "visualization" }],
deleteSuccess: true
}
```
Additional params -
```
from_type: string - specify the saved object type. Default is `index-pattern` for data view
for_id: string | string[] - limit the affected saved objects to one or more by id
for_type: string - limit the affected saved objects by type
```
Closes https://github.com/elastic/kibana/issues/153806
## Summary
Under some circumstances passing `override` to `POST
/api/data_views/data_view` would fail. Its now fixed.
To test - Try using the override param from the Kibana dev console. I
found it reproduced the problem before the fix and shows its resolved
after the fix. The problem did not appear in the integration tests.
I suspect the problem had to do with how quickly the delete was
performed - if it completed before the create command then everything
was fine. If it didn't then the error would appear. Passing the
overwrite param to the saved object client eliminates the possibility of
the delete failing to complete.
Closes https://github.com/elastic/kibana/issues/161016
## Summary
- Move field preview to internal route
- Add versioning to route
- Endpoint is called with version
- Response schema validation
Closes https://github.com/elastic/kibana/issues/159158
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Attempt to fix https://github.com/elastic/kibana/issues/156581
The best explanation I have is that the two retrieved visualisations are
not necessarily retrieved in the expected order.
Sorting the results should ensure the expected order.
Will use the flaky test runner to check the fix.
## Summary
This updates connector tiles for Enterprise Search to bring them in line
with 8.9.
<img width="1032" alt="Screenshot 2023-06-26 at 16 26 40"
src="a222a2a3-17ad-4e72-8ce8-57e09d89d05e">
<img width="1035" alt="Screenshot 2023-06-26 at 16 25 04"
src="3ce1e78c-b4fe-404b-b314-e02f3d6f439e">
<img width="1086" alt="Screenshot 2023-06-26 at 16 24 40"
src="14dd6ca5-875f-4c2d-9408-a2fa15abbbec">
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Related to #159582
To better understand the PR, we need to have a look at `esArchiver.load`
function and how it handles `mappings.json`:
- It processes the file and if it finds saved object index (e.g.
`.kibana`), it will delete all SO indexes (.kibana,
.kibana_task_manager, .kibana_alerting_cases, etc.) created on Kibana
start. Then esArchiver will re-create all indexes according to defined
mappings.
In order to keep our tests relevant to the latest code changes, it is
good to avoid re-creating SO indexes and use "official" ones whenever it
is possible. This PR removes/cleans `mappings.json` files where it seems
reasonable. I also deleted few not used archives:
- es_archiver/saved_objects/ui_counters
- es_archiver/saved_objects/usage_counters
- security_solution_cypress/es_archives/empty_kibana
## Summary
Part of [54306](https://github.com/elastic/kibana/issues/154306)
Required #158468 to be merged first.
This PR integrates only the migration part of the new Metric new
formatter task.
This migration implementation defines takes the simplest route for the
migration route of metric visualization formatter:
* if any custom formatter is set in Lens, then apply a `compact` flag on
it to keep it consistent
* if `Default` formatter is set in Lens, then assume nothing and let the
regular formatter hierarchy do its own work
### Example
Original dashboard
After migration
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
