This PR retrospectively adds breaking change items to the release notes:
* 8.10: path change for Download CSV from saved search panel.
* 8.14: requirement to add a yml setting to enable the deprecated
Download CSV functionality, rather than generating a CSV report
---------
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
## Summary
This PR echos verbiage that belongs as a footnote on the Kibana Support
Matrix web page: https://www.elastic.co/support/matrix/#matrix_os
## Checklist
- [x] Include text from the footnote of the Kibana Support Matrix
- [x] Guide users to test reporting features in a pre-production
environment
## Summary
This PR updates the structure of the Dashboards docs and refreshes some
outdated parts of the content.
More updates will be made in future PRs to refresh screenshots and to
refresh the content more in depth.
This new structure and edits:
- distribute the pages in more user-oriented identified themes, for
better findability, scanning, and to ease possible integration of some
of these pages into in-app documentation.
- are more future proof to evolve along with upcoming features.
~I'll leave this PR as a draft until I resolve some link dependencies
coming from other docs sources and check some additional bits of
content.~
Preview available on demand on Slack.
Closes: https://github.com/elastic/platform-docs-team/issues/408 (I'll
create separate issues for remaining items)
Closes: https://github.com/elastic/platform-docs-team/issues/413
Closes: https://github.com/elastic/platform-docs-team/issues/418
## Summary
This adds a publicBaseUrl to the Elasticsearch plugin config so users
can set a publicly accessible URL for Elasticsearch.
---------
Co-authored-by: Rudolf Meijering <skaapgif@gmail.com>
Epic link: https://github.com/elastic/kibana-team/issues/720
## Summary
This PR solves a problem in Reporting code, in that most functional
tests and API integration tests that cover PDF functionality were
referencing the deprecated `printable_pdf` export type. As of this PR,
the tests now use the `printable_pdf_v2` export type.
List of clean up changes:
* Fully remove PNGV1 (leftovers from #162517)
* Update tests and documentation to use PDFV2
* Allow users to see removed types in the listing and download reports
for removed types. This makes it possible to see `csv` and `png` job
types in the current version, even though it's not possible to create
new jobs of those types.
Context: the reporting plugin has two deprecated export types which are
still active in the code:
* `printable_pdf` - this is actively replaced with `printable_pdf_v2'
* `csv_searchsource` - this is replaced with `csv_v2' for certain
applications such as ES|QL export. For general CSV export, `csv_v2' is
currently only offered as an API`.
### Checklist
Delete any items that are not applicable to this PR.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Moving common services to respective new homes.
Resolves: https://github.com/elastic/kibana/issues/188541
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Scaffolding empty plugin for development of day 0 onboarding work. This
plugin is currently disabled by default and will be enabled for ES3 when
it's ready for users.
## Summary
Part of the OTel effort. This PR adds otel datastream patterns into the
default indices that are used by the APM UI.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Caue Marcondes <caue.marcondes@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR offers small updates to documentation on:
* Calling Kibana APIs to automate report generation
* Using Elasticsearch APIs directly to export data
## Summary
Allow deletion of monitor via id param !!
User can now delete monitor via passing id as url param
`DELETE <kibana host>:<port>/api/synthetics/monitors/<config_id>`
Previous bulk delete via list of ids via API body still works as well !!
Docs are updated !!
- Closes https://github.com/elastic/kibana/issues/181568
## Summary
This PR adds a default height limit for values in DocViewer. If the
value is too long, we visually truncate it and add "View more" button
which allows to expand to view the entire value. This way all fields in
the flyout are easily accessible (less scrolling required) even if they
contain long values. ~The height can be configured via the existing
`truncate:maxHeight` Advanced Setting.~ If user expands a value, closes
the flyout and opens it again, the value will be shown as expanded again
for that field.
Related: https://github.com/elastic/kibana/pull/164236
## Testing
Some cases to check while testing:
- varios value formats
- legacy table vs data grid
- doc viewer flyout vs Single Document page
Sample long value:
<details>
<pre>
POST test_this/_doc/
{
"message": """javax.servlet.ServletException: Something bad happened
at
com.example.myproject.OpenSessionInViewFilter.doFilter(OpenSessionInViewFilter.java:60)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.example.myproject.ExceptionHandlerFilter.doFilter(ExceptionHandlerFilter.java:28)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.example.myproject.OutputBufferFilter.doFilter(OutputBufferFilter.java:33)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: com.example.myproject.MyProjectServletException
at com.example.myproject.MyServlet.doPost(MyServlet.java:169)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at
com.example.myproject.OpenSessionInViewFilter.doFilter(OpenSessionInViewFilter.java:30)
... 27 more
Caused by: org.hibernate.exception.ConstraintViolationException: could
not insert: [com.example.myproject.MyEntity]
at
org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:96)
at
org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at
org.hibernate.id.insert.AbstractSelectingDelegate.performInsert(AbstractSelectingDelegate.java:64)
at
org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2329)
at
org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2822)
at
org.hibernate.action.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:71)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:268)
at
org.hibernate.event.def.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:321)
at
org.hibernate.event.def.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:204)
at
org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:130)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:210)
at
org.hibernate.event.def.DefaultSaveEventListener.saveWithGeneratedOrRequestedId(DefaultSaveEventListener.java:56)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:195)
at
org.hibernate.event.def.DefaultSaveEventListener.performSaveOrUpdate(DefaultSaveEventListener.java:50)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:93)
at org.hibernate.impl.SessionImpl.fireSave(SessionImpl.java:705)
at org.hibernate.impl.SessionImpl.save(SessionImpl.java:693)
at org.hibernate.impl.SessionImpl.save(SessionImpl.java:689)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.hibernate.context.ThreadLocalSessionContext$TransactionProtectionWrapper.invoke(ThreadLocalSessionContext.java:344)
at $Proxy19.save(Unknown Source)
at com.example.myproject.MyEntityService.save(MyEntityService.java:59)
<-- relevant call (see notes below)
at com.example.myproject.MyServlet.doPost(MyServlet.java:164)
... 32 more
Caused by: java.sql.SQLException: Violation of unique constraint
MY_ENTITY_UK_1: duplicate value(s) for column(s) MY_COLUMN in statement
[...]
at org.hsqldb.jdbc.Util.throwError(Unknown Source)
at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown Source)
at
com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)
at
org.hibernate.id.insert.AbstractSelectingDelegate.performInsert(AbstractSelectingDelegate.java:57)
... 54 more"""
}
</pre>
</details>
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
---------
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
## Summary
Documentation updates to distinguish the main parts of Reporting:
* service framework
* CSV export
* Screenshot export (PNG/PDF)
Additionally, this PR attempts to consistently apply the admonitions
regarding limitations of the different export types.
Part of https://github.com/elastic/kibana-team/issues/720
---------
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
This PR introduces an Inference plugin.
## Goals
- Provide a single place for all interactions with large language models
and other generative AI adjacent tasks.
- Abstract away differences between different LLM providers like OpenAI,
Bedrock and Gemini
- Host commonly used LLM-based tasks like generating ES|QL from natural
language and knowledge base recall.
- Allow us to move gradually to the _inference endpoint without
disrupting engineers.
## Architecture and examples
## Terminology
The following concepts are referenced throughout this POC:
- **chat completion**: the process in which the LLM generates the next
message in the conversation. This is sometimes referred to as inference,
text completion, text generation or content generation.
- **tasks**: higher level tasks that, based on its input, use the LLM in
conjunction with other services like Elasticsearch to achieve a result.
The example in this POC is natural language to ES|QL.
- **tools**: a set of tools that the LLM can choose to use when
generating the next message. In essence, it allows the consumer of the
API to define a schema for structured output instead of plain text, and
having the LLM select the most appropriate one.
- **tool call**: when the LLM has chosen a tool (schema) to use for its
output, and returns a document that matches the schema, this is referred
to as a tool call.
## Usage examples
```ts
class MyPlugin {
setup(coreSetup, pluginsSetup) {
const router = coreSetup.http.createRouter();
router.post(
{
path: '/internal/my_plugin/do_something',
validate: {
body: schema.object({
connectorId: schema.string(),
}),
},
},
async (context, request, response) => {
const [coreStart, pluginsStart] = await coreSetup.getStartServices();
const inferenceClient = pluginsSetup.inference.getClient({ request });
const chatComplete$ = inferenceClient.chatComplete({
connectorId: request.body.connectorId,
system: `Here is my system message`,
messages: [
{
role: MessageRole.User,
content: 'Do something',
},
],
});
const message = await lastValueFrom(
chatComplete$.pipe(withoutTokenCountEvents(), withoutChunkEvents())
);
return response.ok({
body: {
message,
},
});
}
);
}
}
```
## Implementation
The bulk of the work here is implementing a `chatComplete` API. Here's
what it does:
- Formats the request for the specific LLM that is being called (all
have different API specifications).
- Executes the specified connector with the formatted request.
- Creates and returns an Observable, and starts reading from the stream.
- Every event in the stream is normalized to a format that is close to
(but not exactly the same) as OpenAI's format, and emitted as a value
from the Observable.
- When the stream ends, the individual events (chunks) are concatenated
into a single message.
- If the LLM has called any tools, the tool call is validated according
to its schema.
- After emitting the message, the Observable completes
There's also a thin wrapper around this API, which is called the
`output` API. It simplifies a few things:
- It doesn't require a conversation (list of messages), a simple `input`
string suffices.
- You can define a schema for the output of the LLM.
- It drops the token count events that are emitted
- It simplifies the event format (update & complete)
### Observable event streams
These APIs, both on the client and the server, return Observables that
emit events. When converting the Observable into a stream, the following
things happen:
- Errors are caught and serialized as events sent over the stream (after
an error, the stream ends).
- The response stream outputs data as [server-sent
events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events)
- The client that reads the stream, parses the event source as an
Observable, and if it encounters a serialized error, it deserializes it
and throws an error in the Observable.
### Errors
All known errors are instances, and not extensions, from the
`InferenceTaskError` base class, which has a `code`, a `message`, and
`meta` information about the error. This allows us to serialize and
deserialize errors over the wire without a complicated factory pattern.
### Tools
Tools are defined as a record, with a `description` and optionally a
`schema`. The reason why it's a record is because of type-safety. This
allows us to have fully typed tool calls (e.g. when the name of the tool
being called is `x`, its arguments are typed as the schema of `x`).
## Notes for reviewers
- I've only added one reference implementation for a connector adapter,
which is OpenAI. Adding more would create noise in the PR, but I can add
them as well. Bedrock would need simulated function calling, which I
would also expect to be handled by this plugin.
- Similarly, the natural language to ES|QL task just creates dummy
steps, as moving the entire implementation would mean 1000s of
additional LOC due to it needing the documentation, for instance.
- Observables over promises/iterators: Observables are a well-defined
and widely-adopted solution for async programming. Promises are not
suitable for streamed/chunked responses because there are no
intermediate values. Async iterators are not widely adopted for Kibana
engineers.
- JSON Schema over Zod: I've tried using Zod, because I like its
ergonomics over plain JSON Schema, but we need to convert it to JSON
Schema at some point, which is a lossy conversion, creating a risk of
using features that we cannot convert to JSON Schema. Additionally,
tools for converting Zod to and [from JSON Schema are not always
suitable
](https://github.com/StefanTerdell/json-schema-to-zod#use-at-runtime).
I've implemented my own JSON Schema to type definition, as
[json-schema-to-ts](https://github.com/ThomasAribart/json-schema-to-ts)
is very slow.
- There's no option for raw input or output. There could be, but it
would defeat the purpose of the normalization that the `chatComplete`
API handles. At that point it might be better to use the connector
directly.
- That also means that for LangChain, something would be needed to
convert the Observable into an async iterator that returns
OpenAI-compatible output. This is doable, although it would be nice if
we could just use the output from the OpenAI API in that case.
- I have not made room for any vendor-specific parameters in the
`chatComplete` API. We might need it, but hopefully not.
- I think type safety is critical here, so there is some TypeScript
voodoo in some places to make that happen.
- `system` is not a message in the conversation, but a separate
property. Given the semantics of a system message (there can only be
one, and only at the beginning of the conversation), I think it's easier
to make it a top-level property than a message type.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR updates the Log pattern analysis in Discover documentation to
reflect the latest changes in the feature. It also removes the technical
preview admonition at the top of the page.
