This changes the generated types tables in the docs to say `date`
instead of `datetime`. That's the name of the field in Elasticsearch so
it's a lot less confusing to call it that.
Closes#111650
This profiles additional timing information for each individual driver.
To the results from `profile` it adds the start and stop time for each
driver. That was already in the task status. To the profile and task
status it also adds the number of times the driver slept and some more
detailed history about a few of those times.
Explanation time! The compute engine splits work into some number of
`Drivers` per node. Each `Driver` is a single threaded entity - it runs
on a thread for a while then does one of three things: 1. Finishes 2.
Goes async because one of it's `Operator`s has gone async 3. Yields the
thread pool because it has run for too long
This PR measures the second two. At this point only three operators can
go async: * ENRICH * Reading from an empty exchange * Writing to a full
exchange
We're quite interested the these sleeps at the moment because they think
they may be slowing things down. Here's what it looks like when a driver
goes async because it wants to read from an empty exchange:
```
... the rest of the profile ...
"sleeps" : {
"counts" : {
"exchange empty" : 2
},
"first" : [
{
"reason" : "exchange empty",
"sleep" : "2024-08-13T19:45:57.943Z",
"sleep_millis" : 1723578357943,
"wake" : "2024-08-13T19:45:58.159Z",
"wake_millis" : 1723578358159
},
{
"reason" : "exchange empty",
"sleep" : "2024-08-13T19:45:58.164Z",
"sleep_millis" : 1723578358164,
"wake" : "2024-08-13T19:45:58.165Z",
"wake_millis" : 1723578358165
}
],
"last": [same as above]
```
Every time the driver goes async we count it in the `counts` map -
grouped by the reason the driver slept. We also record the sleep and
wake times for the first and last ten times the driver sleeps. In this
case it only slept twice, so the `first` and `last` ten times is the
same array.
This should give us a good sense about why drivers sleep while using a
limited amount of memory per driver.
This modifies the ESQL test infrastructure to generate more of the
documentation for functions. It generates the *Description* section, the
*Examples* section, and the *Parameters* section as separate files so we
can use them as needed. It also generates a `layout` file that's just
a guess as to how to render the whole thing. In some cases it'll work
and we can use that instead of hand maintaining a "top level"
description file for the function.
Most newly generated files are unused. We have to chose to pick them up
by replacing the sections we were manually maintaining with an include
of the generated section. Or by replacing the entire hand maintained
file with the generated top level file.
Relates to #104247
