Fixes https://github.com/elastic/ingest-dev/issues/3217
## Summary
Add rate limiting to "install by upload" endpoint.
Implemented with a cache that is set with the timestamp of each install
by upload, independently from the package name/version. If the time
elapsed since the last timestamp it's less than retry time (10s), the
endpoint will return `429 Too many requests`.
### Testing
- Upload a package with
```
curl -XPOST -H 'content-type: application/zip' -H 'kbn-xsrf: true' http://localhost:5601/YOUR_PATH/api/fleet/epm/packages -u elastic:changeme --data-binary @PACKAGE_NAME.zip
```
- Upload another package shortly after. It can be the same one or
another one, as the rate limiting is applied across all uploads, no
matter the package name.
- If the second upload happens <10s after the first one, should return
error `429 Too Many Requests. Please wait 10s before uploading again.`
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Related to https://github.com/elastic/kibana/issues/7104
Update supertest, superagent, and the corresponding type package, to
their latest version.
(of course, types had some signature changes and we're massively using
supertest in all our FTR suites so the whole Kibana multiverse has to
review it)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Adding new defaults for the stream batch size and concurrency.
Now, the default batch size is increased from 300 to 5000.
The default concurrency is set to 4.
## Summary of Performance Characteristics
Using a list of over 70 archives from within the repo, we've benchmarked
the speed before changing the performance characteristics, and after.
One champion example is the
`x-pack/test/functional/es_archives/getting_started/shakespeare`
archive.
It's load time decreased from **~2.5 minutes**, to less than **18
seconds**, in a serverless environment.
There are a little over a handful where the time actually got worse, but
most are only milliseconds slower.
None are a full second slower.
---------
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
closes https://github.com/elastic/kibana/issues/164667
## 📝 Summary
This PR modifies the legend for `Document Count` graph in stack
monitoring to allow showing the exact document count when the user
hovers on a specific point.
## ✅ Testing
1. Make sure to have any dummy index with millions of documents
2. Navigate to Indices tab in Stack Monitoring
3. Click on the created index and observe the Document Count Graph
4. Hovering on any point should show the exact document count through
the legend below the graph
## 🎥 Demo
56747cf7-7914-4742-884a-9f9a9f59e9e6
### Summary
**Sets up the foundations for
https://github.com/elastic/kibana/issues/146000**
- created a new test server under
`x-pack/test/monitoring_api_integration/` that allows loading of
packages at kibana startup
- a test runner utility which is a simple for loop executing the
supplied test twice, one time with `metricbeat` data and a second time
with `package` data
- a utility that allows transformation of package data into metricbeat
data
**Adds API tests for the beats package**
- created a test case for each API exposed
- removed the duplicates from
`x-pack/test/api_integration/apis/monitoring`
-----
_See the included
[README](b55de5c1cc/x-pack/test/monitoring_api_integration/README.md)
for additional details_
This directory defines a custom test server that provides bundled
integrations
packages to the spawned test Kibana. This allows us to install those
packages at
startup, with all their assets (index templates, ingest pipelines..),
without
having to reach a remote package registry.
With the packages and their templates already installed we don't have to
provide
the static mappings in the tests archives. This has the benefit of
reducing our
disk footprint and setup time but more importantly it enables an easy
upgrade path
of the mappings so we can verify no breaking changes were introduced by
bundling
the new versions of the packages.
_Note that while Stack Monitoring currently supports 3 collection modes,
the tests
in this directory only focus on metricbeat and elastic-agent data. Tests
for legacy
data are defined under `x-pack/test/api_integration/apis/monitoring`._
Since an elastic-agent integration spawns the corresponding metricbeat
module under
the hood (ie when an agent policy defines elasticsearch metrics data
streams,
a metricbeat process with the elasticsearch module will be spawned), the
output
documents are _almost_ identical. This means that we can easily
transform documents
from a source (elastic-agent) to another (metricbeat), and have the same
tests run
against both datasets.
Note that we don't have to install anything for the metricbeat data
since the mappings
are already installed by elasticseach at startup, and available at
`.monitoring-<component>-8-mb`
patterns. So we are always running the metricbeat tests against the
latest version of
the mappings.
We could have a similar approach for packages, for example by installing
the latest
packages versions from public EPR before the test suites run, instead of
using pinned
versions. Besides the questionable reliance on remote services for
running tests,
this is also dangerous given that packages are released in a continuous
model.
This means that whenever the test suite would execute against the latest
version
of packages it would be too late, as in already available to users.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
