## Summary
- Closeselastic/kibana-operations#245
- Convert FIPS base image from UBI to `chainguard-base-fips`
- Add FIPS base image updates to Renovate
- Adjust naming scheme for FIPS image from `kibana-ubi-fips` to
`kibana-fips`
- Adds new image flavor `kibana-cloud-fips`
- Adds support for `ci:build-cloud-fips-image` label
- Move Cloud image building to its own step instead of being part of
`Build Kibana Distribution` step so it will be triggered when the build
is reused and the `build` step is skipped.
Currently Kibana forwards `query_range_secs` and `query_offset_secs` to
mark the selected time range when reporting TTFMP event. This format
caused some challenges to identify `from`, `to` date offsets in
visualizations.
To simplify, the PR renames and sends the three fields explicitly:
- `query_from_offset_secs` offset to `0` (now), with -ve for past and
+ve for future dates
- `query_to_offset_secs` offset to `0` (now), with -ve for past and +ve
for future dates
- `query_range_secs` same as previously sent
_This approach is followed after a discussion, and based on the
[gist](https://gist.github.com/andrewvc/1f04a57a336d768e4ec5ff2eff06ba54)
excerpt:_
```
Earliest date -> QueryFrom
Newest date -> QueryTo
Duration -> QueryRange
```
### Indexing
These fields then should be mapped in the EBT indexer to ingest in the
top level of the document, eventually removing the need to create
runtime fields in data views for visualizations.
Also, runtime fields in data views should be updated to reflect this
change. For backward compatibility, the runtime fields can cater both
the old and new field names conditionally.
### Testing
- Ensure that the TTFMP events are correctly reporting the date ranges.
### Example
## Summary
The `/packages` folder at the root of the Kibana repository used to
contain a lot of packages.
In the context of SKA, they have been gradually moved to various
locations:
* `src/platform/packages`
* `x-pack/platform/packages`
* `src/core/packages`
Currently, only `devOnly: true` packages are left in this folder. This
comprises libraries for CLI scripts as well as testing utilities.
With this PR, we are moving ~half of these packages under
`src/platform/packages/(private|shared)/`.
In particular, we are moving those packages that are being used from
platform and/or solutions.
Since they are `"devOnly": true`, this means they are ONLY used from
tests, cypress tests, storybook configs, ./scripts/ folders inside some
modules, or other non-prod-time logic. Nonetheless, they are effectively
referenced from platform and/or solutions code, hence I decided they
should be placed under `platform` folders.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adding documentation to help teams diagnose FIPS pipeline errors in
test.
Linked to FIPS compliant OpenSSL/Node setup docs
---------
Co-authored-by: Jeramy Soucy <jeramy.soucy@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Relates to https://github.com/elastic/kibana/issues/181995
This PR updates the examples to include availability information and
another description.
Co-authored-by: Jean-Louis Leysens <jeanlouis.leysens@elastic.co>
Saved objects declared as `hidden` can only be accessed with a client
that explicitly includes hidden types.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
---------
Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com>
* Remove duplicated “File service” entry from nav
* Move Screenshotting to main Tutorials section in nav
* Add “Updating Puppeteer and Chromium” to nav as a sub-item of
screenshotting
* Move files for Screenshotting/Chromium out of the SharedUX space to
`dev_docs/tutorials/screenshotting`
## Summary
It’s common request for Dev teams to run specific journeys on a PR to
compare performance metrics against the `main` branch. These requests
usually focus on a particular area, such as the Dashboard or Discover
app.
To streamline the process, this PR groups relevant journeys into
categories that can be triggered through an environment variable. For
example, setting `JOURNEYS_GROUP=dashboard` will execute only the three
dashboard-specific journeys, which are (usually) sufficient for
evaluating the performance impact of code changes within the Dashboard
app.
Current Process for Triggering Performance Builds:
- Create a new kibana-single-user-performance
[build](https://buildkite.com/elastic/kibana-single-user-performance#new)
- Provide the following arguments:
Branch: `refs/pull/<PR_number>/head`
Under Options, set the environment variable:
`JOURNEYS_GROUP=<group_name>`
Currently supported journey groups:
- kibanaStartAndLoad
- crud
- dashboard
- discover
- maps
- ml
[Build example
](https://buildkite.com/elastic/kibana-single-user-performance/builds/14427)
Each group focuses on a specific set of journeys tied to its respective
area in Kibana, allowing for more targeted performance testing. Since
running group takes ~5-10 min on bare metal worker, it should not delay
the regular (every 3h) runs against `main` branch
test locally with `node scripts/run_performance.js --group <group_name>`
## Summary
Added documentation explaining how SO migrations on serverless work
## Preview
<img width="741" alt="Screenshot 2024-03-22 at 16 15 13"
src="2217c01f-8447-4f22-a782-a07ff221aa42">
## Summary
Moving synthtrace clients init inside kbn-journeys:
esArchiver does not always solve the issue with data generation. We
already have afew journeys using Synthtrace instead and expect more to
come.
In order to simplify the process of creating new journeys, this PR moves
Synthtrace client initialisation into kbn-journey package and exposes a
way to define client type, generator function & its input arguments:
```
import { Journey, SynthtraceOptions } from '@kbn/journeys';
import { subj } from '@kbn/test-subj-selector';
import { generateApmData } from '../synthtrace_data/apm_data';
export const journey = new Journey({
synthtrace: {
type: 'apm',
generator: generateApmData,
options: {
from: new Date(Date.now() - 1000 * 60 * 15),
to: new Date(Date.now() + 1000 * 60 * 15),
},
},
})
```
PR also needs review from teams who use Synthtrace to understand if the implementation is matching expectations.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
The source code classifier we currently have was incorrectly classifying
e2e journey files as `non-package` instead of `tests or mocks` as it was
not using the name standards we used for FTR files.
We could have created a `functional-tests` package for the performance
folder (which is what we want to do in the future) but because we don't
have the feature to create ownerless packages it would not be easy to
find a given owner for that folder.
As such I'm just opting for a second solution which is applying the same
name standards to this journeys folder as we have for FTR and changing a
little the classifier to recognise it.
This should fix the problem found at
https://github.com/elastic/kibana/pull/178017.
Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
## Summary
Closes elastic/kibana-operations/issues/24
This adds a second flavor of UBI image (`kibana-ubi-fips`) which has a
FIPS compliant version of OpenSSL compiled and linked to Node. Using the
label `ci:build-docker-fips` will create the image in CI and push to the
registry.
The FIPS image start the Kibana NodeJS process using the FIPS compliant
OpenSSL version. Kibana will start in this state but crash during
runtime because there are many code changes required for it to be FIPS
compliant, including `node_module` usage. I attempted numerous ways to
load other OpenSSL providers alongside the FIPS provider, but it always
led to Kibana crashing on invalid algorithm usage.
---------
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
I had to change `waitForRender` since `page.waitForFunction` tries to
run a script on page and it is not working due to CSP settings on Cloud.
Instead of injecting a script, we use a classical API to find
elements/attributes in the DOM.
Since `PUT /internal/core/_settings` is merged in 8.11.0, journeys run
on Cloud with on-fly labels update is supported starting deployments
8.11.0+. I added error message for 404 code just in case someone runs it
on earlier version.
`many_fields_discover` journey was update since on Cloud the data view
used by scenario is not selected by default.
How it works:
Create a deployment with QAF and re-configure it for journey run:
```
export EC_DEPLOYMENT_NAME=my-run-8.11
qaf elastic-cloud deployments create --stack-version 8.11.0-SNAPSHOT --environment staging --region gcp-us-central1
qaf elastic-cloud deployments configure-for-performance-journeys
```
Run any journey, e.g. many_fields_discover
```
TEST_CLOUD=1 TEST_ES_URL=https://username:pswd@es_url:443 TEST_KIBANA_URL=https://username:pswd@kibana-ur_url node scripts/functional_test_runner --config x-pack/performance/journeys/many_fields_discover.ts
```
You should see a log about labels being updated:
```
Updating telemetry & APM labels: {"testJobId":"local-a3272047-6724-44d1-9a61-5c79781b06a1","testBuildId":"local-d8edbace-f441-4ba9-ac83-5909be3acf2a","journeyName":"many_fields_discover","ftrConfig":"x-pack/performance/journeys/many_fields_discover.ts"}
```
And then able to find APM logs for the journey in
[Ops](https://kibana-ops-e2e-perf.kb.us-central1.gcp.cloud.es.io:9243/app/apm/services?comparisonEnabled=true&environment=ENVIRONMENT_ALL&kuery=labels.testJobId%20%3A%20%22local-d79a878c-cc7a-423b-b884-c9b6b1a8d781%22&latencyAggregationType=avg&offset=1d&rangeFrom=now-24h%2Fh&rangeTo=now&serviceGroup=&transactionType=request)
cluster
## Summary
Expands on the HTTP versioning tutorial with more information about
`internal` vs `public` endpoints.
## Screenshots
<img width="875" alt="Screenshot 2023-06-01 at 15 52 18"
src="f8f790a0-31ec-4123-89db-b1f01aa17845">
<img width="923" alt="Screenshot 2023-06-01 at 15 52 33"
src="f97ad89b-face-4176-a814-8e17ecd3d2a5">
* Add deferred migrations parameter.
* Update outdated documents query to take into account deferred migrations.
* Update outdated documents query to take into account the core migration version.
* Update read operations in the saved objects repository to perform deferred migrations.
## Summary
This PR adds changes and instructions how to run existing performance
journeys against ESS instances.
As previously discussed with @pheyos , there are few manual steps to be
done before running the actual journey:
1. Create cloud deployment and re-configure it the way APM traces are
reported the monitoring cluster
2. Check out the branch matching deployment version (main ->
8.9.0-SNAPSHOT), create user with `superuser` role (we don't test
functional features, only performance metrics)
Then you can run the journey like a regular cloud test suite:
```
export TEST_KIBANA_URL=https://<username>:<password>@<kibana_url>
export TEST_ES_URL=https://<username>:<password>@j<elasticsearch_url>:<port>
export TEST_CLOUD=1
node scripts/functional_test_runner.js --config x-pack/performance/journeys/$YOUR_JOURNEY_NAME.ts
```
<img width="1357" alt="image"
src="https://user-images.githubusercontent.com/10977896/236499836-2738fecb-a5a9-4e58-a10b-915b962af037.png">
---------
Co-authored-by: Jon <jon@budzenski.me>
## Summary
This contribution adds a new tutorial for how plugin developers must
approach developing and maintaining versionable HTTP APIs.
## How to review
1. Pull down the changes
2. Run `./scripts/dev_docs.sh`
3. Open http://localhost:3000 in your browser
4. Go to "Versioning HTTP APIs" in the side nav
## Summary
Adds a dev docs guide for the core-approved versioning strategy. This
strategy is subject to some iteration but is based on the work we did
for the Saved Objects Management plugin in
https://github.com/elastic/kibana/pull/149495.
Closes https://github.com/elastic/kibana/issues/149929
## How to test
1. Run `./scripts/dev_docs.sh`
2. Navigate to "Versioning interfaces" in the side nav menu
---------
Co-authored-by: Luke Elmers <lukeelmers@gmail.com>
This PR updates docs around Kibana performance effort:
- how to create single user performance journeys, custom metrics with
EBT and review test results
- how to create api capacity test and where to find its test results
