## Summary
resolves https://github.com/elastic/kibana/issues/159454
Remove experimental message from saved objects import and export apis.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
We've had some questions around legacy migrations that are flagged as
deprecated in the type. This updates the typescript doc comment to
hopefully clarify that.
## Summary
This PR is part of the Kibana Sustainable Architecture effort.
The goal is to start categorising Kibana packages into _generic
platform_ (`group: "platform"`) vs _solution-specific_.
```
group?: 'search' | 'security' | 'observability' | 'platform'
visibility?: 'private' | 'shared'
```
Uncategorised modules are considered to be `group: 'common', visibility:
'shared'` by default.
We want to prevent code from solution A to depend on code from solution
B.
Thus, the rules are pretty simple:
* Modules can only depend on:
* Modules in the same group
* OR modules with 'shared' visibility
* Modules in `'observability', 'security', 'search'` groups are
mandatorily `visibility: "private"`.
Long term, the goal is to re-organise packages into dedicated folders,
e.g.:
```
x-pack/platform/plugins/private
x-pack/observability/packages
```
For this first wave, we have categorised packages that seem
"straightforward":
* Any packages that have:
* at least one dependant module
* all dependants belong to the same group
* Categorise all Core packages:
* `@kbn/core-...-internal` => _platform/private_
* everything else => _platform/shared_
* Categorise as _platform/shared_ those packages that:
* Have at least one dependant in the _platform_ group.
* Don't have any `devOnly: true` dependants.
### What we ask from you, as CODEOWNERS of the _package manifests_, is
that you confirm that the categorisation is correct:
* `group: "platform", visibility: "private"` if it's a package that
should only be used from platform code, not from any solution code. It
will be loaded systematically in all serverless flavors, but solution
plugins and packages won't be able to `import` from it.
* `group: "platform", visibility: "shared"` if it's a package that can
be consumed by both platform and solutions code. It will be loaded
systematically in all serverless flavors, and anybody can import / use
code from it.
* `group: "observability" | "security" | "search", visibility:
"private"` if it's a package that is intented to be used exclusively
from a given solution. It won't be accessible nor loaded from other
solutions nor platform code.
Please refer to
[#kibana-sustainable-architecture](https://elastic.slack.com/archives/C07TCKTA22E)
for any related questions.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/158027
## Summary
Simply dedupes references to objects if they are part of the
missing_references in the copy saved objects to SO endpoint
### Notes
- Update forEach over SOs to a regular for loop since we had a couple of
early exit scenarios
- Checks against the set for references already added to the missing
list and adds only if not present
------
**Old response: Note the duplicate references**
<img width="400" alt="Screenshot 2024-11-14 at 01 52 54"
src="https://github.com/user-attachments/assets/67078080-e39d-43b2-bf7c-7abb76866fa4">
**New response**
<img width="800" alt="Screenshot 2024-11-14 at 01 50 41"
src="https://github.com/user-attachments/assets/776db189-af8c-4522-bb03-f8efbb7cdcd9">
### Release note
Dedupe results from copy saved objects to spaces API when object
contains references to other objects.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
fix https://github.com/elastic/kibana/issues/197721.
The route deprecation field changed from a boolean to an object, where
the object contains information that is used in deprecation issues that
the Upgrade Assistant shows.
This PR makes the necessary changes in the deprecated Saved Objects HTTP
APIs.
This PR also includes a release notes entry for the API deprecations
that was missing.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### How to test this:
- Pull PR, run es against a trial license and start kibana as usual.
- Make a curl request to Kibana to get the config saved object:
```
curl --location 'localhost:5601/abc/api/saved_objects/config/9.0.0' \
--header 'Content-Type: application/json' \
--header 'Accept-Encoding: gzip, deflate, br' \
--header 'kbn-xsrf: kibana' \
--header 'Kbn-Version: 9.0.0' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='
```
- Navigate to Upgrade Assistant and observe Kibana has at least 1
deprecation warning.
- View Kibana's warnings, you should see a warning entry for `The "GET
/api/saved_objects/{type}/{id}" route is deprecated`
- click on the deprecation and you should see more detail about the
deprecated API that's been used in the flyout:
- resolve the deprecation warning
- Kibana should continue to issue requests to the deprecated SO HTTP
APIs because these APIs have not been removed yet.
### Risk Matrix
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| End user concern from the deprecation warning that says the routes
have been deprecated | Low | Low | The APIs have been deprecated since
8.7 and recommends using public APIs instead. |
### For maintainers
- [x] This will appear in the **Release Notes** and follow the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes a regression by adding back the functionality that was originally
introduced in https://github.com/elastic/kibana/pull/162404: skipping
migrations (transform and forwardCompatibilitySchema) when the fields
option is specified
In working on this we uncovered some problem scenarios we previously
hadn't thought through (will create separate issues for these):
1. Even if we can't reliably run the transform part of migrations on a
partial document we should still run the `forwardCompatibilitySchema` so
that unknown fields from future versions are dropped.
However, we realised most teams missed the detail in our
[docs](https://github.com/elastic/kibana/blob/main/docs/developer/architecture/core/saved-objects-service.asciidoc?plain=1#L429-L457)
that call out that forwardCompatibilitySchemas shouldn't assert on data
but only drop unknown fields. This means if we would run the
forwardCompatibilitySchema today it would most often fail on partial
documents.
3. When `fields` contain an encrypted field but not all AAD fields,
encryption would fail and log an error (potentially triggering
serverless alerts).
Saved objects declared as `hidden` can only be accessed with a client
that explicitly includes hidden types.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
---------
Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com>
Closes https://github.com/elastic/kibana/issues/194692
## Summary
Allow consumers of ESOs to explicitly opt out of the strict highly
random UID requirements while registering the ESO type
### Description
The `getValidId` method was updated to allow consumers of Encrypted
Saved Objects to explicitly opt-out of the enforced random ID
requirement.
This change is added during ESO registration - consumers can now pass a
new field to opt-out of random UIDs.
Additional changes
- Updated canSpecifyID logic:
- The canSpecifyID condition now also checks if enforceRandomId is
explicitly set to false.
This opt-out approach allows specific ESOs to bypass the random ID
enforcement without affecting the default behavior, keeping it secure by
default.
During the registration phase of the saved object, consumers can now
specify if they'd like to opt-out of the random ID
```
savedObjects.registerType({
name: TYPE_WITH_PREDICTABLE_ID,
//...
});
encryptedSavedObjects.registerType({
type: TYPE_WITH_PREDICTABLE_ID,
//...
enforceRandomId: false,
});
```
### Release notes
Improves Encrypted Saved Objects (ESO) ID validation by adding an
enforceRandomId parameter, allowing consumers to opt out of the default
random ID requirement for specific use cases.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
- [ ] This will appear in the **Release Notes** and follow the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Jeramy Soucy <jeramy.soucy@elastic.co>
## Summary
Hello, this is a follow up PR to #192234 . The previous PR added
simplistic logging to the saved objects importer. The goal now is to
enhance the logs with information on the saved objects being imported,
how they are imported, and by displaying any errors.
#### `import_saved_objects.ts`:
- Logs specific types being imported
- Logs size limit and overwrite status
- Logs Success/Fail messages
#### Changes to `saved_objects_importer.ts`:
- Passes the logger to `importSavedObjectsFromStream`
- Removes "starting import"
#### Changes to `import_saved_objects.test.ts`:
- Updates it for the new logger parameter
#### Changes to `import.test.ts`:
- Uses the mock logger provided by core, instead of using a custom one
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
- [ ] This will appear in the **Release Notes** and follow the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
# Summary
Adds a new API deprecations feature inside core.
This feature enabled plugin developers to mark their versioned and
unversioned public routes as deprecated.
These deprecations will be surfaced to the users through UA to help them
understand the deprecation and address it before upgrading. This PR also
surfaces these deprecations to UA.
Closes https://github.com/elastic/kibana/issues/117241
1. Core service to flag deprecated routes
2. UA code to surface and resolve deprecated routes
## Flagging a deprecated Route
### The route deprecation option
We have three types of route deprecations:
- `type: bump`: A version bump deprecation means the API has a new
version and the current version will be removed in the future in favor
of the newer version.
- `type: remove`: This API will be completely removed. You will no
longer be able to use it in the future.
- `type: migrate`: This API will be migrated to a different API and will
be removed in the future in favor of the other API.
All route deprecations expect a documentation link to help users
navigate. We might add a generic documentation link and drop this
requirement in the future but for now this is required.
### Deprecated Route Example
Full examples can be found in the `routing_example` example plugin
located in this directory:
`examples/routing_example/server/routes/deprecated_routes`
```ts
router[versioned?].get(
{
path: '/',
options: {
deprecated: {
documentationUrl: 'https://google.com',
severity: 'warning',
reason: {
type: 'bump',
newApiVersion: '2024-10-13',
},
},
},
},
async (context, req, res) => {
...
```
## Surfaced API deprecations in UA
The list of deprecated APIs will be listed inside Kibana deprecations
along with the already supported config deprecations.
<img width="1728" alt="image"
src="https://github.com/user-attachments/assets/5bece704-b80b-4397-8ba2-6235f8995e4a">
Users can click on the list item to learn more about each deprecation
and mark it as resolved
<img width="1476" alt="image"
src="https://github.com/user-attachments/assets/91c9207b-b246-482d-a5e4-21d0c61582a8">
### Marking as resolved
Users can click on mark as resolved button in the UA to hide the
deprecation from the Kiban deprecations list.
We keep track on when this button was clicked and how many times the API
has been called. If the API is called again the deprecation will
re-appear inside the list. We might add a feature in the future to
permenantly supress the API deprecation from showing in the list through
a configuration (https://github.com/elastic/kibana/issues/196089)
If the API has been marked as resolved before we show this in the flyout
message:
> The API GET /api/deprecations/ has been called 25 times. The last time
the API was called was on Monday, October 14, 2024 1:08 PM +03:00.
> The api has been called 2 times since the last time it was marked as
resolved on Monday, October 14, 2024 1:08 PM +03:00
Once marked as resolved the flyout exists and we show this to the user
until they refresh the page
<img width="1453" alt="image"
src="https://github.com/user-attachments/assets/8bb5bc8b-d1a3-478f-9489-23cfa7db6350">
## Telemetry:
We keep track of 2 new things for telemetry purposes:
1. The number of times the deprecated API has been called
2. The number of times the deprecated API has been resolved (how many
times the mark as resolved button in UA was clicked)
## Code review
- [x] Core team is expected to review the whole PR
- [ ] Docs team to review the copy and update the UA displayed texts
(title, description, and manual steps)
- [x] kibana-management team is expected to review the UA code changes
and UI
- [ ] A few teams are only required to approve this PR and update their
`deprecated: true` route param to the new deprecationInfo object we now
expect. There is an issue tracker to address those in separate PRs later
on: https://github.com/elastic/kibana/issues/196095
## Testing
Run kibana locally with the test example plugin that has deprecated
routes
```
yarn start --plugin-path=examples/routing_example --plugin-path=examples/developer_examples
```
The following comprehensive deprecated routes examples are registered
inside the folder:
`examples/routing_example/server/routes/deprecated_routes`
Run them in the console to trigger the deprecation condition so they
show up in the UA:
```
# Versioned routes: Version 1 is deprecated
GET kbn:/api/routing_example/d/versioned?apiVersion=1
GET kbn:/api/routing_example/d/versioned?apiVersion=2
# Non-versioned routes
GET kbn:/api/routing_example/d/removed_route
POST kbn:/api/routing_example/d/migrated_route
{}
```
1. You can also mark as deprecated in the UA to remove the deprecation
from the list.
2. Check the telemetry response to see the reported data about the
deprecated route.
3. Calling version 2 of the API does not do anything since it is not
deprecated unlike version `1` (`GET
kbn:/api/routing_example/d/versioned?apiVersion=2`)
4. Internally you can see the deprecations counters from the dev console
by running the following:
```
GET .kibana_usage_counters/_search
{
"query": {
"bool": {
"should": [
{"match": { "usage-counter.counterType": "deprecated_api_call:total"}},
{"match": { "usage-counter.counterType": "deprecated_api_call:resolved"}},
{"match": { "usage-counter.counterType": "deprecated_api_call:marked_as_resolved"}}
]
}
}
}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana-team/issues/1113
Merges some tested code paths into `v2_migration.test.ts` which uses an
archive that is generated in 9.0.0.
Gets rid of the related failing tests.
## Summary
We will be enforcing restricted access to internal HTTP APIs [from
9.0](https://github.com/elastic/kibana/issues/186781). This PR is part 1
of audit checking that our public APIs have their access tag set
explicitly to ensure they are still available to end users after we
start enforcing HTTP API restrictions. APIs reviewed in this PR
([docs](https://www.elastic.co/guide/en/kibana/current/dashboard-import-api.html)):
<img width="260" alt="Screenshot 2024-09-11 at 11 25 55"
src="https://github.com/user-attachments/assets/499b1f1f-8e01-4463-9410-4500e438cd23">
## Note to reviewers
This audit is focussed on set `access: 'public'` where needed. Per the
screenshot our public-facing documentation is taken as the source of
truth for which APIs should be public. This may differ per offering so
please consider whether a given HTTP API should be public on both
serverless and stateful offerings.
## Risks
* If we miss an API that should be public, end users will encounter a
`400` response when they try to use the HTTP API on 9.0
* If we set an API's access to "public" it will not have the same
restrictions applied to it.
## Summary
This PR has breadth, but not depth. This adds 3 new `eslint` rules. The
first two protect against the use of code generated from strings (`eval`
and friends), which will not work client-side due to our CSP, and is not
something we wish to support server-side. The last rule aims to prevent
a subtle class of bugs, and to defend against a subset of prototype
pollution exploits:
- `no-new-func` to be compliant with our CSP, and to prevent code
execution from strings server-side:
https://eslint.org/docs/latest/rules/no-new-func
- `no-implied-eval` to be compliant with our CSP, and to prevent code
execution from strings server-side:
https://eslint.org/docs/latest/rules/no-implied-eval. Note that this
function implies that it prevents no-new-func, but I don't see [test
cases](https://github.com/eslint/eslint/blob/main/tests/lib/rules/no-implied-eval.js)
covering this behavior, so I think we should play it safe and enable
both rules.
- `no-prototype-builtins` to prevent accessing shadowed properties:
https://eslint.org/docs/latest/rules/no-prototype-builtins
In order to be compliant with `no-prototype-builtins`, I've migrated all
usages and variants of `Object.hasOwnProperty` to use the newer
[`Object.hasOwn`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/hasOwn).
## Summary
Part of https://github.com/elastic/kibana/issues/186530
Follow-up of https://github.com/elastic/kibana/pull/187064
The goal of this PR is to provide the necessary means to allow
implementing the [Counting
views](https://docs.google.com/document/d/1W77qoweixcjrq0sEKh_LjIk3j33Xyy9umod9mG9BlOM/edit)
part of the _Dashboards++_ initiative.
We do this by extending the capabilities of the _usage counters_ APIs:
* We support custom retention periods. Currently data is only kept in SO
indices for 5 days. Having 90 days worth of counting was required for
Dashboards++.
* We expose a Search API that will allow retrieving persisted counters.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
At the moment, our package generator creates all packages with the type
`shared-common`. This means that we cannot enforce boundaries between
server-side-only code and the browser, and vice-versa.
- [x] I started fixing `packages/core/*`
- [x] It took me to fixing `src/core/` type to be identified by the
`plugin` pattern (`public` and `server` directories) vs. a package
(either common, or single-scoped)
- [x] Unsurprisingly, this extended to packages importing core packages
hitting the boundaries eslint rules. And other packages importing the
latter.
- [x] Also a bunch of `common` logic that shouldn't be so _common_ 🙃
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/186530.
This PR sets the basis for allowing namespaced usage counters.
It relocates `usage-counters` SO type from `.kibana` to a dedicated
`.kibana_usage_counters`.
Furthermore, the original SO type is removed, and replaced by 2 separate
types:
* `server-counters`
* `ui-counters`
Note that these 2 steps are necessary if we want to leverage
`namespaces` property of the saved objects.
We can't currently update the `namespaceType: 'agnostic'` without
causing a migration.
Thus, these two types will be defined as `namespaceType: single`.
Up until now, UI counters were stored under a special `domainId:
uiCounter`.
This forced a workaround that consisted in storing `appName:eventName`
in the `counterName` property of the SO.
Having a dedicated SO type for them allows to store `appName` as
`domainId`, avoiding the need for a
[workaround](https://github.com/elastic/kibana/blob/main/src/plugins/usage_collection/common/ui_counters.ts).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Reopening https://github.com/elastic/kibana/pull/186326 with my account,
non-internal PRs are just terrible to work with
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@elastic.co>
## Summary
Addresses https://github.com/elastic/kibana/issues/177831.
The PR introduces specific steps to check that
`cluster.routing.allocation.enable` has a suitable value for _reindex
migrations_.
Up until now, this check was done systematically after the `INIT` step.
Now, a couple new dedicated steps have been introduced, which allow
verifying this setting on _reindex migrations_ only (highlighted in
orange):
## Summary
Addresses https://github.com/elastic/kibana/issues/185918
The idea is to simply check whether the index that a migrator is trying
to `write_block` (aka the source of the reindex operation) matches the
target index name. In this case:
* We assume that other migrators are half way through, ahead of us.
* We abort operation and trust other instances' migrators to finish the
job.
* Subsequent restart, when migration has finished, should basically be a
no-op.
## Summary
We recently "discovered" that ES calls performed from the SOR were not
having the default retry mechanism used by the ES client.
This is caused by the way we construct the ES client wrapper used for
the SOR, that forces the `maxRetries` option to `0`
8911bfabb0/packages/core/saved-objects/core-saved-objects-api-server-internal/src/lib/repository_es_client.ts (L38-L40)
I initially tried to fully get rid of `retryCallCluster` in
https://github.com/elastic/kibana/pull/184658, as we assumed that
re-enabling the retry mechanism would be sufficient, but this was
causing flakiness in test suites (uncaught rejections in plugins not
properly stopping after ES and Kibana are shut down, with
`NoLivingConnectionsError` errors...)
So instead, I just re-enabled the ES client native mechanism by removing
the `maxRetries` override.
## Summary
Closes https://github.com/elastic/ingest-dev/issues/3252
## Add integration
Added warning to Add integration when the integration requires root
privilege and the selected existing agent policy has unprivileged agents
enrolled.
To verify:
- enroll an agent with docker (it has unprivileged: true)
- try to add an integration that requires root e.g. auditd_manager
- verify that when trying to save the integration, the warning callout
is part of the confirm deploy modal
<img width="807" alt="image"
src="420da729-a4f4-4861-9767-001699629397">
## Add agent flyout
Added warning to Add agent flyout when an unprivileged agent is detected
in combination with an agent policy that has integrations requiring root
To verify:
- add an integration to an agent policy that requires root e.g.
auditd_manager
- open Add agent flyout, verify that the warning callout is visible
<img width="1273" alt="image"
src="e4ae1d73-358b-4d3c-9ca0-27e88bc734a6">
### Open question:
- Do we want to show the warning on `Add agent flyout` only for newly
enrolled agents (in the last 10 mins like we query enrolled agents), or
any unprivileged agents that are enrolled to this policy?
- Decision: No longer applicable as we decided to not show a count here
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This pull request introduces a new SO used for keeping relations between
artifactId and policyId. It addresses an issue where some users found
the single SO structure containing too many nested entries. Originally,
we planned to rewrite the existing Manifest Manager. However, during the
POC implementation, it became clear that the effort required to refactor
and retest the existing solution would be substantial. Therefore, this
pull request can be considered as the first step in transitioning our
approach from one SO to this new, distributed one.
The main idea behind these changes is to modify the structure of the SO,
rather than the logic of the Manifest Manager. To accomplish this, we
need to retrieve the SO from the new source, translate it into the
existing SO format (many SOs to one), execute the unchanged operations
of the Manifest Manager on artifacts, translate the resulting SO into
multiple SOs, and save them.
This change is expected to be deployed with a Feature Flag, and we need
to ensure that everything continues to function correctly in both cases.
Therefore, I've introduced a new FTR suite with the Feature Flag
enabled, which should be run alongside tests with the Feature Flag
disabled. This suite contains duplicated test files that depend on SO
logic. When we activate the Feature Flag, these tests should replace the
existing ones, as the Unified Manifest SO will become the default
approach.
It appears that there is no need to introduce any kind of migrations, as
the Manifest Manager is capable of recreating missing SOs (which has
been tested).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Follow-up of #176803
Now that `8.14` has been branched and it is about to be released, we can
remove the test that ensures the `HASH_TO_VERSION_MAP` is aligned.
This PR also rolls back a few changes on the map that shouldn't have
been performed, as they belong to `8.15.0`.
## Summary
Fix https://github.com/elastic/kibana/issues/183112
Add a new `mergeAttributes` option for both `update` and `bulkUpdate`,
which, when set to `false`, allows to perform a "full" update (fully
replacing the current attributes of the document) instead of a "partial"
one (merging the attributes).
Technically, ES doesn't really support it, so it was only made possible
due to the "client-side" update we implemented for ZDT / SO version BWC.
Closes https://github.com/elastic/kibana/issues/180377
## Summary
Add a new `support_agentless` property in agent policy and in
preconfiguration; this property is only allowed when the environment has
both `isServerless` set to `true` and `agentless` feature flag enabled,
otherwise policy creation/update will throw error `supports_agentless is
only allowed in serverless environments that support agentless feature`.
No UI change is required for now as this property will be needed as part
of a wider support to agentless policies.
## Testing
### Serverless
- Run serverless env configured for agentless following [this
guide](https://docs.elastic.dev/security-solution/cloud-security/serverless/develop-for-kibana#agentless-local-set-up)
- Make sure to have `agentless` feature flag enabled
- Create an agent policy with `support_agentless` property:
```
POST kbn:/api/fleet/agent_policies
{
"name": "New agent policy",
"namespace": "default",
"supports_agentless": true
}
```
- Update an existing agent policy with the new property:
```
PUT kbn:/api/fleet/agent_policies/<opolicy_id>
{
"name": "New agent policy",
"supports_agentless": true
}
```
- Create a preconfigured agent policy in kibana.dev.yml, and verify it
that it's correct via `GET kbn:/api/fleet/agent_policies`:
```
xpack.fleet.agentPolicies: [
{
"name": "Agentless Policy",
"id": "agentless",
"is_managed": true,
"namespace": "default",
"supports_agentless": true,
},
]
```
- Note that if `agentless` feature flag is disabled, any of the above
will throw an error.
### Stateful
Spin up a stateful env and verify that all of the previous commands fail
with `400` and above error message.
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/177323
## Summary
Allow to override `inputs` in `PUT package_policies/:id` endpoint. This
functionality will be useful for support and troubleshooting purposes,
but it shouldn't be used in place of normal updates to the policies.
- `inputs` parameters are saved in package policy SO `overrides` field
and then
[merged](https://github.com/elastic/kibana/pull/181453/files#diff-ee6c1fe752205768ab54e6107a869041bcbb6130a0c982fa169bf5aba570a30eR84)
to the full agent policy
- `compiled_streams` and `compiled_inputs` are not allowed
Example with an Nginx policy:
```
PUT kbn:/api/fleet/package_policies/d4fd9578-534f-4e1a-bc75-dfbd4ff0aa14
{
"overrides": {
"inputs": {
"logfile-system-d4fd9578-534f-4e1a-bc75-dfbd4ff0aa14": {
"log_level": "debug"
}
}
}
}
```
Result in full agent policy:
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR attempts to make it easier to quantity the time we're spending
waiting on ES during Kibana startup.
- Add a log entry once successfully connected to ES, surfacing the info
of how much time we waited.
- Add two new metric to our `kibana_started` event:
- the time we spent waiting for ES
- the time it took to perform the SO migration
Note that for "BWC" reasons (primarily - and simplicity's sake too)
we've not subtracting the time we spent from the `start` lifecycle
timing we already had.
## Summary
Part of https://github.com/elastic/kibana/issues/169547
View docs at [Changed
pages](https://kibana_169928.docs-preview.app.elstc.co/diff)
Add monitor api public api
### Testing
Make sure you have some monitors populated before testing this PR and
before switching to the branch
- [ ] Try editing already added monitors via API
- [ ] Test adding monitors via API, and then edit those via and
subsqeuently try editing via API the same monitor
- [ ] Test editing monitors via API
- [ ] Test deleting monitors via API
- [ ] Test getting monitors via API
- [ ] Testing private as well public locations
Basic workflow that i am interesting in testing is to make sure, you can
add/edit via both API and UI without any issues
Test each of HTTP/TCP/ICMP browser examples
<img width="1728" alt="image"
src="3575d93a-5f04-4c80-ac62-038643f466f8">
---------
Co-authored-by: Justin Kambic <jk@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co>
