## Summary
This PR adds support for parsing multiple field mappings from an
existing pipeline configuration. This is only relevant when listing
attachable pipelines; the new pipeline creation process is not affected.
This change is a pre-requisite for allowing ELSER pipelines to be
attached.
Tested for backward compatibility:
* Existing pipeline selection dropdown (temporarily disabled filtering
of ELSER pipelines)
Note the "Source field" and "Destination field" attributes show the
first mapping's fields only (even if the pipeline is configured with
multiple mappings). This is pending redesign and will be addressed in a
later PR.
* Fields - ELSER pipeline (temporarily removed field modification
widgets)
* Fields - non-ELSER pipeline
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- The new profiling items will be only visible when the profiling plugin
has been already installed. Otherwise, these are going to be hidden.
- The profiling plugin exposes three new Locators to facilitate the
navigation to the Flamegraph, TopN functions and Stacktraces pages.
- Add `new` badge property on the section component
<img width="486" alt="Screenshot 2023-06-14 at 1 55 09 PM"
src="6e353bfc-050c-4294-a4e4-fc46205d5d0e">
234863a4-0d99-4140-a5b5-702896b2c4a3
ee1635bd-5127-41d6-b434-4cee9b5ebe92
46ec9bb7-2cd0-43fc-9a1e-0d6eef70612f
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#155190
## Summary
This PR adds the possibility to pin different rows inside the metadata
table in asset details embeddable. The pins are persisted in the local
storage and should be available after refreshing/reopening the host
flyout. The order and sorting are explained in [this
comment](https://github.com/elastic/kibana/issues/155190#issuecomment-1523335704),
so basically we keep the original sorting order of the table (`host`,
`cloud`, `agent`) also for the pins.
## Testing
- Go to hosts view and open a single host flyout (metadata tab)
- Try to add / remove pins
- Check if the pins are persisted after a page refresh
62873e7e-b5f0-444c-94ff-5e19f2f46f58
## Summary
- Updates the `fleet:check-deleted-files-task` to include the indexes
that store files for delivery to the Host (currently used only by
Endpoint integration)
## Summary
This PR adds support for applying a KQL filter to the good/total
metrics.
<img width="858" alt="image"
src="c271352c-10fd-49f1-89b8-a352b69f7f7c">
## Summary
This PR fixes#161101 by changing the server side validation to allow a
null value for `maxBurnRateThreshold`. This value is only used on the
client side to display the `X hours until error budget exhaustion.`
message. This value is also used in the client validation but due to the
architecture of how client side validation works, we have to add this to
the params.
Closes https://github.com/elastic/security-team/issues/7048
`addAlertsToCase` func from `add_alerts_to_case.ts` that is being used
only in the two failing tests had `perPage` set to `10000` which started
throwing `The provided perPage value is too high. The maximum allowed
perPage value is 100.`.
## Summary
This PR adds current year dynamcially to notify user HTML email template
footer.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Connected to https://github.com/elastic/kibana/issues/146945
## Summary
| Description | Limit | Done? | Documented?
| ------------- | ---- | :---: | ---- |
| Total number of comment characters | 30.000 | ✅ | Yes
|
- Tests.
- Updated Documentation.
### Checklist
Delete any items that are not applicable to this PR.
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Release Notes
Post and Patch comment API limits total number of characters per comment
to 30000
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/158986
This PR is applying a very simple fix on the unified search bar. It
makes the dataview to not be required to load the text area.
The dataview(s) is required for the suggestions but not for running a
query.
There is currently a bug
https://github.com/elastic/kibana/issues/158986#issuecomment-1620099640
on the apm side. It is happening because the url update is not allowed
when the dataview is not set. This needs to be fixed on the apm side so
I synced with the apm team and they are going to take it.
Add multiple formats for geo_point fields and make geo conversion tools
part of field_format/common/utils
This is needed because various users need locations displayed in
different formats. It would be problematic to allow users to change the
index field formatting for everyone.
WKT and LL where already formatters availible.
I added MGRS and DMS.
## Summary
Summarize your PR. If it involves visual changes include a screenshot or
gif.
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Sean Sullivan <ssullivan@spectric.com>
Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
Closes#159368
## Summary
I saw that the test was failing because of a missing environment set. I
guess that happened because the test runner doesn't include it like it
is included locally so I removed that check so the test shouldn't be
flaky.
There was some failure, for IM rule tests.
Probably because we are comparing alerts in the array, they can be in a
different order, so sorting should help
## Summary
Resolves: https://github.com/elastic/kibana/issues/157883
# Summary:
This PR acts as a POC for how we would want to version our rule API
types in preparation for the versioning of our HTTP endpoints.
There is now an ES persisted type (called `RuleAttributes`, akin to the
old `RawRule` type). This type should never be used at the business
logic level as we want the ability not to reference saved object
attributes directly in the application. Instead, we should transform
this saved-object to its corresponding domain object type.
HTTP types (`CreateBodyParams`, `RuleResponse`) are now located in
`x-pack/plugins/alerting/common/routes/rule` with a versioning structure
like:
And follows the guideline here:
https://docs.elastic.dev/kibana-dev-docs/versioning-interfaces
Domain object types (rule for example) are derived from the
`config-schema` schemas using `TypeOf`, this was done to facilitate the
reuse of validation schemas that we might want to run for strict IO
validation, potentially at the `rulesClient` level.
## API:
Only the `createRule` endpoint has been converted for the sake of this
POC, other endpoints might have broken types that will be fixed once we
have a finalized pattern for dealing with versioned types.
At the API route level, I think it would be wise to import versioned
types in our APIs, this way, it forces the developer to deal with broken
types when we have a breaking change to our rule domain object.
The API route level is also responsible for transforming domain objects
to response types, usually, this just entails renaming property names
from camel case to snake case.
in `create_rule_route.ts`
```ts
import type { CreateRuleRequestBodyV1, CreateRuleRequestParamsV1 } from '../../../../common/routes/rule/create';
import type { RuleParamsV1 } from '../../../../common/routes/rule/rule_response';
...
// In the Handler:
const rulesClient = (await context.alerting).getRulesClient();
const createRuleData: CreateRuleRequestBodyV1<RuleParamsV1> = req.body;
const params: CreateRuleRequestParamsV1 = req.params;
const createdRule: Rule<RuleParamsV1> = (await rulesClient.create<RuleParamsV1>({
data: transformCreateBodyV1<RuleParamsV1>(createRuleData),
options: { id: params?.id },
})) as Rule<RuleParamsV1>;
const response: CreateRuleResponseV1<RuleParamsV1> = {
body: transformRuleToRuleResponseV1<RuleParamsV1>(createdRule),
};
return res.ok(response);
```
### RulesClient -> Create
At the rules client level, we now need to transform saved-objects to the
domain objects after we perform CRUD operations on them. We can also
consider validating schemas here too since other solutions are using our
rules client directly instead of through the API.
I don't think we need to version rules client input and output types
since the route level types should deal with breaking changes already.
Therefore, we can freely import the latest of any domain object types
(Rule, for example)
```ts
import { Rule, RuleDomain, RuleParams } from '../types';
```
The flow is the following:
```
1. calling rulesClient.create() with API body and params.
2. perform schema validation on params
3. perform other validation not achievable using config-schema
4. perform rule create -> returns a `savedObject` type
5. convert `savedObject` ES type to the domain object type (`RuleDomain`)
6. convert the domain object type to a public domain object type (`Rule`)
7. We can also validate the created rule using config-schema
```
# Open questions:
- Should we validate input params at both the route and the rules client
level? I think we should since our rules client is shared.
- How do we want to version and validate rule params? Right now they're
typed as generics.
- Should we leave all date fields as `string`, even for domain objects?
Since config-schema is unable to define a `Date` type.
- Should we support partial rule domain object types at all? I can't
really think why we should even for `updates`, since we need to
reconstruct the overall rule to send back to the client.
- Should we have `undefined | null` types in the domain object? I know
we need `null` specifically for the ES types since there is a different
between updating a field as `undefined` or `null`, but this shouldn't
manifest itself in the business logic.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
**Fixes: https://github.com/elastic/kibana/issues/161094**
## Summary
- Adds support for rule type changes in the
`/internal/detection_engine/prebuilt_rules/upgrade/_review` endpoint.
- Previously, if any rule had a different `type` in its
`current_version` compared to its `target_version` the request would
fail with `500`.
- This PR:
- updates this behaviour to accept rule type changes
- creates a new `calculateAllFieldsDiff` method that is responsible for
calculating diffs among all fields of all rule types. Used exclusively
when there has been a rule type change between the current version and
the target version (which can normally happen through upgrades of the
`security_detection_engine` package) OR when the base version has a
different type as the current version (which should not happen under
normal conditions and user behaviour).
- updates the diffable fields types for each specifc rule type (e.g.:
`DiffableCustomQueryFields`,`DiffableEqlFields`,`DiffableThreatMatchFields`,
etc) , replacing the `data_query` field name for either `eql_query` (for
EQL type rules) or `kql_query` (for all others).
## How to test
1. With a clean Kibana state, use the
`xpack.securitySolution.prebuiltRulesPackageVersion` config to force
Kibana to install a package that contains the rules with their original
type:
```
xpack.securitySolution.prebuiltRulesPackageVersion: '8.3.1'
```
2. Install the four "offending" rules, [listed
below.](https://github.com/elastic/kibana/pull/161247#issuecomment-1622132120)
3. Remove the config, restart Kibana and navigate to the Rules Page so
that the latest package is installed.
4. Navigate to the Rule Updates table. The four installed rules should
have updates available. Update them.
5. All the listed rule types should be updated, as well as their
corresponding fields.
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: jpdjere <jpdjeredjian@gmail.com>
## Summary
- [x] Adds an agent policy watcher to check if an agent policy is valid
for a specific license. If it is not valid, it will set the
platinum-only features (as of now, only tamper protection) to false.
- [x] Moves `generateNewAgentPolicyWithDefaults` up to `common/services`
- [x] Unit Tests
# Screenshot
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Removing two endpoints in the kibana server which are no longer used by
ML on the client side.
`/anomaly_detectors/${jobId}/_reset`
This has been superseded by `/jobs/reset_jobs` which takes multiple job
IDs
`/anomaly_detectors/{jobId}/results/records`
I believe this has never been used, as it didn't have a client side
function.
Relates to https://github.com/elastic/kibana/issues/157980
## Summary
Combines the two following test plans into one:
- [Test plan for the legacy workflows of installing/upgrading prebuilt
rules](https://docs.google.com/document/d/1d_1DYnHlnCaPznWTjeCxhoaRUwxc2O_V0LToAPG0xLE/edit)
- this google doc is deprecated and will be replaced by the consolidated
test plan created in this PR
- [Test plan for the new workflows of installing/upgrading prebuilt
rules](https://docs.google.com/document/d/1cYvwtpzk0uLn5R88BlCX-fRIwR2n_NoqkSjIXkY9k34/edit)
- this google doc is deprecated and will be replaced by the consolidated
test plan created in this PR
Adds a template for writing new test plans.
---------
Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This tiny PR ensures that widget contents are not rendered until
expanded (in the expandable flyout). This will
prevent unnecessary requests being sent when we open the flyout.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
closes: https://github.com/elastic/kibana/issues/159685
- Renaming _x-pack/plugins_:
`serverless_security` -> `security_solution_serverless`
`ess_security` -> `security_solution_ess`
- All the related configurations and types have also been renamed.
- i18n translation prefixes updated
- relocation of internal `security_solution_serverless` directories to
be consistent with `security_solution_ess`
### Eslint
I also added the plugins in the `.eslintrc` configuration, defining the
same rules as the `security_solution` plugin.
All eslint errors have been addressed (mainly _type_ imports errors)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Connected to https://github.com/elastic/kibana/issues/146945
## Summary
| Description | Limit | Done? | Documented?
| ------------- | ---- | :---: | ---- |
| Total number of cases/user actions/comments per page | 100 |
✅ | No | N/A |
| Maximum number of cases/user actions/comments returned from the API |
10.000 | ✅ | No | N/A |
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Release Notes
Max value for perPage parameter in find Cases API is now 100.
Max value for perPage parameter in find User Actions API is now 100.
Max value for perPage parameter in find Comments API is now 100.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: lcawl <lcawley@elastic.co>
Fixes#161117
## Summary
If `xpack.observability.unsafe.alertDetails.logs.enabled` is enabled, we
will use the new alert details page URL in `context.alertDetailsUrl`
otherwise, we send the user to the alerts page filtered for that alert.
(Partially brings back [the logic for alert details
URL](https://github.com/elastic/kibana/pull/157987/files#diff-a71ca536380c1fde8805744b23566ce795707f92b94a03af73347cac46ccac63L1027)
and
[getAlertDetailsConfig](https://github.com/elastic/kibana/pull/157987/files#diff-a71ca536380c1fde8805744b23566ce795707f92b94a03af73347cac46ccac63L1027))
## 🧪 How to test
1. Set `xpack.observability.unsafe.alertDetails.logs.enabled` as false
in Kibana yml config or remove the config
2. Create a log threshold rule with an action for both active state and
recovered state
3. When the alert is triggered, check the default message, it should
include the alertDetailsURL, by clicking on that, you should land on the
alerts page filtered for that alert
4. Make the alert recovered and check and similar URL should be
generated
New alert details page:
1. Set `xpack.observability.unsafe.alertDetails.logs.enabled` as true in
Kibana yml config
2. Repeat the steps 2,3,4 as mentioned before
3. This time, you should land on the new alert details page
