Commit graph

105 commits

Author SHA1 Message Date
Jean-Louis Leysens
53b0015340
[OAS] Remove Elastic-Api-Version (#202923)
## Summary

Due to the much slower rate of versioning public APIs we have decided to
remove the Elastic-Api-Version body and header information from the spec
entirely.

This also cleans up the spec when rendered on our hosted OAS docs.

Removes:
1. `Elastic-Api-Version` from the request/response bodies for all public
APIs
2. `Elastic-Api-Version` header parameter from all public APIs

Docs for internal APIs will still have this included as they always
require a version environments.

## To revewers

Important changes are all in `packages/kbn-router-to-openapispec` and
`src/core/server/integration_tests/http/oas.test.ts`, the rest is very
minor or generated/snapshotted changes.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-12-05 17:05:42 +01:00
Nick Peihl
c8866e4ce3
[Dashboards][OAS] Generate API docs for Dashboards API (#199215) 2024-12-04 17:33:10 -05:00
Julia Bardi
230d6617ab
[Fleet] fix schema validation to allow undefined/null (#202732)
## Summary

Fix a few issues encountered with schema validation.

One of them reported here:
https://discuss.elastic.co/t/fleet-error-updating-policy-settings/371332

The other encountered locally when testing upgrades:
```
"Failed output validation: [request body.items.0.upgrade_details]: expected a plain object value, but found [null] instead."
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-12-03 11:12:10 -06:00
Kevin Delemme
8fe4c44192
feat(slo): allow configuration of advanced settings from UI (#200822) 2024-12-02 16:19:17 -05:00
Lisa Cawley
54370b209c
[OpenAPI] Fix Serverless API base URL (#202373) 2024-12-02 12:09:03 -08:00
Jesus Wahrman
9b99070470
[docs] Remove experimental message from saved objects import and export apis (#202173)
## Summary

resolves https://github.com/elastic/kibana/issues/159454

Remove experimental message from saved objects import and export apis.


### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-12-02 12:05:52 +01:00
Pablo Machado
06b7993bd9
[SecuritySolution] Entity Engine status tab (#201235)
## Summary

* Add two tabs to the Entity Store page
  * The import entities tab has all the bulk upload content
  * The status tab has the new content created on this PR
* Move the "clear entity store data" button to the header according to
design mockups.
* Delete unused stats route
* Rename `enablement` API docs to `enable`
* Add a new parameter to the status API (`withComponents`)
  * Should I make it snake cased?

### import entities tab
![Screenshot 2024-11-27 at 15 07
01](https://github.com/user-attachments/assets/c433e217-781e-4792-8695-2ee609efa654)


### status tab
![Screenshot 2024-11-27 at 15 07
20](https://github.com/user-attachments/assets/8970c023-22b3-4e83-a444-fa3ccf78ea42)


## How to test it
- Open security solution app with data
- Go to entity store page
- You shouldn't see the new tab because the engine is disabled
- Enable the engine and wait
- Click on the new tab that showed up
- It should list user and host engine components, and everything should
be installed
- Delete or misconfigure some of the resources, the new status should be
reflected on the tab.


## TODO:
- [x] Rebase main after https://github.com/elastic/kibana/pull/199762 is
merged
  - [x] Remove temporary status hook
- [x] Fix the"clear entity data" button. It should re-fetch the status
API.







### Checklist

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-29 09:29:04 -06:00
Jen Huang
3188cda4e3
[UII] Add status tracking for agentless integrations (#199567)
## Summary

Resolves https://github.com/elastic/ingest-dev/issues/3933. For
deployments that support agentless, integrations with agentless
deployment mode enabled will allow the status of agentless integration
policies to be tracked.

### Key technical changes

- A new field `supports_agentless` was added to package policies. This
field already exists on agent policies. When an agentless integration is
created, `supports_agentless: true` is now added to both the package
policy and its parent agent policy.
- This allows easier filtering for agentless integrations as we avoid
having to retrieve & check against every parent agent policy.
- This also means existing agentless policies do not get this new status
tracking UI, only new ones created after this change. Since agentless is
not yet GA, I think this is okay.
- `/api/fleet/agent_status/data` now takes optional query params
`pkgName` and `pkgVersion`. When both are specified, the API will check
if agent(s) have ingested data for only that package's datastreams.

## UI walkthrough
<details>
<summary>🖼️ Click to show screenshots</summary>

1. **Integration policies** page now shows two tables for integrations
meeting the above condition, one for agentless policies and one for
agent-based policies:


![image](https://github.com/user-attachments/assets/58c6a932-9bda-4229-ba5f-d341bdbd539a)

2. Clicking the status badge in the agentless policies table opens a
flyout with two steps: confirm agentless enrollment and confirm incoming
data:


![image](https://github.com/user-attachments/assets/e19e6ba0-f40d-48a7-a524-0373934ac46a)

3. Confirm agentless enrollment polls for an agent enrolled into that
integration policy's agent policy. If that agent is reporting an
unhealthy status, the integration component UI is shown. This UI is the
same one used on Fleet > Agents > Agent details page and shows all
components reported by that agent:


![image](https://github.com/user-attachments/assets/ce214f7f-4bdd-48e5-a5eb-a1e8fcc7a512)

4. Once a healthy agentless enrollment is established, confirm incoming
data starts polling for data for that integration ingested by that agent
ID in the past 5 minutes:


![image](https://github.com/user-attachments/assets/7f3de40b-3418-4174-b529-e805407949b6)

5. If data could not be retrieved in 5 minutes, an error message shows
while polling continues in the background:


![image](https://github.com/user-attachments/assets/a3fd198e-1570-4357-9b7f-e541a769d33f)

6. If data is retrieved, a success message is shown:


![image](https://github.com/user-attachments/assets/f4e442af-ca60-4448-9bfb-3f244cd03c2d)
</details>

## Testing
Easiest way to test is use the Cloud deployment from this PR. Enable
Beta integrations and navigate to CSPM. Add a CSPM integration using
`Agentless` setup technology. Then you can track the status of the
agentless deployment on the Integrations policies tab.

For local testing, the following is required to simulate agentless
agent:
1. Add the following to kibana.dev.yml:
```
xpack.cloud.id: 'anything-to-pass-cloud-validation-checks'
xpack.fleet.agentless.enabled: true
xpack.fleet.agentless.api.url: 'https://localhost:8443'
xpack.fleet.agentless.api.tls.certificate: './config/certs/ess-client.crt'
xpack.fleet.agentless.api.tls.key: './config/certs/ess-client.key'
xpack.fleet.agentless.api.tls.ca: './config/certs/ca.crt'
```
2. Apply [this
patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to prevent attempt to create agentless pod
3. Enroll a Fleet Server as usual
4. Enable Beta integrations and navigate to CSPM. Add a CSPM integration
using `Agentless` setup technology.
5. Enroll a normal Elastic Agent to the agent policy for that CSPM
integration by using the token from Enrollment tokens

## To-do
- [x] API tests
- [x] Unit UI tests
- [x] Manual Cloud tests
- [x] File docs request
  - https://github.com/elastic/ingest-docs/issues/1466
- [ ] Update troubleshooting guide link once available

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-26 09:12:14 +01:00
Julia Bardi
973c69533b
[Fleet] flag package policy SO to trigger agent policy bump (#200536)
## Summary

Closes https://github.com/elastic/kibana/issues/193352

Update:

Using a new SO field `bump_agent_policy_revision` in package policy type
to mark package policies for update, this will trigger an agent policy
revision bump.

The feature supports both legacy and new package policy SO types, and
queries policies from all spaces.

To test, add a model version change to the package policy type and save.
After Fleet setup is run, the agent policies using the package policies
should be bumped and deployed.
The same effect can be achieved by manually updating a package policy SO
and loading Fleet UI to trigger setup.
```
        '2': {
          changes: [
            {
              type: 'data_backfill',
              backfillFn: (doc) => {
                return { attributes: { ...doc.attributes, bump_agent_policy_revision: true } };
              },
            },
          ],
        },

  curl -sk -XPOST --user fleet_superuser:password -H 'content-type:application/json' \     -H'x-elastic-product-origin:fleet' \
     http://localhost:9200/.kibana_ingest/_update_by_query -d '
     { "query": {
      "match": {
        "type": "fleet-package-policies"
      }
    },"script": {
      "source": "ctx._source[\"fleet-package-policies\"].bump_agent_policy_revision = true",
      "lang": "painless"
    }
  }'

```

```
[2024-11-20T14:40:30.064+01:00][INFO ][plugins.fleet] Found 1 package policies that need agent policy revision bump
[2024-11-20T14:40:31.933+01:00][DEBUG][plugins.fleet] Updated 1 package policies in space space1 in 1869ms, bump 1 agent policies
[2024-11-20T14:40:35.056+01:00][DEBUG][plugins.fleet] Deploying 1 policies
[2024-11-20T14:40:35.493+01:00][DEBUG][plugins.fleet] Deploying policies: 7f108cf2-4cf0-4a11-8df4-fc69d00a3484:10
```

TODO:
- the same flag has to be added on agent policy and output types, and
the task extended to update them
  - I plan to do this in another pr, so that this doesn't become too big
- add integration test if possible

### Scale testing
Tested with 500 agent policies split to 2 spaces, 1 integration per
policy and bumping the flag in a new saved object model version, the
bump task took about 6s.
The deploy policies step is async, took about 30s.
```
[2024-11-20T15:53:55.628+01:00][INFO ][plugins.fleet] Found 501 package policies that need agent policy revision bump
[2024-11-20T15:53:57.881+01:00][DEBUG][plugins.fleet] Updated 250 package policies in space space1 in 2253ms, bump 250 agent policies
[2024-11-20T15:53:59.926+01:00][DEBUG][plugins.fleet] Updated 251 package policies in space default in 4298ms, bump 251 agent policies
[2024-11-20T15:54:01.186+01:00][DEBUG][plugins.fleet] Deploying 250 policies

[2024-11-20T15:54:29.989+01:00][DEBUG][plugins.fleet] Deploying policies: test-policy-space1-1:4, ...
[2024-11-20T15:54:33.538+01:00][DEBUG][plugins.fleet] Deploying policies: policy-elastic-agent-on-cloud:4, test-policy-default-1:4, ...

```

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-25 10:59:18 +01:00
Jan Monschke
82108f134e
[SecuritySolution][Timeline] Refactor timeline HTTP API (#200633)
## Summary

The timeline API endpoints are currently implemented from a mix of HTTP
and GraphQL practices. Since GraphQL has been removed for a long time
now, we should make sure the endpoints conform to HTTP best practices.
This will allow us to simplify the API- and client-logic. Further,
third-parties accessing these APIs will have an easier time integrating.

### Usage of HTTP status codes

Depending on the error, the API endpoints currently return a `200` with
`{ code: 404, message: '(...)' }` or an actual HTTP error response with
e.g. `403` and the message in the body. The practice of returning 200s
with embedded error codes comes from GraphQL, where error codes are
always embedded.

Example of a current HTTP response of a failed timeline request:

```
HTTP status: 200
HTTP body:
{
  "error_code": 409,
  "messsage": "there was a conflict"
}
```

Going forward, all endpoints should return proper error codes and embed
the error messages in the response body.
```
HTTP status: 409
HTTP body:
{
  "messsage": "there was a conflict"
}
```

### Removal of `{}` responses

Some timeline endpoints might return empty objects in case they were not
able to resolve/retrieve some SOs. The empty object implies a `404`
response. This creates complications on the client that now have to
provide extra logic for how to interpret empty objects.

Example of a current request of one of the endpoints that allows empty
responses.
```
HTTP status: 200
{}
```
The absence of an object, on some of the listed endpoints, indicates a
404 or the top-level or embedded saved object.

Going forward, the endpoints should not return empty objects and instead
return the proper HTTP error code (e.g. `404`) or a success code.

```
HTTP status: 404
```

### No more nested bodies

Another relic of the GraphQL time is the nesting of request bodies like
this:

```
HTTP status: 200
HTTP body:
{
  "data": {
    "persistTimeline": {
      (actual timeline object)
    }
  }
}
```

Combined with sometimes returning empty objects and potentially
returning a status code in the body, makes it overly complicated for
clients to reason about the response.

Going forward, the actual object(s) should be returned as a top-level
JSON object, omitting `data.persistX`.
```
HTTP status: 200
HTTP body:
{
  (actual timeline object)
}
```

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-11-22 07:54:45 +01:00
Lisa Cawley
3938acc83d
[DOCS] Remove technical preview from serverless APIs (#201054) 2024-11-21 09:45:10 +01:00
Jen Huang
0aa63a7ecc
[UII] Expose advanced file logging config in UI (#200274)
## Summary

Resolves [#192237](https://github.com/elastic/kibana/issues/192237).
This PR exposes the following Elastic Agent file logging configuration
options in the agent policy advanced settings UI:

```
agent.logging.to_files
agent.logging.files.rotateeverybytes 
agent.logging.files.keepfiles
agent.logging.files.interval
```

<img width="1237" alt="image"
src="https://github.com/user-attachments/assets/8de9023c-29a0-4ecf-803a-d8c0c4b87616">

This PR also does some clean up on the default values for all these
configured advanced settings so that when user has not touched them, the
default values do not get written into the agent policy saved object.
[More info
here](https://github.com/elastic/kibana/pull/200274#discussion_r1849142612).

It also fixes adds missing response schemas for the advanced settings.

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-20 09:41:50 -08:00
Tiago Vila Verde
3757e64127
[Entity Analytics] [Entity Store] Refactor entity store enablement (server side) (#199638)
## Summary

This PR adds 2 new endpoints regarding enablement of the Entity Store:
* `api/entity_store/enable`, which initializes entity engines for both
`user` and `host` entities
* `api/entity_store/status`, which computes a global store status based
on the individual engine status

In addition, running initialization of multiple engines in parallel is
now allowed.


### How to test

1. Use dev tools to call `POST kbn:/api/entity_store/enable`
2. Check that two engines were created and that the status is
`installing` by calling `GET kbn:/api/entity_store/status`
3. Wait a few seconds and keep calling the `status` endpoint. Once
initialization finishes, the status should switch to `running`

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-19 14:11:24 +01:00
Lisa Cawley
ba6ffec9e7
[OpenAPI] Fix fleet filepath API parameter (#199538) 2024-11-14 16:05:39 -06:00
Alex Szabo
364b226019
[CI] Fix OAS snapshot case (#200241)
## Summary
Fixes currently broken on-merge by committing the OAS update from
capture_oas_snapshot.sh
2024-11-14 19:58:21 +00:00
Antonio
e2702ff591
[ResponseOps] [Cases] Attach file to case API (#198377)
Fixes #22832

## Summary

This PR adds the possibility of adding Files/Attachments to Case in
Kibana via an API call.

### How to test

The new API URL is `https://localhost:5601/api/cases/<CASE_ID>/files`.
You can either use postman or curl to test.

1. Start by creating a case.
2. Call the new API
```
curl --location 'https://localhost:5601/api/cases/<CASE_ID>/files' \
--header 'kbn-xsrf: true' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--form 'filename="Notice"' \
--form 'mimeType="text/plain"' \
--form 'file=@"<FULL_PATH_TO_THE_FILE_YOU_WANT_TO_UPLOAD>"'
```
<img width="1090" alt="Screenshot 2024-10-30 at 15 41 26"
src="https://github.com/user-attachments/assets/b018f92d-2603-4bf1-ac12-f01452f35303">

3. Confirm the user action was created.
<img width="383" alt="Screenshot 2024-10-30 at 15 48 45"
src="https://github.com/user-attachments/assets/04952b8f-e8fb-4f19-a72f-54030f496fe9">

4. Confirm the file exists in the case and:
    - it can be downloaded as expected.
    - it can be previewed as expected(not every MIME type allows this).


### Release Notes

Files can now be attached to cases directly via API.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: lcawl <lcawley@elastic.co>
2024-11-14 18:05:11 +01:00
Lisa Cawley
50f0016cd7
[OpenAPI][DOCS] Add descriptions for alerting rule flapping properties (#200112) 2024-11-14 07:54:51 -08:00
Nicolas Chaulet
a21743cb44
[Fleet] Fix OAS snapshot (#200186)
## Summary

Fix OAS snapshot

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-11-14 15:51:47 +01:00
Nicolas Chaulet
75679b21b1
[Fleet] Revert enrollment api key list removal (#200087) 2024-11-14 07:52:13 -05:00
Jean-Louis Leysens
8278b0650a
[OAS] Publish OAS bundles to bump.sh (#197482)
## Summary

Publish OAS docs to bump.sh on merge to `main` or `8.x`

## To reviewers
* For now actual publication requires a manual step on bump.sh (so
things aren't going live immediately)
* Will get to serverless OAS docs next!

## Blockers

* Address vulnerable deps before merging:
https://github.com/bump-sh/cli/issues/583

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-14 09:15:47 +01:00
David Sánchez
a79695deff
[Security Solution][EDR Workflows] Remove deprecated API's (#199598)
## Summary

- This adds api removals for deprecated API's.
- Also updates OAS documentations and api types.
- Also updates/removes test.

These changes are intended to be for `9.0` and Serverless GA.

API's removed:
- `/api/endpoint/isolate`
- `/api/endpoint/unisolate`
- `/api/endpoint/policy/summaries`
- `/api/endpoint/suggestions/{suggestion_type}`
- `/api/endpoint/action_log/{agent_id}`

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-13 10:59:20 +01:00
Kevin Delemme
efcc2ab004
chore(slo): Migrate to server-route-repository (#198726) 2024-11-12 16:24:53 -05:00
Maxim Palenov
06986e4a86
[Security Solution] Add Alert Suppression editable component (#198673)
**Partially addresses:** https://github.com/elastic/kibana/issues/171520

## Summary

This PR adds is built on top of https://github.com/elastic/kibana/pull/193828 and https://github.com/elastic/kibana/pull/196948 and adds an Alert Suppression editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow.

## Details

https://github.com/elastic/kibana/issues/171520 required adding editable components for each field diffable rule field. Alert Suppression edit component was extracted from Define Rule Step Component into a separate reusable component. To simplify the logic it was split into common Alert Suppression and Threshold Alert Suppression since the latter is a specific use case.

## Caveats

Upgrade prebuilt rules workflow is quite different from rule creation and editing. In create and edit rule forms users are capable to change any field at their will. Upgrade prebuilt rules workflow allow to modify only specific fields having diff in the current rule upgrade.

There are fields which depend on each other. In particular Alert Suppression isn't supported for EQL sequence though it's addressed in https://github.com/elastic/kibana/pull/189725. 

- Alert Suppression editable component in Three Way Diff workflow isn't disabled EQL sequence rule queries. Alert suppression support for rules with EQL sequence queries is implemented in https://github.com/elastic/kibana/pull/189725. 

- Machine learning rule type require running selected machine learning jobs otherwise input could be disabled in case of there are no fields to pick from otherwise a warning message below the combobox is shown.

## How to test

The simplest way to test is via patching installed prebuilt rules via Rule Patch API. Please follow steps below

- Enable Prebuilt rule customization feature by adding a `prebuiltRulesCustomizationEnabled` feature flag
- Run Kibana locally
- Install a prebuilt rule, e.g. `Potential Code Execution via Postgresql` with rule_id `2a692072-d78d-42f3-a48a-775677d79c4e`
- Patch the installed rule by running a query below

```bash
curl -X PATCH --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"rule_id":"2a692072-d78d-42f3-a48a-775677d79c4e","version":1,"alert_suppression":{"group_by":["host.name"]}}' http://localhost:5601/kbn/api/detection_engine/rules
```

- Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `Potential Code Execution via Postgresql` rule -> expand `EQL Query` to see EQL Query -> press `Edit` button

## Screenshots

Custom query prebuilt rule (UI looks similar for EQL, Indicator Match, New Terms and ES|QL rule types)

![image](https://github.com/user-attachments/assets/86015d5b-e252-4d0b-9aa3-fc14679a493b)

Machine learning prebuilt rule with a diff in alert suppression

![image](https://github.com/user-attachments/assets/210246cd-27fd-4976-befc-dee023101ec9)

Threshold prebuilt rule

![image](https://github.com/user-attachments/assets/44b0c1bc-4134-4d58-bd9a-e8e2d4c50802)
2024-11-12 08:46:39 -06:00
Lisa Cawley
ef0428ffec
[OpenAPI][Fleet] Add missing operation summaries (#199548) 2024-11-08 14:13:10 -08:00
Nicolas Chaulet
4a568964e4
[Fleet] Remove deprecated topics property for kafka output in favor of topic (#199226) 2024-11-08 16:45:09 +01:00
Christiane (Tina) Heiligers
0df2e98381
Changes deprecated SO HTTP APIs deprecation field to object (#197936)
fix https://github.com/elastic/kibana/issues/197721.

The route deprecation field changed from a boolean to an object, where
the object contains information that is used in deprecation issues that
the Upgrade Assistant shows.

This PR makes the necessary changes in the deprecated Saved Objects HTTP
APIs.

This PR also includes a release notes entry for the API deprecations
that was missing.
![Screenshot 2024-10-29 at 12 01
29](https://github.com/user-attachments/assets/5c47c697-fbae-4b2e-8c6c-cd4701a667df)

### Checklist

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


### How to test this:

- Pull PR, run es against a trial license and start kibana as usual.
- Make a curl request to Kibana to get the config saved object: 

```
curl --location 'localhost:5601/abc/api/saved_objects/config/9.0.0' \
--header 'Content-Type: application/json' \
--header 'Accept-Encoding: gzip, deflate, br' \
--header 'kbn-xsrf: kibana' \
--header 'Kbn-Version: 9.0.0' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='
```

- Navigate to Upgrade Assistant and observe Kibana has at least 1
deprecation warning.
- View Kibana's warnings, you should see a warning entry for `The "GET
/api/saved_objects/{type}/{id}" route is deprecated`

![Screenshot 2024-11-06 at 16 26
26](https://github.com/user-attachments/assets/3b6a5644-3e5e-403e-a0f6-015686675b9f)

- click on the deprecation and you should see more detail about the
deprecated API that's been used in the flyout:

![Screenshot 2024-11-06 at 16 26
44](https://github.com/user-attachments/assets/696aaf8f-fb6b-4c61-bc3c-b3745f85059a)

- resolve the deprecation warning
- Kibana should continue to issue requests to the deprecated SO HTTP
APIs because these APIs have not been removed yet.


### Risk Matrix

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| End user concern from the deprecation warning that says the routes
have been deprecated | Low | Low | The APIs have been deprecated since
8.7 and recommends using public APIs instead. |

### For maintainers

- [x] This will appear in the **Release Notes** and follow the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-07 14:49:05 -06:00
Jean-Louis Leysens
08cf1eed3b
[OAS] Delete (empty) staging files again (#199152) 2024-11-07 13:25:08 +01:00
Nicolas Chaulet
15c1ceb475
[Fleet] Remove deprecated epm APIs (#198434) 2024-11-06 08:38:26 -05:00
Nicolas Chaulet
ce9f6222d8
[Fleet] remove deprecated settings API endpoints (#198799) 2024-11-06 08:01:46 -05:00
Sid
b12e7d0e79
[Authz] Fix description generation for Open API spec for an API (#198054)
Closes https://github.com/elastic/kibana/issues/198058. 

Adds a fix for https://github.com/elastic/kibana/pull/197001

## Summary
There was an error in how descriptions were added to the Open API spec
for a given route - for the specific case when both a route description
and security authz required privileges were present. The code with the
error is:
https://github.com/elastic/kibana/pull/197001/files#diff-5942307fac5a7b321e7f317bacd2837a7f766f3e79d5aad285513b1f82951b46R79-R80

This PR fixes that error. 


Also updated: Description field for required privileges now includes a
more intuitive descriptor: `Required authorization` as well as a line
break.

<img width="838" alt="image"
src="https://github.com/user-attachments/assets/e6af0459-28e8-40e5-873d-924d1a49b01b">

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-11-04 09:57:45 -06:00
Nicolas Chaulet
115dbec0ff
[Fleet] Remove deprecated APIs for agents endpoints (#198313) 2024-11-01 08:00:12 -04:00
Milosz Marcinkowski
e0838147bd
OpenAPI docs for APM UI APIs (#197946)
## Summary

Closes #197582

This PR adds missing OpenAPI specification for APM UI APIs.

Documentation specs have been divided into files per path. See
[readme](88e55f92f9/x-pack/plugins/observability_solution/apm/docs/openapi/apm/README.md)
for specific information.
2024-10-31 14:59:19 +01:00
Tiago Vila Verde
4538481be0
[Entity Analytics] [Entity Store] Show errors on entity store enablement (#198263)
## Summary

This PR adds user feedback for errors that happen when enabling the
entity store.
Any errors during the async setup of store resources will show up as
toasts, whist initial INIT request failures will appear as an error
callout.

![Screenshot 2024-10-29 at 16 48
03](https://github.com/user-attachments/assets/12aa9af3-1e27-44b1-85e5-5053255bd333)
![Screenshot 2024-10-29 at 16 47
19](https://github.com/user-attachments/assets/31790981-599b-4fba-a423-b75e31dbe7be)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-10-30 22:44:43 -05:00
Jean-Louis Leysens
e53d68c26d
[OAS] Beautify generated operation ids (#198132) 2024-10-30 20:11:24 +01:00
Ying Mao
dd90b67a87
[Response Ops][Actions] Remove deprecated HTTP APIs (#197510)
Resolves https://github.com/elastic/kibana/issues/90382

## Summary

Removes legacy action APIs for 9.0 and updates all tests that still used
the legacy APIs to use the current APIs. Also did some renaming of
action -> connector in the files I had to touch.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-10-29 15:20:12 -04:00
Ersin Erdal
babb0feb8e
Consolidate Rule schemas (#195613)
Towards: #172513

This PR removes `RawRule` type from `alerting/server/type` and
`RuleAttributes` schema/types.
And uses the `RawRule` that is used for ModelVersions instead of them.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-10-28 17:07:35 +01:00
Philippe Oberti
1065bbf03c
[Security Solution][Notes] - fix createdBy filter for notes management page (#197706) 2024-10-25 15:37:23 -05:00
Lisa Cawley
02fab7248d
[OpenAPI][DOCS] Add descriptions, examples, responses for role APIs (#195527)
Co-authored-by: Elena Shostak <165678770+elena-shostak@users.noreply.github.com>
2024-10-25 10:13:57 -07:00
Lisa Cawley
106eb7b114
[OpenAPI] Add work in progress statement to API intro (#197703) 2024-10-24 14:38:33 -07:00
Jean-Louis Leysens
77ce0fcbcf
[OAS] Removing staging files (#197455)
## Summary

Removing staging files for now. We can always revert this if it turns
out we need them.

## Reviewers

* I might be missing something in removing these, but right now I think
they are making commit noise and not really serving a purpose
* I moved the `final_merge` step to after the OAS snapshot capture
2024-10-24 09:52:13 -05:00
Pablo Machado
0bafb9632c
[SecuritySolution] Update entity store source field (#197186)
## Summary

In this PR the source field will only store the first identified index
for an entity.
The PR also updates the entities list panel to display a textual
description of the source index and adds a new source field filter.



https://github.com/user-attachments/assets/c7aad254-f871-4035-9dac-89decce31a55



### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Mark Hopkin <mark.hopkin@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-10-23 10:05:05 +02:00
Ahmad Bamieh
c417196905
[Core] [UA] Support API Deprecations (#196081)
# Summary

Adds a new API deprecations feature inside core.
This feature enabled plugin developers to mark their versioned and
unversioned public routes as deprecated.
These deprecations will be surfaced to the users through UA to help them
understand the deprecation and address it before upgrading. This PR also
surfaces these deprecations to UA.

Closes https://github.com/elastic/kibana/issues/117241

1. Core service to flag deprecated routes
2. UA code to surface and resolve deprecated routes

## Flagging a deprecated Route

### The route deprecation option
We have three types of route deprecations:

- `type: bump`: A version bump deprecation means the API has a new
version and the current version will be removed in the future in favor
of the newer version.
- `type: remove`: This API will be completely removed. You will no
longer be able to use it in the future.
- `type: migrate`: This API will be migrated to a different API and will
be removed in the future in favor of the other API.


All route deprecations expect a documentation link to help users
navigate. We might add a generic documentation link and drop this
requirement in the future but for now this is required.

### Deprecated Route Example
Full examples can be found in the `routing_example` example plugin
located in this directory:
`examples/routing_example/server/routes/deprecated_routes`

```ts
router[versioned?].get(
    {
      path: '/',
      options: {
        deprecated: {
           documentationUrl: 'https://google.com',
           severity: 'warning',
           reason: {
              type: 'bump',
              newApiVersion: '2024-10-13',
            },
        },
      },
    },
    async (context, req, res) => {
...
```

## Surfaced API deprecations in UA

The list of deprecated APIs will be listed inside Kibana deprecations
along with the already supported config deprecations.
<img width="1728" alt="image"
src="https://github.com/user-attachments/assets/5bece704-b80b-4397-8ba2-6235f8995e4a">


Users can click on the list item to learn more about each deprecation
and mark it as resolved
<img width="1476" alt="image"
src="https://github.com/user-attachments/assets/91c9207b-b246-482d-a5e4-21d0c61582a8">



### Marking as resolved
Users can click on mark as resolved button in the UA to hide the
deprecation from the Kiban deprecations list.
We keep track on when this button was clicked and how many times the API
has been called. If the API is called again the deprecation will
re-appear inside the list. We might add a feature in the future to
permenantly supress the API deprecation from showing in the list through
a configuration (https://github.com/elastic/kibana/issues/196089)

If the API has been marked as resolved before we show this in the flyout
message:
> The API GET /api/deprecations/ has been called 25 times. The last time
the API was called was on Monday, October 14, 2024 1:08 PM +03:00.
> The api has been called 2 times since the last time it was marked as
resolved on Monday, October 14, 2024 1:08 PM +03:00


Once marked as resolved the flyout exists and we show this to the user
until they refresh the page
<img width="1453" alt="image"
src="https://github.com/user-attachments/assets/8bb5bc8b-d1a3-478f-9489-23cfa7db6350">


## Telemetry:
We keep track of 2 new things for telemetry purposes:
1. The number of times the deprecated API has been called
2. The number of times the deprecated API has been resolved (how many
times the mark as resolved button in UA was clicked)

## Code review
- [x] Core team is expected to review the whole PR
- [ ] Docs team to review the copy and update the UA displayed texts
(title, description, and manual steps)
- [x] kibana-management team is expected to review the UA code changes
and UI
- [ ] A few teams are only required to approve this PR and update their
`deprecated: true` route param to the new deprecationInfo object we now
expect. There is an issue tracker to address those in separate PRs later
on: https://github.com/elastic/kibana/issues/196095

## Testing

Run kibana locally with the test example plugin that has deprecated
routes
```
yarn start --plugin-path=examples/routing_example --plugin-path=examples/developer_examples
```

The following comprehensive deprecated routes examples are registered
inside the folder:
`examples/routing_example/server/routes/deprecated_routes`

Run them in the console to trigger the deprecation condition so they
show up in the UA:

```
# Versioned routes: Version 1 is deprecated
GET kbn:/api/routing_example/d/versioned?apiVersion=1
GET kbn:/api/routing_example/d/versioned?apiVersion=2

# Non-versioned routes
GET kbn:/api/routing_example/d/removed_route
POST kbn:/api/routing_example/d/migrated_route
{}
```

1. You can also mark as deprecated in the UA to remove the deprecation
from the list.
2. Check the telemetry response to see the reported data about the
deprecated route.
3. Calling version 2 of the API does not do anything since it is not
deprecated unlike version `1` (`GET
kbn:/api/routing_example/d/versioned?apiVersion=2`)
4. Internally you can see the deprecations counters from the dev console
by running the following:
```
GET .kibana_usage_counters/_search
{
    "query": {
        "bool": {
            "should": [
              {"match": { "usage-counter.counterType": "deprecated_api_call:total"}},
              {"match": { "usage-counter.counterType": "deprecated_api_call:resolved"}},
              {"match": { "usage-counter.counterType": "deprecated_api_call:marked_as_resolved"}}
            ]
        }
    }
}

```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: florent-leborgne <florent.leborgne@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-10-22 19:57:37 +03:00
Cristina Amico
3be33bd3e9
[Fleet] Display outputs in agent list table and agent details (#195801)
Closes https://github.com/elastic/kibana/issues/192339

## Summary

Display two additional columns with Outputs hosts in agent list table
and agent details section
- The two columns show monitoring output and the integrations output and
link to the output flyout in settings
- Display a badge that show the outputs set per integration introduced
by https://github.com/elastic/kibana/pull/189125
- Same info displayed in agent details as well

To achieve this, I added two new endpoints.

1. Endpoint that fetches all the outputs associated with a single agent
policy (outputs defined on agent policy or default defined in global
settings and if any, outputs per integration)
```
GET kbn:/api/fleet/agent_policies/<AGENT_POLICY_ID>/outputs
```

2. Endpoint that fetches the outputs as above, for a defined set of
agent policy ids
```
POST kbn:/api/fleet/agent_policies/outputs
{
  "ids": ["policy_id1", "policy_id2", ...]
}
```
The reason to pass an array of ids is to ensure that we fetch the info
only for the policies displayed in the table at any given moment.


### Screenshots
**Agent list**
![Screenshot 2024-10-16 at 17 51
57](https://github.com/user-attachments/assets/3ee08df1-9562-497f-9621-4a913b3dad74)
![Screenshot 2024-10-16 at 17 52
05](https://github.com/user-attachments/assets/72b9da7d-872a-45f8-b02d-29184ffb2179)

**Agent details**
![Screenshot 2024-10-16 at 17 52
20](https://github.com/user-attachments/assets/b99aaf9e-14f1-44b8-9776-3e0136775af8)


### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers
- [ ] This will appear in the **Release Notes** and follow the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-10-22 14:46:24 +02:00
Nicolas Chaulet
ce7616ad59
[Fleet] Limit pagination size when retrieving full policy or withAgentCount (#196887) 2024-10-21 14:44:32 -04:00
Jeramy Soucy
c73bfd2480
Enable custom roles and spaces in serverless projects (#195584)
Closes #194933
Closes #192282

## Summary

This PR updates the serverless project yml files to

- enable custom roles for Elasticsearch and Security projects
- enable multiple spaces (max 100) for all serverless project types

### Tests
Additionally, this PR adjust the serverless test suites. Originally,
testing of roles and spaces endpoints was achieved from the feature flag
test config. Now that these features are enabled by default, the tests
have been migrated to the standard serverless test configs.

Affected tests:
-
x-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts
-
x-pack/test_serverless/functional/test_suites/common/platform_security/navigation/management_nav_cards.ts
-
x-pack/test_serverless/functional/test_suites/common/platform_security/roles.ts
-
x-pack/test_serverless/functional/test_suites/common/spaces/spaces_management.ts
-
x-pack/test_serverless/functional/test_suites/common/spaces/spaces_selection.ts
- Feature flag configs/indices
- Project specific configs/indices
- Base serverless config

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
2024-10-21 11:25:35 -05:00
Lisa Cawley
2656e5a4ff
[DOCS][OpenAPI] Add descriptions, examples, tags, responses for space APIs (#195333)
Co-authored-by: Elena Shostak <165678770+elena-shostak@users.noreply.github.com>
2024-10-18 14:03:06 -05:00
Philippe Oberti
bf7c5aaf29
[Security Solution][Notes] - fix an issue that breaks the notes management page, an enum value was missing from the api (#196912)
## Summary

_**Notes: please ignore the spaces introduced in the first commit,
Webstorm is trying to be smart... CI should take care of fixing that in
a second commit shortly**_

This PR fixes a small issue that prevents the notes management page to
load. The issue comes from the fact that one of the enum values passed
to filter notes was missing from the open api spec. This issue was
introduced in this recent
[PR](https://github.com/elastic/kibana/pull/195501).

Current notes management borken page
![Screenshot 2024-10-18 at 10 30
13 AM](https://github.com/user-attachments/assets/4926a62f-1ebf-4698-8a13-bf761d77f4ba)

This is the error in the network tab
![Screenshot 2024-10-18 at 10 30
29 AM](https://github.com/user-attachments/assets/90b56246-c116-4050-bcfa-2c6668274e74)

This PR fixes the issue
![Screenshot 2024-10-18 at 10 27
01 AM](https://github.com/user-attachments/assets/7d3338ce-ad73-4be5-b94c-15bcf0234680)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-10-18 13:09:02 -05:00
Lisa Cawley
7b106f7235
[DOCS][OpenAPI] Change x-technical-preview to x-state in case APIs (#195325) 2024-10-17 16:14:34 -07:00
Philippe Oberti
66708b26c5
[Security Solution][Notes] - allow filtering by note association (#195501) 2024-10-16 04:32:51 -05:00
Philippe Oberti
d85b51db22
[Security Solution][Notes] - allow filtering by user (#195519) 2024-10-16 04:42:23 +02:00