github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-06-27 18:51:07 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Security Solution] OpenAPI docs bundler (#171526)

Browse source 
**Addresses:** https://github.com/elastic/security-team/issues/7981

## Summary

This PR adds an OpenAPI spec bundler to simplify integration with the
Docs Engineering team. The bundler produces a single bundled file by
dereferencing and inlining some of external references and bundling them
and paths into a single file.

## Details

Currently we maintain a number of schema files inside
`x-pack/plugins/security_solution/common/api/**.schema.yaml` and it
might be hard for external teams to keep track of all the changes in our
schemas. By creating a singular schema file, we provide a clear
integration point for others.

The bundler addresses the following issues

- hide endpoints that we don't want to expose (Endpoints related to
features hidden under a feature flag and all internal endpoints should
be excluded from the file)
- hide not finished data structures related to features hidden under a
feature flag or data structures that are not designed to be public (For
example `RuleActionAlertsFilter` or `RuleActionParams` are exposed
directly from the Alerting framework and might be considered
implementation details, we don't want to document interfaces that are
not designed to be public so hiding them is a good option)
- modify spec based on presence of `x-modify` property (Instead of
exposing `x-modify: partial` we need to make the exported data structure
partial and instead of exposing `x-modify: required` we need to make the
exported data structure required)
- remove any internal attributes used for code generation like
`x-codegen-enabled` and `x-modify`
- inline some of the reused data structures (We have a lot of low-level
reusable data structures `in common_attributes.schema.yaml` which might
make the final documentation hardly usable from the UX perspective, so
we can inline them)

and lives in a new `@kbn/openapi-bundler` package under
`packages/kbn-openapi-bundler` folder.

### Related changes

- Implicit version type `version: 2023-10-31` has been changed to
explicit string type `version: '2023-10-31'` for all specs under
`security_solution/common/api` folder. Implicit type causes `js-yaml`
parsing it as a `Data` JS object leading to serializing it like
`2023-10-31T00:00:00.000Z`.
- `ListRequestQuery` schema in
`security_solution/common/api/endpoint/actions/list.schema.yaml ` has
been renamed to `EndpointActionListRequestQuery` to avoid conflicts with
`ListRequestQuery` in
`security_solution/common/api/endpoint/metadata/list_metadata.schema.yaml`.
While it's not an issue to have completely different schemas sharing the
same name in different files it may be an indication of pitfalls in the
API design. I'd say it's an open question if such cases need to be
always resolved automatically or reviewed manually. At this moment the
bundler can't resolve such conflicts.

## How to test?

There is a a new JS script added to Security Solution plugin located at
`x-pack/plugins/security_solution/scripts/openapi/bundle.js` with a
corresponding entry in `package.json` named `openapi:bundle`.

To test the PR change directory to Security Solution plugin's root
folder and run the bundler like below

```sh
cd x-pack/plugins/security_solution
yarn openapi:bundle
```

It should produce a bundled OpenAPI spec at
`x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml`.

## Open issues

- [x] Circular references (implemented in
dfdf0a51ea)
- [x] Mix of OpenAPI `3.0` and `3.1` specs (Maybe convert automatically
to `3.1`?). Folder like OpenAPI bundling format implemented
[here](0ae7ad5abb)
allows to mix OpenAPI `3.0` and `3.1` specs.

## Improvements

- [ ] Flexible configuration
- [ ] CLI support?

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Maxim Palenov 2023-11-29 14:07:31 +01:00 • committed by GitHub
parent 27f9a0069c
commit e3d95e9b72
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
109 changed files with 3590 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.github/CODEOWNERS vendored
View file

@ -559,6 +559,7 @@ x-pack/plugins/observability @elastic/obs-ux-management-team
x-pack/plugins/observability_shared @elastic/observability-ui x-pack/plugins/observability_shared @elastic/observability-ui
x-pack/test/security_api_integration/plugins/oidc_provider @elastic/kibana-security x-pack/test/security_api_integration/plugins/oidc_provider @elastic/kibana-security
test/common/plugins/otel_metrics @elastic/obs-ux-infra_services-team test/common/plugins/otel_metrics @elastic/obs-ux-infra_services-team
packages/kbn-openapi-bundler @elastic/security-detection-rule-management
packages/kbn-openapi-generator @elastic/security-detection-rule-management packages/kbn-openapi-generator @elastic/security-detection-rule-management
packages/kbn-optimizer @elastic/kibana-operations packages/kbn-optimizer @elastic/kibana-operations
packages/kbn-optimizer-webpack-helpers @elastic/kibana-operations packages/kbn-optimizer-webpack-helpers @elastic/kibana-operations

1
package.json
View file

@ -1244,6 +1244,7 @@
"@kbn/managed-vscode-config-cli": "link:packages/kbn-managed-vscode-config-cli", "@kbn/managed-vscode-config-cli": "link:packages/kbn-managed-vscode-config-cli",
"@kbn/management-storybook-config": "link:packages/kbn-management/storybook/config", "@kbn/management-storybook-config": "link:packages/kbn-management/storybook/config",
"@kbn/mock-idp-plugin": "link:packages/kbn-mock-idp-plugin", "@kbn/mock-idp-plugin": "link:packages/kbn-mock-idp-plugin",
"@kbn/openapi-bundler": "link:packages/kbn-openapi-bundler",
"@kbn/openapi-generator": "link:packages/kbn-openapi-generator", "@kbn/openapi-generator": "link:packages/kbn-openapi-generator",
"@kbn/optimizer": "link:packages/kbn-optimizer", "@kbn/optimizer": "link:packages/kbn-optimizer",
"@kbn/optimizer-webpack-helpers": "link:packages/kbn-optimizer-webpack-helpers", "@kbn/optimizer-webpack-helpers": "link:packages/kbn-optimizer-webpack-helpers",

493
packages/kbn-openapi-bundler/README.md Normal file
View file

@ -0,0 +1,493 @@
# OpenAPI Specs Bundler for Kibana
`@kbn/openapi-bundler` is a tool for transforming multiple OpenAPI specification files (source specs) into a single bundled specification file (target spec).
This can be used for API docs generation purposes. This approach allows you to:
- Abstract away the knowledge of where you keep your OpenAPI specs, how many specs there are, and how to find them. The Docs team should only know where a single file is located - the bundle.
- Omit internal API endpoints from the bundle.
- Omit API endpoints that are hidden behind a feature flag and haven't been released yet.
- Omit parts of schemas that are hidden behind a feature flag (e.g. a new property added to an existing response schema).
- Omit custom OpenAPI attributes from the bundle, such as `x-codegen-enabled`, `x-internal`, and `x-modify` (see below).
- Transform the target schema according to the custom OpenAPI attributes, such as `x-modify`.
- Resolve references and inline some of them for better readability. The bundled file contains only local references and paths.
## Getting started
To let this package help you with bundling your OpenAPI specifications you should have OpenAPI specification describing your API endpoint request and response schemas along with common types used in your API. Refer [@kbn/openapi-generator](../kbn-openapi-generator/README.md) and [OpenAPI 3.0.3](https://swagger.io/specification/v3/) (support for [OpenAPI 3.1.0](https://swagger.io/specification/) is planned to be added soon) for more details.
Following recommendations provided in `@kbn/openapi-generator` you should have OpenAPI specs defined under a common folder something like `my-plugin/common/api`.
Currently package supports only programmatic API. As the next step you need to create a JavaScript script file like below and put it to `my-plugin/scripts/openapi`
```ts
require('../../../../../src/setup_node_env');
const { bundle } = require('@kbn/openapi-bundler');
const { resolve } = require('path');
// define ROOT as `my-plugin` instead of `my-plugin/scripts/openapi`
// pay attention to this constant when your script's location is different
const ROOT = resolve(__dirname, '../..');
bundle({
rootDir: ROOT, // Root path e.g. plugin root directory
sourceGlob: './**/*.schema.yaml', // Glob pattern to find OpenAPI specification files
outputFilePath: './target/openapi/my-plugin.bundled.schema.yaml', //
});
```
And add a script entry to your `package.json` file
```json
{
"author": "Elastic",
...
"scripts": {
...
"openapi:bundle": "node scripts/openapi/bundle"
}
}
```
Finally you should be able to run OpenAPI bundler via
```bash
yarn openapi:bundle
```
This command will produce a bundled file `my-plugin/target/openapi/my-plugin.bundled.schema.yaml` containing
all specs matching `./**/*.schema.yaml` glob pattern.
Here's an example how your source schemas can look like and the expected result
- `example1.schema.yaml`
```yaml
openapi: 3.0.3
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
get:
operationId: MyGetEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
- `example2.schema.yaml`
```yaml
openapi: 3.0.3
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
post:
x-internal: true
operationId: MyPostEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
And the target spec will look like
```yaml
openapi: 3.0.3
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
get:
operationId: MyGetEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
post:
operationId: MyPostEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
components:
schemas: {}
```
## Supported custom (`x-` prefixed) properties
OpenAPI specification allows to define custom properties. They can be used to describe extra functionality that is not covered by the standard OpenAPI Specification. We currently support the following custom properties
- [x-internal](#x-internal) - marks source spec nodes the bundler must NOT include in the target spec
- [x-modify](#x-modify) - marks nodes to be modified by the bundler
- [x-inline](#x-inline) - marks reference nodes to be inlined when bundled
### `x-internal`
Marks source spec nodes the bundler must NOT include in the target spec.
**Supported values**: `true`
When bundler encounters a node with `x-internal: true` it doesn't include this node into the target spec. It's useful when it's necessary to hide some chunk of OpenAPI spec because functionality supporting it is hidden under a feature flag or the chunk is just for internal use.
#### Examples
The following spec defines an API endpoint `/api/path/to/endpoint` accepting `GET` and `POST` requests. It has `x-internal: true` defined in `post` section meaning it won't be included in the target spec.
```yaml
openapi: 3.0.3
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
get:
operationId: MyGetEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
post:
x-internal: true
operationId: MyPostEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
The target spec will look like
```yaml
openapi: 3.0.3
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
get:
operationId: MyGetEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
`x-internal: true` can also be defined next to a reference.
```yaml
openapi: 3.0.3
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
get:
operationId: MyGetEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
post:
$ref: '#/components/schemas/MyPostEndpointResponse'
x-internal: true
components:
schemas:
MyPostEndpointResponse:
operationId: MyPostEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
The target spec will look like
```yaml
openapi: 3.0.3
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
get:
operationId: MyGetEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
### `x-modify`
Marks nodes to be modified by the bundler.
**Supported values**: `partial` or `required`
Value `partial` leads to removing `required` property making params under `properties` optional. Value `required` leads to adding or extending `required` property by adding all param names under `properties`.
#### Examples
The following spec has `x-modify: partial` at `schema` section. It makes params optional for a PATCH request.
```yaml
openapi: 3.0.0
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
patch:
operationId: MyPatchEndpoint
requestBody:
required: true
content:
application/json:
schema:
x-modify: partial
type: object
properties:
param1:
type: string
enum: [val1, val2, val3]
param2:
type: number
required:
- param1
- param2
```
The target spec will look like
```yaml
openapi: 3.0.0
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
patch:
operationId: MyPatchEndpoint
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
param1:
type: string
enum: [val1, val2, val3]
param2:
type: number
```
The following spec has `x-modify: required` at `schema` section. It makes params optional for a PATCH request.
```yaml
openapi: 3.0.0
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
put:
operationId: MyPutEndpoint
requestBody:
required: true
content:
application/json:
schema:
x-modify: required
type: object
properties:
param1:
type: string
enum: [val1, val2, val3]
param2:
type: number
```
The target spec will look like
```yaml
openapi: 3.0.0
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
patch:
operationId: MyPatchEndpoint
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
param1:
type: string
enum: [val1, val2, val3]
param2:
type: number
required:
- param1
- param2
```
`x-modify` can also be defined next to a reference.
```yaml
openapi: 3.0.0
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
patch:
operationId: MyPatchEndpoint
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/PatchProps'
x-modify: partial
components:
schemas:
PatchProps:
type: object
properties:
param1:
type: string
enum: [val1, val2, val3]
param2:
type: number
required:
- param1
- param2
```
The target spec will look like
```yaml
openapi: 3.0.0
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
patch:
operationId: MyPatchEndpoint
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
param1:
type: string
enum: [val1, val2, val3]
param2:
type: number
```
### `x-inline`
Marks reference nodes to be inlined when bundled.
**Supported values**: `true`
`x-inline: true` can be specified at a reference node itself (a node with `$ref` key) or at a node `$ref` resolves to. When bundler encounters such a node it assigns (copies keys via `Object.assign()`) the latter node (a node`$ref` resolves to) to the first node (a node with `$ref` key). This way target won't have referenced component in `components` as well.
#### Examples
The following spec defines an API endpoint `/api/path/to/endpoint` accepting `POST` request. It has `x-inline: true` specified in `post` section meaning reference `#/components/schemas/MyPostEndpointResponse` will be inlined in the target spec.
```yaml
openapi: 3.0.3
info:
title: My endpoint
version: '2023-10-31'
paths:
/api/path/to/endpoint:
post:
$ref: '#/components/schemas/MyPostEndpointResponse'
x-inline: true
components:
schemas:
MyPostEndpointResponse:
operationId: MyPostEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```
The target spec will look like
```yaml
openapi: 3.0.3
info:
title: Bundled specs file. See individual paths.verb.tags for details
version: not applicable
paths:
/api/path/to/endpoint:
post:
operationId: MyPostEndpoint
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
```

9
packages/kbn-openapi-bundler/index.ts Normal file
View file

@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export * from './src/openapi_bundler';

13
packages/kbn-openapi-bundler/jest.config.js Normal file
View file

@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
module.exports = {
preset: '@kbn/test',
rootDir: '../..',
roots: ['<rootDir>/packages/kbn-openapi-bundler'],
};

6
packages/kbn-openapi-bundler/kibana.jsonc Normal file
View file

@ -0,0 +1,6 @@
{
"devOnly": true,
"id": "@kbn/openapi-bundler",
"owner": "@elastic/security-detection-rule-management",
"type": "shared-common"
}

7
packages/kbn-openapi-bundler/package.json Normal file
View file

@ -0,0 +1,7 @@
{
"description": "OpenAPI specs bundler for Kibana",
"license": "SSPL-1.0 OR Elastic License 2.0",
"name": "@kbn/openapi-bundler",
"private": true,
"version": "1.0.0"
}

40
packages/kbn-openapi-bundler/src/__test__/conflicting_but_equal_refs_in_different_specs/expected.yaml Normal file
View file

@ -0,0 +1,40 @@
spec1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/ConflictTestSchema'
spec2.schema.yaml:
openapi: 3.0.3
info:
title: Another test endpoint
version: '2023-10-31'
paths:
/api/another_api:
put:
operationId: AnotherTestEndpointPut
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/ConflictTestSchema'
shared_components.schema.yaml:
components:
schemas:
ConflictTestSchema:
type: integer
minimum: 1

21
packages/kbn-openapi-bundler/src/__test__/conflicting_but_equal_refs_in_different_specs/spec1.schema.yaml Normal file
View file

@ -0,0 +1,21 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ConflictTestSchema'
components:
schemas:
ConflictTestSchema:
type: integer
minimum: 1

21
packages/kbn-openapi-bundler/src/__test__/conflicting_but_equal_refs_in_different_specs/spec2.schema.yaml Normal file
View file

@ -0,0 +1,21 @@
openapi: 3.0.3
info:
title: Another test endpoint
version: '2023-10-31'
paths:
/api/another_api:
put:
operationId: AnotherTestEndpointPut
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ConflictTestSchema'
components:
schemas:
ConflictTestSchema:
type: integer
minimum: 1

23
packages/kbn-openapi-bundler/src/__test__/conflicting_refs_in_different_specs/spec1.schema.yaml Normal file
View file

@ -0,0 +1,23 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ConflictTestSchema'
components:
schemas:
ConflictTestSchema:
type: string
enum:
- value1
- value2

21
packages/kbn-openapi-bundler/src/__test__/conflicting_refs_in_different_specs/spec2.schema.yaml Normal file
View file

@ -0,0 +1,21 @@
openapi: 3.0.3
info:
title: Another test endpoint
version: '2023-10-31'
paths:
/api/another_api:
put:
operationId: AnotherTestEndpointPut
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ConflictTestSchema'
components:
schemas:
ConflictTestSchema:
type: integer
minimum: 1

44
packages/kbn-openapi-bundler/src/__test__/different_endpoint_versions/expected.yaml Normal file
View file

@ -0,0 +1,44 @@
version1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer
version2.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-11-11'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer
field2:
type: string
shared_components.schema.yaml:
components: {}

18
packages/kbn-openapi-bundler/src/__test__/different_endpoint_versions/version1.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer

20
packages/kbn-openapi-bundler/src/__test__/different_endpoint_versions/version2.schema.yaml Normal file
View file

@ -0,0 +1,20 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-11-11'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer
field2:
type: string

42
packages/kbn-openapi-bundler/src/__test__/different_openapi_versions/expected.yaml Normal file
View file

@ -0,0 +1,42 @@
spec1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer
spec2.schema.yaml:
openapi: 3.1.0
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field2:
type: string
shared_components.schema.yaml:
components: {}

18
packages/kbn-openapi-bundler/src/__test__/different_openapi_versions/spec1.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer

18
packages/kbn-openapi-bundler/src/__test__/different_openapi_versions/spec2.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.1.0
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field2:
type: string

50
packages/kbn-openapi-bundler/src/__test__/inline_ref/expected.yaml Normal file
View file

@ -0,0 +1,50 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
anyOf:
- type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
- $ref: './shared_components.schema.yaml#/components/schemas/TestSchema2'
- $ref: './shared_components.schema.yaml#/components/schemas/TestSchema3'
shared_components.schema.yaml:
components:
schemas:
TestSchema2:
x-inline: false
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
TestSchema3:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1

52
packages/kbn-openapi-bundler/src/__test__/inline_ref/spec.schema.yaml Normal file
View file

@ -0,0 +1,52 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
anyOf:
- $ref: '#/components/schemas/TestSchema1'
- $ref: '#/components/schemas/TestSchema2'
- $ref: '#/components/schemas/TestSchema3'
components:
schemas:
TestSchema1:
x-inline: true
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
TestSchema2:
x-inline: false
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
TestSchema3:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1

26
packages/kbn-openapi-bundler/src/__test__/modify_partial_node/expected.yaml Normal file
View file

@ -0,0 +1,26 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
shared_components.schema.yaml:
components: {}

26
packages/kbn-openapi-bundler/src/__test__/modify_partial_node/spec.schema.yaml Normal file
View file

@ -0,0 +1,26 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
x-modify: partial
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
required:
- field1
- field2

26
packages/kbn-openapi-bundler/src/__test__/modify_partial_ref/expected.yaml Normal file
View file

@ -0,0 +1,26 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
shared_components.schema.yaml:
components: {}

31
packages/kbn-openapi-bundler/src/__test__/modify_partial_ref/spec.schema.yaml Normal file
View file

@ -0,0 +1,31 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/TestSchema'
x-modify: partial
components:
schemas:
TestSchema:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
required:
- field1
- field2

29
packages/kbn-openapi-bundler/src/__test__/modify_required_node/expected.yaml Normal file
View file

@ -0,0 +1,29 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
required:
- field1
- field2
shared_components.schema.yaml:
components: {}

23
packages/kbn-openapi-bundler/src/__test__/modify_required_node/spec.schema.yaml Normal file
View file

@ -0,0 +1,23 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
x-modify: required
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1

29
packages/kbn-openapi-bundler/src/__test__/modify_required_ref/expected.yaml Normal file
View file

@ -0,0 +1,29 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
required:
- field1
- field2
shared_components.schema.yaml:
components: {}

28
packages/kbn-openapi-bundler/src/__test__/modify_required_ref/spec.schema.yaml Normal file
View file

@ -0,0 +1,28 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/TestSchema'
x-modify: required
components:
schemas:
TestSchema:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1

21
packages/kbn-openapi-bundler/src/__test__/recursive_ref_specs/common.schema.yaml Normal file
View file

@ -0,0 +1,21 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths: {}
components:
schemas:
CircularTestSchema:
type: string
data:
items:
$ref: '#/components/schemas/AnotherCircularTestSchema'
AnotherCircularTestSchema:
anyof:
- $ref: '#/components/schemas/CircularTestSchema'
- type: string
enum:
- value1
- value2

50
packages/kbn-openapi-bundler/src/__test__/recursive_ref_specs/expected.yaml Normal file
View file

@ -0,0 +1,50 @@
spec1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/CircularTestSchema'
spec2.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/CircularTestSchema'
shared_components.schema.yaml:
components:
schemas:
CircularTestSchema:
type: string
data:
items:
$ref: '#/components/schemas/AnotherCircularTestSchema'
AnotherCircularTestSchema:
anyof:
- $ref: '#/components/schemas/CircularTestSchema'
- type: string
enum:
- value1
- value2

15
packages/kbn-openapi-bundler/src/__test__/recursive_ref_specs/spec1.schema.yaml Normal file
View file

@ -0,0 +1,15 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './common.schema.yaml#/components/schemas/CircularTestSchema'

15
packages/kbn-openapi-bundler/src/__test__/recursive_ref_specs/spec2.schema.yaml Normal file
View file

@ -0,0 +1,15 @@
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './common.schema.yaml#/components/schemas/CircularTestSchema'

27
packages/kbn-openapi-bundler/src/__test__/recursive_spec/expected.yaml Normal file
View file

@ -0,0 +1,27 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema: &ref0
type: object
properties:
- name: field1
required: false
schema: *ref0
- field2:
required: false
schema:
type: string
shared_components.schema.yaml:
components: {}

23
packages/kbn-openapi-bundler/src/__test__/recursive_spec/spec.schema.yaml Normal file
View file

@ -0,0 +1,23 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema: &ref0
type: object
properties:
- name: field1
required: false
schema: *ref0
- field2:
required: false
schema:
type: string

24
packages/kbn-openapi-bundler/src/__test__/self_recursive_ref/expected.yaml Normal file
View file

@ -0,0 +1,24 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/CircularTestSchema'
shared_components.schema.yaml:
components:
schemas:
CircularTestSchema:
type: string
data:
$ref: '#/components/schemas/CircularTestSchema'

22
packages/kbn-openapi-bundler/src/__test__/self_recursive_ref/spec.schema.yaml Normal file
View file

@ -0,0 +1,22 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/CircularTestSchema'
components:
schemas:
CircularTestSchema:
type: string
data:
$ref: '#/components/schemas/CircularTestSchema'

31
packages/kbn-openapi-bundler/src/__test__/skip_internal/expected.yaml Normal file
View file

@ -0,0 +1,31 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
anyOf:
- $ref: './shared_components.schema.yaml#/components/schemas/TestSchema1'
- type: object
shared_components.schema.yaml:
components:
schemas:
TestSchema1:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1

57
packages/kbn-openapi-bundler/src/__test__/skip_internal/spec.schema.yaml Normal file
View file

@ -0,0 +1,57 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
anyOf:
- $ref: '#/components/schemas/TestSchema1'
- $ref: '#/components/schemas/TestSchema2'
x-internal: true
- type: object
properties:
x-internal: true
field1:
$ref: '#/components/schemas/TestSchema3'
components:
schemas:
TestSchema1:
# x-internal is not supported here
# x-internal: true
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
TestSchema2:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1
TestSchema3:
type: object
properties:
field1:
type: string
enum: [value1]
field2:
type: integer
minimum: 1

22
packages/kbn-openapi-bundler/src/__test__/skip_internal_endpoint/expected.yaml Normal file
View file

@ -0,0 +1,22 @@
spec1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer
shared_components.schema.yaml:
components: {}

18
packages/kbn-openapi-bundler/src/__test__/skip_internal_endpoint/spec1.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer

18
packages/kbn-openapi-bundler/src/__test__/skip_internal_endpoint/spec2.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/internal/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field2:
type: string

13
packages/kbn-openapi-bundler/src/__test__/spec_with_external_ref/common.schema.yaml Normal file
View file

@ -0,0 +1,13 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths: {}
components:
schemas:
TestSchema:
type: string
enum:
- value1
- value2

25
packages/kbn-openapi-bundler/src/__test__/spec_with_external_ref/expected.yaml Normal file
View file

@ -0,0 +1,25 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/TestSchema'
shared_components.schema.yaml:
components:
schemas:
TestSchema:
type: string
enum:
- value1
- value2

15
packages/kbn-openapi-bundler/src/__test__/spec_with_external_ref/spec.schema.yaml Normal file
View file

@ -0,0 +1,15 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './common.schema.yaml#/components/schemas/TestSchema'

25
packages/kbn-openapi-bundler/src/__test__/spec_with_local_ref/expected.yaml Normal file
View file

@ -0,0 +1,25 @@
spec.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/TestSchema'
shared_components.schema.yaml:
components:
schemas:
TestSchema:
type: string
enum:
- value1
- value2

23
packages/kbn-openapi-bundler/src/__test__/spec_with_local_ref/spec.schema.yaml Normal file
View file

@ -0,0 +1,23 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/TestSchema'
components:
schemas:
TestSchema:
type: string
enum:
- value1
- value2

42
packages/kbn-openapi-bundler/src/__test__/two_simple_specs/expected.yaml Normal file
View file

@ -0,0 +1,42 @@
spec1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer
spec2.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field2:
type: string
shared_components.schema.yaml:
components: {}

18
packages/kbn-openapi-bundler/src/__test__/two_simple_specs/spec1.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field1:
type: integer

18
packages/kbn-openapi-bundler/src/__test__/two_simple_specs/spec2.schema.yaml Normal file
View file

@ -0,0 +1,18 @@
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
field2:
type: string

13
packages/kbn-openapi-bundler/src/__test__/two_specs_with_external_ref/common.schema.yaml Normal file
View file

@ -0,0 +1,13 @@
openapi: 3.0.3
info:
title: Test endpoint
version: '2023-10-31'
paths: {}
components:
schemas:
TestSchema:
type: string
enum:
- value1
- value2

42
packages/kbn-openapi-bundler/src/__test__/two_specs_with_external_ref/expected.yaml Normal file
View file

@ -0,0 +1,42 @@
spec1.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/TestSchema'
spec2.schema.yaml:
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './shared_components.schema.yaml#/components/schemas/TestSchema'
shared_components.schema.yaml:
components:
schemas:
TestSchema:
type: string
enum:
- value1
- value2

15
packages/kbn-openapi-bundler/src/__test__/two_specs_with_external_ref/spec1.schema.yaml Normal file
View file

@ -0,0 +1,15 @@
openapi: 3.0.3
info:
title: Test endpoint GET
version: '2023-10-31'
paths:
/api/some_api:
get:
operationId: TestEndpointGet
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './common.schema.yaml#/components/schemas/TestSchema'

15
packages/kbn-openapi-bundler/src/__test__/two_specs_with_external_ref/spec2.schema.yaml Normal file
View file

@ -0,0 +1,15 @@
openapi: 3.0.3
info:
title: Test endpoint POST
version: '2023-10-31'
paths:
/api/some_api:
post:
operationId: TestEndpointPost
responses:
'200':
description: Successful response
content:
application/json:
schema:
$ref: './common.schema.yaml#/components/schemas/TestSchema'

12
packages/kbn-openapi-bundler/src/bundler/__mocks__/ref_resolver.ts Normal file
View file

@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export const RefResolver = jest.fn().mockImplementation(() => ({
resolveRef: jest.fn(),
resolveDocument: jest.fn(),
}));

105
packages/kbn-openapi-bundler/src/bundler/bundle_document.ts Normal file
View file

@ -0,0 +1,105 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { isAbsolute } from 'path';
import { RefResolver } from './ref_resolver';
import { processDocument } from './process_document';
import { BundleRefProcessor } from './document_processors/bundle_refs';
import { createSkipNodeWithInternalPropProcessor } from './document_processors/skip_node_with_internal_prop';
import { createModifyPartialProcessor } from './document_processors/modify_partial';
import { createSkipInternalPathProcessor } from './document_processors/skip_internal_path';
import { ResolvedDocument, ResolvedRef } from './types';
import { createRemovePropsProcessor } from './document_processors/remove_props';
import { createModifyRequiredProcessor } from './document_processors/modify_required';
import { X_CODEGEN_ENABLED, X_INLINE, X_INTERNAL, X_MODIFY } from './known_custom_props';
import { RemoveUnusedComponentsProcessor } from './document_processors/remove_unused_components';
import { isPlainObjectType } from '../utils/is_plain_object_type';
export class SkipException extends Error {
constructor(public documentPath: string, message: string) {
super(message);
}
}
export interface BundledDocument extends ResolvedDocument {
bundledRefs: ResolvedRef[];
}
/**
* Bundles document into one file and performs appropriate document modifications.
*
* Bundling assumes external references defined via `$ref` are included into the result document.
* Some of the references get inlined.
*
* Document modification includes the following
* - skips nodes with `x-internal: true` property
* - skips paths started with `/internal`
* - modifies nodes having `x-modify`
*
* @param absoluteDocumentPath document's absolute path
* @returns bundled document
*/
export async function bundleDocument(absoluteDocumentPath: string): Promise<BundledDocument> {
if (!isAbsolute(absoluteDocumentPath)) {
throw new Error(
`bundleDocument expects an absolute document path but got "${absoluteDocumentPath}"`
);
}
const refResolver = new RefResolver();
const resolvedDocument = await refResolver.resolveDocument(absoluteDocumentPath);
if (!hasPaths(resolvedDocument.document as MaybeObjectWithPaths)) {
// Specs without paths defined are usually considered as shared. Such specs have `components` defined
// and referenced by the specs with paths defined. In this case the shared specs have been
// handled already and must be skipped.
//
// An additional case when it's a rogue spec. Rogue specs are skipped as well as they don't contribute
// to the API endpoints.
throw new SkipException(resolvedDocument.absolutePath, 'Document has no paths defined');
}
const bundleRefsProcessor = new BundleRefProcessor(X_INLINE);
const removeUnusedComponentsProcessor = new RemoveUnusedComponentsProcessor();
await processDocument(resolvedDocument, refResolver, [
createSkipNodeWithInternalPropProcessor(X_INTERNAL),
createSkipInternalPathProcessor('/internal'),
createModifyPartialProcessor(),
createModifyRequiredProcessor(),
createRemovePropsProcessor([X_MODIFY, X_CODEGEN_ENABLED]),
bundleRefsProcessor,
removeUnusedComponentsProcessor,
]);
if (isPlainObjectType(resolvedDocument.document.components)) {
removeUnusedComponentsProcessor.removeUnusedComponents(resolvedDocument.document.components);
}
// If document.paths were removed by processors skip the document
if (!hasPaths(resolvedDocument.document as MaybeObjectWithPaths)) {
throw new SkipException(
resolvedDocument.absolutePath,
'Document has no paths after processing the document'
);
}
return { ...resolvedDocument, bundledRefs: bundleRefsProcessor.getBundledRefs() };
}
interface MaybeObjectWithPaths {
paths?: unknown;
}
function hasPaths(document: MaybeObjectWithPaths): boolean {
return (
typeof document.paths === 'object' &&
document.paths !== null &&
Object.keys(document.paths).length > 0
);
}

74
packages/kbn-openapi-bundler/src/bundler/document_processors/bundle_refs.ts Normal file
View file

@ -0,0 +1,74 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { Document, ResolvedRef, TraverseDocumentContext, RefNode } from '../types';
import { hasProp } from '../../utils/has_prop';
import { isChildContext } from '../is_child_context';
import { inlineRef } from './utils/inline_ref';
import { insertRefByPointer } from '../../utils/insert_by_json_pointer';
/**
* Node processor to bundle and conditionally dereference document references.
*
* Bundling means all external references like `../../some_file.schema.yaml#/components/schemas/SomeSchema` saved
* to the result document under corresponding path `components` -> `schemas` -> `SomeSchema` and `$ref` property's
* values is updated to `#/components/schemas/SomeSchema`.
*
* Conditional dereference means inlining references when `inliningPredicate()` returns `true`. If `inliningPredicate`
* is not passed only bundling happens.
*/
export class BundleRefProcessor {
private refs: ResolvedRef[] = [];
constructor(private inliningPropName: string) {}
ref(node: RefNode, resolvedRef: ResolvedRef, context: TraverseDocumentContext): void {
if (!resolvedRef.pointer.startsWith('/components/schemas')) {
throw new Error(`$ref pointer must start with "/components/schemas"`);
}
if (
hasProp(node, this.inliningPropName, true) ||
hasProp(resolvedRef.refNode, this.inliningPropName, true)
) {
inlineRef(node, resolvedRef);
delete node[this.inliningPropName];
} else {
const rootDocument = this.extractRootDocument(context);
if (!rootDocument.components) {
rootDocument.components = {};
}
node.$ref = this.saveComponent(
resolvedRef,
rootDocument.components as Record<string, unknown>
);
this.refs.push(resolvedRef);
}
}
getBundledRefs(): ResolvedRef[] {
return this.refs;
}
private saveComponent(ref: ResolvedRef, components: Record<string, unknown>): string {
insertRefByPointer(ref.pointer, ref.refNode, components);
return `#${ref.pointer}`;
}
private extractRootDocument(context: TraverseDocumentContext): Document {
while (isChildContext(context)) {
context = context.parentContext;
}
return context.resolvedDocument.document;
}
}

38
packages/kbn-openapi-bundler/src/bundler/document_processors/modify_partial.ts Normal file
View file

@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { DocumentNodeProcessor } from '../types';
import { hasProp } from '../../utils/has_prop';
import { inlineRef } from './utils/inline_ref';
import { X_MODIFY } from '../known_custom_props';
/**
* Creates a node processor to modify a node by removing `required` property when
* `x-modify: partial` property is presented in the node.
*/
export function createModifyPartialProcessor(): DocumentNodeProcessor {
return {
ref(node, resolvedRef) {
if (!hasProp(node, X_MODIFY, 'partial')) {
return;
}
// Inline the ref node because we are gonna modify it
inlineRef(node, resolvedRef);
delete node.required;
},
leave(node) {
if (!hasProp(node, X_MODIFY, 'partial')) {
return;
}
delete node.required;
},
};
}

77
packages/kbn-openapi-bundler/src/bundler/document_processors/modify_required.ts Normal file
View file

@ -0,0 +1,77 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import chalk from 'chalk';
import { logger } from '../../logger';
import { isPlainObjectType } from '../../utils/is_plain_object_type';
import { DocumentNodeProcessor } from '../types';
import { hasProp } from '../../utils/has_prop';
import { X_MODIFY } from '../known_custom_props';
import { inlineRef } from './utils/inline_ref';
/**
* Creates a node processor to modify a node by add or extending `required` property
* when `x-modify: required` property is presented in the node.
*/
export function createModifyRequiredProcessor(): DocumentNodeProcessor {
return {
ref(node, resolvedRef) {
if (!hasProp(node, X_MODIFY, 'required')) {
return;
}
if (!hasProp(resolvedRef.refNode, 'properties')) {
logger.warning(
`Unable to apply ${chalk.blueBright(X_MODIFY)} to ${chalk.cyan(
resolvedRef.pointer
)} because ${chalk.blueBright('properties')} property was not found`
);
return;
}
if (!isPlainObjectType(resolvedRef.refNode.properties)) {
logger.warning(
`Unable to apply ${chalk.blueBright(X_MODIFY)} to ${chalk.cyan(
resolvedRef.pointer
)} because ${chalk.blueBright('properties')} property was not an object`
);
return;
}
// Inline the ref node because we are gonna modify it
inlineRef(node, resolvedRef);
node.required = Object.keys(resolvedRef.refNode.properties);
},
leave(node) {
if (!hasProp(node, X_MODIFY, 'required')) {
return;
}
if (!hasProp(node, 'properties')) {
logger.warning(
`Unable to apply ${chalk.blueBright(X_MODIFY)} to ${chalk.cyan(
node
)} because ${chalk.blueBright('properties')} property was not found`
);
return;
}
if (!isPlainObjectType(node.properties)) {
logger.warning(
`Unable to apply ${chalk.blueBright(X_MODIFY)} to ${chalk.cyan(
node
)} because ${chalk.blueBright('properties')} property was not an object`
);
return;
}
node.required = Object.keys(node.properties);
},
};
}

31
packages/kbn-openapi-bundler/src/bundler/document_processors/remove_props.ts Normal file
View file

@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { isPlainObjectType } from '../../utils/is_plain_object_type';
import { DocumentNodeProcessor } from '../types';
/**
* Creates a node processor to remove specified by `propNames` properties.
*/
export function createRemovePropsProcessor(propNames: string[]): DocumentNodeProcessor {
return {
leave(node) {
if (!isPlainObjectType(node)) {
return;
}
for (const propName of propNames) {
if (!node[propName]) {
continue;
}
delete node[propName];
}
},
};
}

43
packages/kbn-openapi-bundler/src/bundler/document_processors/remove_unused_components.ts Normal file
View file

@ -0,0 +1,43 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { hasProp } from '../../utils/has_prop';
import { isPlainObjectType } from '../../utils/is_plain_object_type';
import { PlainObjectNode, ResolvedRef } from '../types';
/**
* Helps to remove unused components.
*
* To achieve it requires including in document processors list to collect encountered refs
* and then `removeUnusedComponents()` should be invoked after document processing to perform
* actual unused components deletion.
*/
export class RemoveUnusedComponentsProcessor {
private refs = new Set();
ref(node: unknown, resolvedRef: ResolvedRef): void {
// If the reference has been inlined by one of the previous processors skip it
if (!hasProp(node, '$ref')) {
return;
}
this.refs.add(resolvedRef.pointer);
}
removeUnusedComponents(components: PlainObjectNode): void {
if (!isPlainObjectType(components.schemas)) {
return;
}
for (const schema of Object.keys(components.schemas)) {
if (!this.refs.has(`/components/schemas/${schema}`)) {
delete components.schemas[schema];
}
}
}
}

24
packages/kbn-openapi-bundler/src/bundler/document_processors/skip_internal_path.ts Normal file
View file

@ -0,0 +1,24 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { DocumentNodeProcessor } from '../types';
/**
* Creates a node processor to skip paths starting with `/internal` and omit them from the result document.
*/
export function createSkipInternalPathProcessor(skipPathPrefix: string): DocumentNodeProcessor {
return {
enter(_, context) {
if (typeof context.parentKey === 'number') {
return false;
}
return context.parentKey.startsWith(skipPathPrefix);
},
};
}

21
packages/kbn-openapi-bundler/src/bundler/document_processors/skip_node_with_internal_prop.ts Normal file
View file

@ -0,0 +1,21 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { DocumentNodeProcessor } from '../types';
/**
* Creates a node processor to skip nodes having provided `skipProperty` property
* and omit them from the result document.
*/
export function createSkipNodeWithInternalPropProcessor(
skipProperty: string
): DocumentNodeProcessor {
return {
enter: (node) => skipProperty in node,
};
}

11
packages/kbn-openapi-bundler/src/bundler/document_processors/types.ts Normal file
View file

@ -0,0 +1,11 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export interface InlinableRefNode {
$ref?: string;
}

20
packages/kbn-openapi-bundler/src/bundler/document_processors/utils/inline_ref.ts Normal file
View file

@ -0,0 +1,20 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { cloneDeep } from 'lodash';
import { DocumentNode, ResolvedRef } from '../../types';
import { InlinableRefNode } from '../types';
export function inlineRef(node: DocumentNode, resolvedRef: ResolvedRef): void {
// Make sure unwanted side effects don't happen when child nodes are processed
const deepClone = cloneDeep(resolvedRef.refNode);
Object.assign(node, deepClone);
delete (node as InlinableRefNode).$ref;
}

15
packages/kbn-openapi-bundler/src/bundler/is_child_context.ts Normal file
View file

@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { TraverseChildDocumentContext, TraverseDocumentContext } from './types';
export function isChildContext(
context: TraverseDocumentContext
): context is TraverseChildDocumentContext {
return 'parentContext' in context;
}

31
packages/kbn-openapi-bundler/src/bundler/known_custom_props.ts Normal file
View file

@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
/**
* `x-internal: true` marks nodes the bundler must NOT include in the result bundled document. Any other values are ignored.
*/
export const X_INTERNAL = 'x-internal';
/**
* `x-internal: true` marks reference nodes the bundler must inline in the result bundled document.
*/
export const X_INLINE = 'x-inline';
/**
* `x-modify` marks nodes to be modified by the bundler. `partial` and `required` values are supported.
*
* - `partial` leads to removing `required` property making params under `properties` optional
* - `required` leads to adding or extending `required` property by adding all param names under `properties`
*/
export const X_MODIFY = 'x-modify';
/**
* `x-codegen-enabled` is used by the code generator package `@kbn/openapi-generator` and shouldn't be included
* in result bundled document.
*/
export const X_CODEGEN_ENABLED = 'x-codegen-enabled';

142
packages/kbn-openapi-bundler/src/bundler/merge_documents.ts Normal file
View file

@ -0,0 +1,142 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import deepEqual from 'fast-deep-equal';
import { basename, dirname, join } from 'path';
import chalk from 'chalk';
import { parseRef } from '../utils/parse_ref';
import { insertRefByPointer } from '../utils/insert_by_json_pointer';
import { DocumentNodeProcessor, PlainObjectNode, ResolvedDocument, ResolvedRef } from './types';
import { BundledDocument } from './bundle_document';
import { processDocument } from './process_document';
type MergedDocuments = Record<string, ResolvedDocument>;
type MergedResult = Record<string, PlainObjectNode>;
const SHARED_COMPONENTS_FILE_NAME = 'shared_components.schema.yaml';
export async function mergeDocuments(bundledDocuments: BundledDocument[]): Promise<MergedResult> {
const mergedDocuments: MergedDocuments = {};
const componentsMap = new Map<string, ResolvedRef>();
for (const bundledDocument of bundledDocuments) {
mergeRefsToMap(bundledDocument.bundledRefs, componentsMap);
delete bundledDocument.document.components;
await setRefsFileName(bundledDocument, SHARED_COMPONENTS_FILE_NAME);
mergeDocument(bundledDocument, mergedDocuments);
}
const result: MergedResult = {};
for (const fileName of Object.keys(mergedDocuments)) {
result[fileName] = mergedDocuments[fileName].document;
}
result[SHARED_COMPONENTS_FILE_NAME] = {
components: componentsMapToComponents(componentsMap),
};
return result;
}
function mergeDocument(resolvedDocument: ResolvedDocument, mergeResult: MergedDocuments): void {
const fileName = basename(resolvedDocument.absolutePath);
if (!mergeResult[fileName]) {
mergeResult[fileName] = resolvedDocument;
return;
}
const nonConflictFileName = generateNonConflictingFilePath(
resolvedDocument.absolutePath,
mergeResult
);
mergeResult[nonConflictFileName] = resolvedDocument;
}
function generateNonConflictingFilePath(
documentAbsolutePath: string,
mergeResult: MergedDocuments
): string {
let pathToDocument = dirname(documentAbsolutePath);
let suggestedName = basename(documentAbsolutePath);
while (mergeResult[suggestedName]) {
suggestedName = `${basename(pathToDocument)}_${suggestedName}`;
pathToDocument = join(pathToDocument, '..');
}
return suggestedName;
}
function mergeRefsToMap(bundledRefs: ResolvedRef[], componentsMap: Map<string, ResolvedRef>): void {
for (const bundledRef of bundledRefs) {
const existingRef = componentsMap.get(bundledRef.pointer);
if (!existingRef) {
componentsMap.set(bundledRef.pointer, bundledRef);
continue;
}
if (deepEqual(existingRef.refNode, bundledRef.refNode)) {
continue;
}
throw new Error(
`❌ Unable to bundle documents due to conflicts in references. Schema ${chalk.yellow(
bundledRef.pointer
)} is defined in ${chalk.blue(existingRef.absolutePath)} and in ${chalk.magenta(
bundledRef.absolutePath
)} but has not matching definitions.`
);
}
}
function componentsMapToComponents(
componentsMap: Map<string, ResolvedRef>
): Record<string, unknown> {
const result: Record<string, unknown> = {};
for (const resolvedRef of componentsMap.values()) {
insertRefByPointer(resolvedRef.pointer, resolvedRef.refNode, result);
}
return result;
}
async function setRefsFileName(
resolvedDocument: ResolvedDocument,
fileName: string
): Promise<void> {
// We don't need to follow references
const stubRefResolver = {
resolveRef: async (refDocumentAbsolutePath: string, pointer: string): Promise<ResolvedRef> => ({
absolutePath: refDocumentAbsolutePath,
pointer,
document: resolvedDocument.document,
refNode: {},
}),
resolveDocument: async (): Promise<ResolvedDocument> => ({
absolutePath: '',
document: resolvedDocument.document,
}),
};
const setRefFileProcessor: DocumentNodeProcessor = {
ref: (node) => {
const { pointer } = parseRef(node.$ref);
node.$ref = `./${fileName}#${pointer}`;
},
};
await processDocument(resolvedDocument, stubRefResolver, [setRefFileProcessor]);
}

224
packages/kbn-openapi-bundler/src/bundler/process_document.test.ts Normal file
View file

@ -0,0 +1,224 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { processDocument } from './process_document';
import { RefResolver } from './ref_resolver';
import { Document, DocumentNodeProcessor } from './types';
jest.mock('./ref_resolver');
describe('processDocument', () => {
it('invokes processors in the provided order', async () => {
const resolvedDocument = {
absolutePath: '/path/to/document',
document: {} as Document,
};
const calls: string[] = [];
const processor1 = {
leave() {
calls.push('processor1');
},
};
const processor2 = {
leave() {
calls.push('processor2');
},
};
processDocument(resolvedDocument, new RefResolver(), [processor1, processor2]);
expect(calls).toEqual(['processor1', 'processor2']);
});
it('invokes callbacks in expected order (enter -> ref -> leave)', async () => {
const document = {
id: 'root',
t1: {
id: 't1',
$ref: '#/TestRef',
},
};
const calls: string[] = [];
const refResolver = new RefResolver();
const processor: DocumentNodeProcessor = {
enter(node) {
calls.push(`enter - ${(node as NodeWithId).id}`);
return false;
},
ref(node) {
calls.push(`ref - ${(node as NodeWithId).id}`);
},
leave(node) {
calls.push(`leave - ${(node as NodeWithId).id}`);
},
};
const refNode = {
id: 'TestRef',
bar: 'foo',
};
(refResolver.resolveRef as jest.Mock).mockResolvedValue({
absolutePath: '/path/to/document',
document: {
TestRef: refNode,
},
refNode,
pointer: '/TestRef',
});
await processDocument(
{
absolutePath: '/path/to/document',
document: document as unknown as Document,
},
refResolver,
[processor]
);
expect(calls).toEqual([
'enter - root',
'enter - t1',
'enter - TestRef',
'leave - TestRef',
'ref - t1',
'leave - t1',
'leave - root',
]);
});
it('removes a node after "enter" callback returned true', async () => {
const nodeToRemove = {
id: 't2',
foo: 'bar',
};
const document = {
t1: {
id: 't1',
},
t2: nodeToRemove,
};
const removeNodeProcessor: DocumentNodeProcessor = {
enter(node) {
return node === nodeToRemove;
},
};
await processDocument(
{
absolutePath: '/path/to/document',
document: document as unknown as Document,
},
new RefResolver(),
[removeNodeProcessor]
);
expect(document).toEqual({
t1: {
id: 't1',
},
});
});
it('handles recursive documents', async () => {
const nodeA: Record<string, unknown> = {
foo: 'bar',
};
const nodeB: Record<string, unknown> = {
bar: ' foo',
};
nodeA.circular = nodeB;
nodeB.circular = nodeA;
const document = {
nodeA,
nodeB,
};
await processDocument(
{
absolutePath: '/path/to/document',
document: document as unknown as Document,
},
new RefResolver(),
[]
);
expect(document).toBeDefined();
});
it('handles self-recursive references', async () => {
const document = {
node: {
$ref: '#/TestComponentCircular',
},
TestComponentCircular: {
$ref: '#/TestComponentCircular',
},
};
const refResolver = new RefResolver();
(refResolver.resolveRef as jest.Mock).mockResolvedValue({
absolutePath: '/path/to/document',
document,
refNode: {
$ref: '#/TestComponentCircular',
},
pointer: '/TestComponentCircular',
});
await processDocument(
{
absolutePath: '/path/to/document',
document: document as unknown as Document,
},
refResolver,
[]
);
expect(document).toBeDefined();
});
it('handles recursive references', async () => {
const document: Record<string, unknown> = {
node: {
$ref: '#/TestComponentCircular',
},
TestComponentCircular: {
$ref: '#/AnotherTestComponentCircular',
},
AnotherTestComponentCircular: {
$ref: '#/TestComponentCircular',
},
};
const refResolver = new RefResolver();
(refResolver.resolveRef as jest.Mock).mockImplementation((_, pointer) => ({
absolutePath: '/path/to/document',
document,
refNode: document[pointer.slice(1)],
pointer,
}));
await processDocument(
{
absolutePath: '/path/to/document',
document: document as unknown as Document,
},
refResolver,
[]
);
expect(document).toBeDefined();
});
});
interface NodeWithId {
id?: string;
}

203
packages/kbn-openapi-bundler/src/bundler/process_document.ts Normal file
View file

@ -0,0 +1,203 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { dirname } from 'path';
import { isPlainObject } from 'lodash';
import { IRefResolver } from './ref_resolver';
import {
DocumentNode,
ResolvedDocument,
TraverseDocumentContext,
ResolvedRef,
DocumentNodeProcessor,
RefNode,
PlainObjectNode,
} from './types';
import { parseRef } from '../utils/parse_ref';
import { toAbsolutePath } from '../utils/to_absolute_path';
import { isPlainObjectType } from '../utils/is_plain_object_type';
import { isChildContext } from './is_child_context';
interface TraverseItem {
node: DocumentNode;
context: TraverseDocumentContext;
/**
* Keeps track of visited nodes to be able to detect circular references
*/
visitedDocumentNodes: Set<DocumentNode>;
parentNode: DocumentNode;
parentKey: string | number;
resolvedRef?: ResolvedRef;
}
export async function processDocument(
resolvedDocument: ResolvedDocument,
refResolver: IRefResolver,
processors: DocumentNodeProcessor[]
): Promise<void> {
const nodesToVisit: TraverseItem[] = [
{
node: resolvedDocument.document,
context: {
resolvedDocument,
},
visitedDocumentNodes: new Set(),
parentNode: resolvedDocument.document,
parentKey: '',
},
];
const postOrderTraversalStack: TraverseItem[] = [];
while (nodesToVisit.length > 0) {
const traverseItem = nodesToVisit.pop() as TraverseItem;
if (!isTraversableNode(traverseItem.node)) {
continue;
}
if (traverseItem.visitedDocumentNodes.has(traverseItem.node)) {
// Circular reference in the current document detected
continue;
}
traverseItem.visitedDocumentNodes.add(traverseItem.node);
if (shouldSkipNode(traverseItem, processors)) {
removeNode(traverseItem);
continue;
}
postOrderTraversalStack.push(traverseItem);
if (isRefNode(traverseItem.node)) {
const currentDocument = isChildContext(traverseItem.context)
? traverseItem.context.resolvedRef
: traverseItem.context.resolvedDocument;
const { path, pointer } = parseRef(traverseItem.node.$ref);
const refAbsolutePath = path
? toAbsolutePath(path, dirname(currentDocument.absolutePath))
: currentDocument.absolutePath;
const absoluteRef = `${refAbsolutePath}#${pointer}`;
if (isCircularRef(absoluteRef, traverseItem.context)) {
continue;
}
const resolvedRef = await refResolver.resolveRef(refAbsolutePath, pointer);
const childContext = {
resolvedRef,
parentContext: traverseItem.context,
followedRef: absoluteRef,
};
traverseItem.resolvedRef = resolvedRef;
nodesToVisit.push({
node: resolvedRef.refNode,
context: childContext,
visitedDocumentNodes: new Set(),
parentNode: traverseItem.parentNode,
parentKey: traverseItem.parentKey,
});
continue;
}
if (Array.isArray(traverseItem.node)) {
for (let i = 0; i < traverseItem.node.length; ++i) {
const nodeItem = traverseItem.node[i];
nodesToVisit.push({
node: nodeItem as DocumentNode,
context: traverseItem.context,
visitedDocumentNodes: traverseItem.visitedDocumentNodes,
parentNode: traverseItem.node,
parentKey: i,
});
}
}
if (isPlainObjectType(traverseItem.node)) {
for (const key of Object.keys(traverseItem.node)) {
const value = traverseItem.node[key];
nodesToVisit.push({
node: value as DocumentNode,
context: traverseItem.context,
visitedDocumentNodes: traverseItem.visitedDocumentNodes,
parentNode: traverseItem.node,
parentKey: key,
});
}
}
}
for (let i = postOrderTraversalStack.length - 1; i >= 0; --i) {
const traverseItem = postOrderTraversalStack[i];
for (const processor of processors) {
// If ref has been inlined by one of the processors it's not a ref node anymore
// so we can skip the following processors
if (isRefNode(traverseItem.node) && traverseItem.resolvedRef) {
processor.ref?.(
traverseItem.node as RefNode,
traverseItem.resolvedRef,
traverseItem.context
);
}
processor.leave?.(traverseItem.node, traverseItem.context);
}
}
}
function isTraversableNode(maybeTraversableNode: unknown): boolean {
// We need to process only objects and arrays. Scalars pass through as is.
return typeof maybeTraversableNode === 'object' && maybeTraversableNode !== null;
}
export function isRefNode(node: DocumentNode): node is { $ref: string } {
return isPlainObject(node) && '$ref' in node;
}
function shouldSkipNode(traverseItem: TraverseItem, processors: DocumentNodeProcessor[]): boolean {
return processors?.some((p) =>
p.enter?.(traverseItem.node, {
...traverseItem.context,
parentNode: traverseItem.parentNode,
parentKey: traverseItem.parentKey,
})
);
}
function removeNode(traverseItem: TraverseItem): void {
if (Array.isArray(traverseItem.parentNode) && typeof traverseItem.parentKey === 'number') {
traverseItem.parentNode.splice(traverseItem.parentKey, 1);
return;
}
delete (traverseItem.parentNode as PlainObjectNode)[traverseItem.parentKey];
}
function isCircularRef(absoluteRef: string, context: TraverseDocumentContext): boolean {
let nextContext: TraverseDocumentContext | undefined = context;
if (!isChildContext(nextContext)) {
return false;
}
do {
if (nextContext.followedRef === absoluteRef) {
return true;
}
nextContext = nextContext.parentContext;
} while (nextContext);
return false;
}

62
packages/kbn-openapi-bundler/src/bundler/ref_resolver.ts Normal file
View file

@ -0,0 +1,62 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import path from 'path';
import { extractByJsonPointer } from '../utils/extract_by_json_pointer';
import { readYamlDocument } from '../utils/read_yaml_document';
import { ResolvedDocument, ResolvedRef } from './types';
export interface IRefResolver {
resolveRef(refDocumentAbsolutePath: string, pointer: string): Promise<ResolvedRef>;
resolveDocument(documentAbsolutePath: string): Promise<ResolvedDocument>;
}
export class RefResolver implements IRefResolver {
private documentsCache = new Map<string, ResolvedDocument>();
async resolveRef(refDocumentAbsolutePath: string, pointer: string): Promise<ResolvedRef> {
const resolvedRefDocument = await this.resolveDocument(refDocumentAbsolutePath);
const refNode = extractByJsonPointer(resolvedRefDocument.document, pointer);
const resolvedRef = {
absolutePath: refDocumentAbsolutePath,
pointer,
document: resolvedRefDocument.document,
refNode,
};
return resolvedRef;
}
async resolveDocument(documentAbsolutePath: string): Promise<ResolvedDocument> {
if (!path.isAbsolute(documentAbsolutePath)) {
throw new Error(
`resolveDocument requires absolute document path, provided path "${documentAbsolutePath}" is not absolute`
);
}
const cachedDocument = this.documentsCache.get(documentAbsolutePath);
if (cachedDocument) {
return cachedDocument;
}
try {
const document = await readYamlDocument(documentAbsolutePath);
const resolvedRef = {
absolutePath: documentAbsolutePath,
document,
};
this.documentsCache.set(documentAbsolutePath, resolvedRef);
return resolvedRef;
} catch (e) {
throw new Error(`Unable to resolve document "${documentAbsolutePath}"`, { cause: e });
}
}
}

134
packages/kbn-openapi-bundler/src/bundler/types.ts Normal file
View file

@ -0,0 +1,134 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
/**
* A plain object node not containing `$ref` property
*/
export type PlainObjectNode = Record<string, unknown>;
/**
* An array node
*/
export type ArrayNode = unknown[];
/**
* A ref node containing `$ref` property besides the others
*/
export interface RefNode extends PlainObjectNode {
$ref: string;
}
/**
* An abstract OpenAPI entry node. Content besides $ref isn't important.
*/
export type DocumentNode = PlainObjectNode | ArrayNode | RefNode;
/**
* Document abstraction. We don't mind OpenAPI `3.0` and `3.1` differences.
*/
export type Document = Record<string, unknown>;
export interface ResolvedDocument {
/**
* Document's absolute path
*/
absolutePath: string;
/**
* Document's root
*/
document: Document;
}
export interface ResolvedRef extends ResolvedDocument {
/**
* Parsed pointer without leading hash symbol (e.g. `/components/schemas/MySchema`)
*/
pointer: string;
/**
* Resolved ref's node pointer points to
*/
refNode: DocumentNode;
}
export interface TraverseRootDocumentContext {
/**
* Root document
*/
resolvedDocument: ResolvedDocument;
parentContext?: undefined;
followedRef?: undefined;
}
export interface TraverseChildDocumentContext {
/**
* Current document after resolving $ref property
*/
resolvedRef: ResolvedRef;
/**
* Context of the parent document the current one in `document` field was referenced via $ref. Empty if it's the root document.
*/
parentContext: TraverseDocumentContext;
/**
* Reference used to resolve the current document
*/
followedRef: string;
}
/**
* Traverse context storing additional information related to the currently traversed node
*/
export type TraverseDocumentContext = TraverseRootDocumentContext | TraverseChildDocumentContext;
export type TraverseDocumentEntryContext = TraverseDocumentContext & {
parentNode: DocumentNode;
parentKey: string | number;
};
/**
* Entry processor controls when a node should be omitted from the result document.
*
* When result is `true` - omit the node.
*/
export type EntryProcessorFn = (
node: Readonly<DocumentNode>,
context: TraverseDocumentEntryContext
) => boolean;
export type LeaveProcessorFn = (node: DocumentNode, context: TraverseDocumentContext) => void;
export type RefProcessorFn = (
node: RefNode,
resolvedRef: ResolvedRef,
context: TraverseDocumentContext
) => void;
/**
* Document or document node processor gives flexibility in modifying OpenAPI specs and/or collect some metrics.
* For convenience it defined handlers invoked upon action or specific node type.
*
* Currently the following node types supported
*
* - ref - Callback function is invoked upon leaving ref node (a node having `$ref` key)
*
* and the following actions
*
* - enter - Callback function is invoked upon entering any type of node element including ref nodes. It doesn't allow
* to modify node's content but provides an ability to remove the element by returning `true`.
*
* - leave - Callback function is invoked upon leaving any type of node. It give an opportunity to modify the document like
* dereference refs or remove unwanted properties.
*/
export interface DocumentNodeProcessor {
enter?: EntryProcessorFn;
leave?: LeaveProcessorFn;
ref?: RefProcessorFn;
}

14
packages/kbn-openapi-bundler/src/logger.ts Normal file
View file

@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { ToolingLog } from '@kbn/tooling-log';
export const logger = new ToolingLog({
level: 'debug',
writeTo: process.stdout,
});

162
packages/kbn-openapi-bundler/src/openapi_bundler.test.ts Normal file
View file

@ -0,0 +1,162 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { existsSync, rmSync } from 'fs';
import { basename, join } from 'path';
import { bundle } from './openapi_bundler';
import { readYamlDocument } from './utils/read_yaml_document';
const rootPath = join(__dirname, '__test__');
const targetAbsoluteFilePath = join(rootPath, 'bundled.yaml');
describe('OpenAPI Bundler', () => {
afterEach(() => {
removeTargetFile();
});
it('bundles two simple specs', async () => {
await bundleFolder('two_simple_specs');
await expectBundleToMatchFile('two_simple_specs', 'expected.yaml');
});
it('bundles one file with a local reference', async () => {
await bundleFolder('spec_with_local_ref');
await expectBundleToMatchFile('spec_with_local_ref', 'expected.yaml');
});
it('bundles one file with an external reference', async () => {
await bundleFolder('spec_with_external_ref');
await expectBundleToMatchFile('spec_with_external_ref', 'expected.yaml');
});
it('bundles files with external references', async () => {
await bundleFolder('two_specs_with_external_ref');
await expectBundleToMatchFile('two_specs_with_external_ref', 'expected.yaml');
});
// Fails because `writeYamlDocument()` has `noRefs: true` setting
// it('bundles recursive spec', async () => {
// await bundleFolder('recursive_spec');
// await expectBundleToMatchFile('recursive_spec', 'expected.yaml');
// });
it('bundles specs with recursive references', async () => {
await bundleFolder('recursive_ref_specs');
await expectBundleToMatchFile('recursive_ref_specs', 'expected.yaml');
});
it('bundles spec with a self-recursive reference', async () => {
await bundleFolder('self_recursive_ref');
await expectBundleToMatchFile('self_recursive_ref', 'expected.yaml');
});
it('bundles one endpoint with different versions', async () => {
await bundleFolder('different_endpoint_versions');
await expectBundleToMatchFile('different_endpoint_versions', 'expected.yaml');
});
it('bundles spec with different OpenAPI versions', async () => {
await bundleFolder('different_openapi_versions');
await expectBundleToMatchFile('different_openapi_versions', 'expected.yaml');
});
it('bundles conflicting but equal references', async () => {
await bundleFolder('conflicting_but_equal_refs_in_different_specs');
await expectBundleToMatchFile('conflicting_but_equal_refs_in_different_specs', 'expected.yaml');
});
it('fails to bundle conflicting references encountered in separate specs', async () => {
await expectBundlingError(
'conflicting_refs_in_different_specs',
/\/components\/schemas\/ConflictTestSchema/
);
});
describe('x-modify', () => {
it('makes properties in an object node partial', async () => {
await bundleFolder('modify_partial_node');
await expectBundleToMatchFile('modify_partial_node', 'expected.yaml');
});
it('makes properties in a referenced object node partial', async () => {
await bundleFolder('modify_partial_ref');
await expectBundleToMatchFile('modify_partial_ref', 'expected.yaml');
});
it('makes properties in an object node required', async () => {
await bundleFolder('modify_required_node');
await expectBundleToMatchFile('modify_required_node', 'expected.yaml');
});
it('makes properties in a referenced object node required', async () => {
await bundleFolder('modify_required_ref');
await expectBundleToMatchFile('modify_required_ref', 'expected.yaml');
});
});
describe('x-inline', () => {
it('inlines a reference', async () => {
await bundleFolder('inline_ref');
await expectBundleToMatchFile('inline_ref', 'expected.yaml');
});
});
describe('skip internal', () => {
it('skips nodes with x-internal property', async () => {
await bundleFolder('skip_internal');
await expectBundleToMatchFile('skip_internal', 'expected.yaml');
});
it('skips endpoints starting with /internal', async () => {
await bundleFolder('skip_internal_endpoint');
await expectBundleToMatchFile('skip_internal_endpoint', 'expected.yaml');
});
});
});
async function bundleFolder(folderName: string): Promise<void> {
await expect(
bundle({
rootDir: join(rootPath, folderName),
sourceGlob: '*.schema.yaml',
outputFilePath: join('..', basename(targetAbsoluteFilePath)),
})
).resolves.toBeUndefined();
}
async function expectBundlingError(
folderName: string,
error: string | RegExp | jest.Constructable | Error | undefined
): Promise<void> {
return await expect(
bundle({
rootDir: join(rootPath, folderName),
sourceGlob: '*.schema.yaml',
outputFilePath: join('..', basename(targetAbsoluteFilePath)),
})
).rejects.toThrowError(error);
}
async function expectBundleToMatchFile(
folderName: string,
expectedFileName: string
): Promise<void> {
expect(existsSync(targetAbsoluteFilePath)).toBeTruthy();
const bundledSpec = await readYamlDocument(targetAbsoluteFilePath);
const expectedAbsoluteFilePath = join(rootPath, folderName, expectedFileName);
const expectedSpec = await readYamlDocument(expectedAbsoluteFilePath);
expect(bundledSpec).toEqual(expectedSpec);
}
function removeTargetFile(): void {
if (existsSync(targetAbsoluteFilePath)) {
rmSync(targetAbsoluteFilePath, { force: true });
}
}

101
packages/kbn-openapi-bundler/src/openapi_bundler.ts Normal file
View file

@ -0,0 +1,101 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import chalk from 'chalk';
import globby from 'globby';
import { basename, dirname, join, resolve } from 'path';
import { BundledDocument, bundleDocument, SkipException } from './bundler/bundle_document';
import { mergeDocuments } from './bundler/merge_documents';
import { removeFilesByGlob } from './utils/remove_files_by_glob';
import { logger } from './logger';
import { writeYamlDocument } from './utils/write_yaml_document';
export interface BundlerConfig {
rootDir: string;
sourceGlob: string;
outputFilePath: string;
}
export const bundle = async (config: BundlerConfig) => {
const {
rootDir,
sourceGlob,
outputFilePath: relativeOutputFilePath = 'target/openapi/bundled.schema.yaml',
} = config;
logger.debug(chalk.bold(`Bundling API route schemas`));
logger.debug(chalk.bold(`Working directory: ${chalk.underline(rootDir)}`));
logger.debug(`👀 Searching for source files`);
const outputFilePath = join(rootDir, relativeOutputFilePath);
const sourceFilesGlob = resolve(rootDir, sourceGlob);
const schemaFilePaths = await globby([sourceFilesGlob]);
logger.info(`🕵️‍♀️ Found ${schemaFilePaths.length} schemas`);
logSchemas(schemaFilePaths);
logger.info(`🧹 Cleaning up any previously generated artifacts`);
await removeFilesByGlob(dirname(outputFilePath), basename(outputFilePath));
logger.debug(`Processing schemas...`);
const resolvedDocuments = await Promise.all(
schemaFilePaths.map(async (schemaFilePath) => {
try {
const resolvedDocument = await bundleDocument(schemaFilePath);
logger.debug(`Processed ${chalk.bold(basename(schemaFilePath))}`);
return resolvedDocument;
} catch (e) {
if (e instanceof SkipException) {
logger.info(`Skipped ${chalk.bold(e.documentPath)}: ${e.message}`);
return;
}
throw e;
}
})
);
const processedDocuments = filterOutSkippedDocuments(resolvedDocuments);
logger.success(`Processed ${processedDocuments.length} schemas`);
const resultDocument = await mergeDocuments(processedDocuments);
try {
await writeYamlDocument(outputFilePath, resultDocument);
logger.success(`📖 Wrote all bundled OpenAPI specs to ${chalk.bold(outputFilePath)}`);
} catch (e) {
logger.error(`Unable to save bundled document to ${chalk.bold(outputFilePath)}: ${e.message}`);
}
};
function logSchemas(schemaFilePaths: string[]): void {
for (const filePath of schemaFilePaths) {
logger.debug(`Found OpenAPI spec ${chalk.bold(filePath)}`);
}
}
function filterOutSkippedDocuments(
documents: Array<BundledDocument | undefined>
): BundledDocument[] {
const processedDocuments: BundledDocument[] = [];
for (const document of documents) {
if (!document) {
continue;
}
processedDocuments.push(document);
}
return processedDocuments;
}

46
packages/kbn-openapi-bundler/src/utils/extract_by_json_pointer.ts Normal file
View file

@ -0,0 +1,46 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { isPlainObjectType } from './is_plain_object_type';
/**
* Extract a node from a document using a provided [JSON Pointer](https://datatracker.ietf.org/doc/html/rfc6901).
*
* JSON Pointer is the second part in [JSON Reference](https://datatracker.ietf.org/doc/html/draft-pbryan-zyp-json-ref-03).
* For example an object `{ $ref: "./some-file.yaml#/components/schemas/MySchema"}` is a reference node.
* Where `/components/schemas/MySchema` is a JSON pointer. `./some-file.yaml` is a document reference.
* Yaml shares the same JSON reference standard and basically can be considered just as a different
* JS Object serialization format. See OpenAPI [Using $ref](https://swagger.io/docs/specification/using-ref/) for more information.
*
* @param document a document containing node to resolve by using the pointer
* @param pointer a JSON Pointer
* @returns resolved document node
*/
export function extractByJsonPointer(document: unknown, pointer: string): Record<string, unknown> {
if (!pointer.startsWith('/')) {
throw new Error('$ref pointer must start with a leading slash');
}
if (!isPlainObjectType(document)) {
throw new Error('document must be an object');
}
let target = document;
for (const segment of pointer.slice(1).split('/')) {
const nextTarget = target[segment];
if (!isPlainObjectType(nextTarget)) {
throw new Error(`JSON Pointer "${pointer}" is not found in "${JSON.stringify(document)}"`);
}
target = nextTarget;
}
return target;
}

21
packages/kbn-openapi-bundler/src/utils/has_prop.ts Normal file
View file

@ -0,0 +1,21 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { isPlainObjectType } from './is_plain_object_type';
export function hasProp<Property extends string, Value extends string | number | boolean>(
node: unknown,
propName: Property,
propValue?: Value
): node is { [key in Property]: Value } & Record<string, unknown> {
if (!isPlainObjectType(node) || !(propName in node)) {
return false;
}
return propValue ? node[propName] === propValue : Boolean(node[propName]);
}

35
packages/kbn-openapi-bundler/src/utils/insert_by_json_pointer.ts Normal file
View file

@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
/**
* Inserts `data` into the location specified by pointer in the `document`.
*
* @param pointer [JSON Pointer](https://datatracker.ietf.org/doc/html/rfc6901)
* @param data An object to insert
* @param document A document to insert to
*/
export function insertRefByPointer(
pointer: string,
data: unknown,
document: Record<string, unknown>
): void {
const segments = pointer.split('/').slice(2);
let target = document;
while (segments.length > 0) {
const segment = segments.shift() as string;
if (!target[segment]) {
target[segment] = {};
}
target = target[segment] as Record<string, unknown>;
}
Object.assign(target, data);
}

13
packages/kbn-openapi-bundler/src/utils/is_plain_object_type.ts Normal file
View file

@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { isPlainObject } from 'lodash';
export function isPlainObjectType(maybeObj: unknown): maybeObj is Record<string, unknown> {
return isPlainObject(maybeObj);
}

31
packages/kbn-openapi-bundler/src/utils/parse_ref.ts Normal file
View file

@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export interface ParsedRef {
path: string;
pointer: string;
}
/**
* Parses [JSON Reference](https://datatracker.ietf.org/doc/html/draft-pbryan-zyp-json-ref-03)
*
* @param ref JSON Reference
* @returns file path and JSON pointer
*/
export function parseRef(ref: string): ParsedRef {
const [filePath, pointer] = ref.split('#');
if (!pointer) {
throw new Error(`Unable to parse $ref "${ref}"`);
}
return {
path: filePath,
pointer,
};
}

17
packages/kbn-openapi-bundler/src/utils/read_yaml_document.ts Normal file
View file

@ -0,0 +1,17 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import fs from 'fs/promises';
import { load } from 'js-yaml';
export async function readYamlDocument(filePath: string): Promise<Record<string, unknown>> {
// Typing load's result to Record<string, unknown> is optimistic as we can't be sure
// there is object inside a yaml file. We don't have this validation layer so far
// but using JSON Schemas here should mitigate this problem.
return load(await fs.readFile(filePath, { encoding: 'utf8' }));
}

23
packages/kbn-openapi-bundler/src/utils/remove_files_by_glob.ts Normal file
View file

@ -0,0 +1,23 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import fs from 'fs/promises';
import globby from 'globby';
import { resolve } from 'path';
/**
* Removes any files matching glob pattern from the target directory
*
* @param path target directory
* @param globPattern files pattern to remove
*/
export async function removeFilesByGlob(path: string, globPattern: string): Promise<void> {
const filesToRemove = await globby([resolve(path, globPattern)]);
await Promise.all(filesToRemove.map((fileName) => fs.unlink(fileName)));
}

25
packages/kbn-openapi-bundler/src/utils/to_absolute_path.ts Normal file
View file

@ -0,0 +1,25 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { join, isAbsolute } from 'path';
/**
* Transforms a path to absolute path. If an absolute path passed to the function it's returned without
* changes. Base path is current working directory by default.
*
* @param maybeAbsolutePath a path to be transformed into absolute path
* @param baseDirPath a path from root to the folder maybeAbsolutePath is relative to
* @returns absolute path
*/
export function toAbsolutePath(maybeAbsolutePath: string, baseDirPath: string): string {
if (isAbsolute(maybeAbsolutePath)) {
return maybeAbsolutePath;
}
return join(baseDirPath, maybeAbsolutePath);
}

22
packages/kbn-openapi-bundler/src/utils/write_yaml_document.ts Normal file
View file

@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import fs from 'fs/promises';
import { dump } from 'js-yaml';
import { dirname } from 'path';
export async function writeYamlDocument(filePath: string, document: unknown): Promise<void> {
try {
const yaml = dump(document, { noRefs: true });
await fs.mkdir(dirname(filePath), { recursive: true });
await fs.writeFile(filePath, yaml);
} catch (e) {
throw new Error(`Unable to write bundled yaml: ${e.message}`, { cause: e });
}
}

10
packages/kbn-openapi-bundler/tsconfig.json Normal file
View file

@ -0,0 +1,10 @@
{
"compilerOptions": {
"outDir": "target/types",
"types": ["jest", "node"]
},
"exclude": ["target/**/*"],
"extends": "../../tsconfig.base.json",
"include": ["**/*.ts"],
"kbn_references": ["@kbn/tooling-log"]
}

2
tsconfig.base.json
View file

@ -1112,6 +1112,8 @@
"@kbn/oidc-provider-plugin/*": ["x-pack/test/security_api_integration/plugins/oidc_provider/*"], "@kbn/oidc-provider-plugin/*": ["x-pack/test/security_api_integration/plugins/oidc_provider/*"],
"@kbn/open-telemetry-instrumented-plugin": ["test/common/plugins/otel_metrics"], "@kbn/open-telemetry-instrumented-plugin": ["test/common/plugins/otel_metrics"],
"@kbn/open-telemetry-instrumented-plugin/*": ["test/common/plugins/otel_metrics/*"], "@kbn/open-telemetry-instrumented-plugin/*": ["test/common/plugins/otel_metrics/*"],
"@kbn/openapi-bundler": ["packages/kbn-openapi-bundler"],
"@kbn/openapi-bundler/*": ["packages/kbn-openapi-bundler/*"],
"@kbn/openapi-generator": ["packages/kbn-openapi-generator"], "@kbn/openapi-generator": ["packages/kbn-openapi-generator"],
"@kbn/openapi-generator/*": ["packages/kbn-openapi-generator/*"], "@kbn/openapi-generator/*": ["packages/kbn-openapi-generator/*"],
"@kbn/optimizer": ["packages/kbn-optimizer"], "@kbn/optimizer": ["packages/kbn-optimizer"],

10
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/rule_schemas.schema.yaml
View file

@ -7,6 +7,7 @@ components:
x-codegen-enabled: true x-codegen-enabled: true
schemas: schemas:
BaseRequiredFields: BaseRequiredFields:
x-inline: true
type: object type: object
properties: properties:
name: name:
@ -24,6 +25,7 @@ components:
- severity - severity
BaseOptionalFields: BaseOptionalFields:
x-inline: true
type: object type: object
properties: properties:
# Field overrides # Field overrides
@ -73,6 +75,7 @@ components:
$ref: './common_attributes.schema.yaml#/components/schemas/RuleActionThrottle' $ref: './common_attributes.schema.yaml#/components/schemas/RuleActionThrottle'
BaseDefaultableFields: BaseDefaultableFields:
x-inline: true
type: object type: object
properties: properties:
# Main attributes # Main attributes
@ -127,12 +130,14 @@ components:
$ref: './common_attributes.schema.yaml#/components/schemas/ThreatArray' $ref: './common_attributes.schema.yaml#/components/schemas/ThreatArray'
BaseCreateProps: BaseCreateProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BaseRequiredFields' - $ref: '#/components/schemas/BaseRequiredFields'
- $ref: '#/components/schemas/BaseOptionalFields' - $ref: '#/components/schemas/BaseOptionalFields'
- $ref: '#/components/schemas/BaseDefaultableFields' - $ref: '#/components/schemas/BaseDefaultableFields'
BasePatchProps: BasePatchProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BaseRequiredFields' - $ref: '#/components/schemas/BaseRequiredFields'
x-modify: partial x-modify: partial
@ -140,6 +145,7 @@ components:
- $ref: '#/components/schemas/BaseDefaultableFields' - $ref: '#/components/schemas/BaseDefaultableFields'
BaseResponseProps: BaseResponseProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BaseRequiredFields' - $ref: '#/components/schemas/BaseRequiredFields'
- $ref: '#/components/schemas/BaseOptionalFields' - $ref: '#/components/schemas/BaseOptionalFields'
@ -199,6 +205,7 @@ components:
$ref: '../../rule_monitoring/model/execution_summary.schema.yaml#/components/schemas/RuleExecutionSummary' $ref: '../../rule_monitoring/model/execution_summary.schema.yaml#/components/schemas/RuleExecutionSummary'
SharedCreateProps: SharedCreateProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BaseCreateProps' - $ref: '#/components/schemas/BaseCreateProps'
- type: object - type: object
@ -207,6 +214,7 @@ components:
$ref: './common_attributes.schema.yaml#/components/schemas/RuleSignatureId' $ref: './common_attributes.schema.yaml#/components/schemas/RuleSignatureId'
SharedUpdateProps: SharedUpdateProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BaseCreateProps' - $ref: '#/components/schemas/BaseCreateProps'
- type: object - type: object
@ -217,6 +225,7 @@ components:
$ref: './common_attributes.schema.yaml#/components/schemas/RuleSignatureId' $ref: './common_attributes.schema.yaml#/components/schemas/RuleSignatureId'
SharedPatchProps: SharedPatchProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BasePatchProps' - $ref: '#/components/schemas/BasePatchProps'
- type: object - type: object
@ -227,6 +236,7 @@ components:
$ref: './common_attributes.schema.yaml#/components/schemas/RuleSignatureId' $ref: './common_attributes.schema.yaml#/components/schemas/RuleSignatureId'
SharedResponseProps: SharedResponseProps:
x-inline: true
allOf: allOf:
- $ref: '#/components/schemas/BaseResponseProps' - $ref: '#/components/schemas/BaseResponseProps'
- $ref: '#/components/schemas/ResponseRequiredFields' - $ref: '#/components/schemas/ResponseRequiredFields'

2
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/get_prebuilt_rules_and_timelines_status/get_prebuilt_rules_and_timelines_status_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Prebuilt Rules Status API endpoint title: Prebuilt Rules Status API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/prepackaged/_status: /api/detection_engine/rules/prepackaged/_status:
get: get:

2
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/install_prebuilt_rules_and_timelines/install_prebuilt_rules_and_timelines_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Install Prebuilt Rules API endpoint title: Install Prebuilt Rules API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/prepackaged: /api/detection_engine/rules/prepackaged:
put: put:

3
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/bulk_actions/bulk_actions_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Bulk Actions API endpoint title: Bulk Actions API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_bulk_action: /api/detection_engine/rules/_bulk_action:
post: post:
@ -169,6 +169,7 @@ components:
type: string type: string
BulkActionBase: BulkActionBase:
x-inline: true
type: object type: object
properties: properties:
query: query:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/bulk_crud/bulk_create_rules/bulk_create_rules_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Bulk Create API endpoint title: Bulk Create API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_bulk_create: /api/detection_engine/rules/_bulk_create:
post: post:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/bulk_crud/bulk_delete_rules/bulk_delete_rules_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Bulk Delete API endpoint title: Bulk Delete API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_bulk_delete: /api/detection_engine/rules/_bulk_delete:
delete: delete:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/bulk_crud/bulk_patch_rules/bulk_patch_rules_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Bulk Patch API endpoint title: Bulk Patch API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_bulk_update: /api/detection_engine/rules/_bulk_update:
patch: patch:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/bulk_crud/bulk_update_rules/bulk_update_rules_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Bulk Update API endpoint title: Bulk Update API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_bulk_update: /api/detection_engine/rules/_bulk_update:
put: put:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/crud/create_rule/create_rule_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Create Rule API endpoint title: Create Rule API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules: /api/detection_engine/rules:
post: post:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/crud/delete_rule/delete_rule_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Delete Rule API endpoint title: Delete Rule API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules: /api/detection_engine/rules:
delete: delete:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/crud/patch_rule/patch_rule_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Patch Rule API endpoint title: Patch Rule API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules: /api/detection_engine/rules:
patch: patch:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/crud/read_rule/read_rule_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Read Rule API endpoint title: Read Rule API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules: /api/detection_engine/rules:
get: get:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/crud/update_rule/update_rule_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Update Rule API endpoint title: Update Rule API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules: /api/detection_engine/rules:
put: put:

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/export_rules/export_rules_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Export Rules API endpoint title: Export Rules API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_export: /api/detection_engine/rules/_export:
summary: Exports rules to an `.ndjson` file summary: Exports rules to an `.ndjson` file

2
x-pack/plugins/security_solution/common/api/detection_engine/rule_management/find_rules/find_rules_route.schema.yaml
View file

@ -1,7 +1,7 @@
openapi: 3.0.0 openapi: 3.0.0
info: info:
title: Find Rules API endpoint title: Find Rules API endpoint
version: 2023-10-31 version: '2023-10-31'
paths: paths:
/api/detection_engine/rules/_find: /api/detection_engine/rules/_find:
get: get:

Some files were not shown because too many files have changed in this diff Show more