github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-23 17:28:26 -04:00
Code Issues Projects Releases Packages Wiki Activity

Add http2 support for Kibana server (#183465)

Browse source 
## Summary

Part of https://github.com/elastic/kibana/issues/7104

Add support for `http2` to the Kibana server. `http2` can be enabled by
setting `server.protocol: http2` in the Kibana config file.

*Note: by default, enabling `http2` requires a valid `h2c`
configuration, meaning that it can only run over HTTPS with TLS1.2+*

```yaml
## kibana.yaml
server.protocol: http2
server.ssl.enabled: true
server.ssl.key: path/to/key
server.ssl.certificate: path/my/cerf
```

## What is this PR doing

### Add HTTP2 support for the Kibana server

#### - Plug http2 to the Kibana server 

Even if HAPI was never officially updated to really support HTTP2,
node's `http`/`https`/`http2` modules are compatible enough to be able
to just instantiate an http2 server/listener and provide it to HAPI "as
a plain https listener". There were some tweaks to do (mostly silencing
a few warnings that HAPI was causing by sending http2-illegal headers
such as `Connection`), but overall, it went smoothly.

#### - Add config validation

By default, Kibana will require a valid `h2c` configuration to accept
enabling `http2`. It means that TLS must be enabled and that TLS1.2+
should at least be in the list of supported SSL protocols
(`server.ssl.supportedProtocols`). Note that default value of this
setting includes TLS1.2 and 1.3.

#### - Add escape hatch to run `h2` without `h2c`

In some situations, it may be required to enable http2 without a valid
`h2c` configuration. Kibana supports it, by setting
`server.http2.allowUnsecure` to `true`.

(*Note, however, that if http2 is enabled without TLS, ALPN protocol
negotiation won't work, meaning that most http2 agents/clients will fail
connecting unless they're explictly configured to use http2.*)

### Add documentation about this new feature

#### - Update the user-facing doc about this new `server.protocol`
setting

Update the user-facing Kibana settings documentation to include this
`http.protocol` setting (and refer to `server.http2.allowUnsecure`)

**Note: this setting, and this feature, are considered as experimental**

### Adapt our dev tooling to support running Kibana with http2 enabled

#### - Add a `--http2` flag to the dev CLI

Enabling this flag will add the proper configuration settings to run
Kibana with `http2` enabled in an (almost) valid `h2c` configutation.

*Note: when using this flag, even if listening on the same port, the
Kibana server will be accessible over https, meaning that you need to
use https in your browser to access it. Aka `http://localhost:5601`
won't work, you need to use `https://localhost:5601`. Also, we're using
the self-signed dev certificates, meaning that you must go though the
scary warning of your browser*

#### - Implement an http2-compatible base-path proxy

The current base path proxy is based on `hapi` and `hapi/h2o2`. I tried
for a bunch hours trying to hack around to make it work with http2
proxying, but ultimately gave up and implemented a new version from
scratch.

Note that with some additional efforts, this new http2 basepath proxy
could probably fully replace the existing one and be used for both http1
and http2 traffic, but it's an optimization / refactoring that did not
feel required for this PR.

### Adapt the FTR to run suites against http2

#### - Add support to run FTR test suite against an h2c-enabled Kibana

Note that with ALPN, clients using http1 should be (and are) able to
communicate with http2 Kibana, given h2c/alpn allows protocol
negitiation. So adapting our FTR tooling was not really about making it
work with http2 (which worked out of the box), but making it work with
**the self signed certifcates we use for https on dev mode**

Note that I'm not a big fan of what I had to do, however, realistically
this was the only possible approach if we want to run arbitrary test
suites with TLS/HTTP2 enabled without massively changing our FTR setup.

Operations and QA, feel free to chime in there, as this is your
territory.

#### - Change some FTR test suites to run against an HTTP2-enabled
server

I added a quick `configureHTTP2` helper function to take any "final" FTR
suite config and mutate it to enable `http2`. I then enabled it on a few
suites locally, to make sure the suites were passing correctly.

I kept two suites running with http2 enabled:
- the `console` oss functional tests
- the `home` oss functional tests

We could possibly enable it for more, but we need to figure out what
kind of strategy we want on that matter (see below)

## What is this pull request NOT doing

#### - Making sure everything works when HTTP2 is enabled

I navigated the applications quite a bit, and did not see anything
broken, however I obviously wasn't able to do a full coverage. Also, the
self-signed certificate was a huge pain to detect issues really caused
by http2 compared to issues because the local setup isn't valid `h2c`.

In theory though (famous last words) anything not doing http/1.1
specific hacks such as bfetch should work fine with http2, given that
even if using non-http2 clients, ALPN should just allow to fallback to
http/1.x (this part was tested)

#### - Enabling HTTP2 by default

PR isn't doing it for obvious reasons. 

#### - Enabling HTTP2 for all FTR suites

First of all, it's not that easy, because it requires adapting various
parts of the config (and even some var env...), and we don't have any
proper way to override config "at the end". For instance, if you add the
http2 config on a top level config (e.g. the oss functional one that is
reuse by the whole world - learned the hard way), it won't work because
higher-level configs redefined (and override) the `browser` part of the
config, loosing the settings added to run the browser in insecure mode.

Secondly, I'm not sure we really need to run that many suites with http2
enabled. I learned working on that PR that we only have like one suite
where https is enabled for the Kibana server, and I feel like it could
be fine to have the same for http2. In theory it's just a protocol
change, unless parts of our apps (e.g. bfetch) are doing things that are
specific to http/1.1, switching to http2 should be an implementation
detail.

But I'd love to get @elastic/kibana-operations and @elastic/appex-qa
opinion on that one, given they have more expertise than I do on that
area.

- Running performances tests

We should absolutely run perf testing between http/1.1 over https and
http/2, to make sure that it goes into the right directly (at least in
term of user perceived speed), but I did not do it in the scope of this
PR (and @dmlemeshko is on PTO so... 😅)

## Release Note

Add support for `http2` to the Kibana server. `http2` can be enabled by
setting `server.protocol: http2` in the Kibana config file.

Note: by default, enabling `http2` requires a valid `h2c` configuration,
meaning that it can only run over HTTPS with TLS1.2+

Please refer to the Kibana config documentation for more details.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Pierre Gayvallet 2024-06-03 09:34:13 +02:00 • committed by GitHub
parent dd35670894
commit dea26c6450
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
52 changed files with 1613 additions and 168 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/setup/settings.asciidoc
View file

@ -456,6 +456,14 @@ identifies this {kib} instance. *Default: `"your-hostname"`*
{kib} is served by a back end server. This
setting specifies the port to use. *Default: `5601`*
`server.protocol`::
experimental[] The http protocol to use, either `http1` or `http2`. Set to `http2` to enable `HTTP/2` support for the {kib} server.
*Default: `http1`*
+
NOTE: By default, enabling `http2` requires a valid `h2c` configuration, meaning that TLS must be enabled via <<server-ssl-enabled, `server.ssl.enabled`>>
and <<server-ssl-supportedProtocols, `server.ssl.supportedProtocols`>>, if specified, must contain at least `TLSv1.2` or `TLSv1.3`. Strict validation of
the `h2c` setup can be disabled by adding `server.http2.allowUnsecure: true` to the configuration.
[[server-requestId-allowFromAnyIp]] `server.requestId.allowFromAnyIp`::
Sets whether or not the `X-Opaque-Id` header should be trusted from any IP address for identifying requests in logs and forwarded to Elasticsearch.

2
package.json
View file

@ -1627,6 +1627,8 @@
"html": "1.0.0",
"html-loader": "^1.3.2",
"http-proxy": "^1.18.1",
"http2-proxy": "^5.0.53",
"http2-wrapper": "^2.2.1",
"ignore": "^5.3.0",
"jest": "^29.6.1",
"jest-canvas-mock": "^2.5.2",

28
packages/core/http/core-http-router-server-internal/src/request.test.ts
View file

@ -217,7 +217,31 @@ describe('CoreKibanaRequest', () => {
});
describe('route.protocol property', () => {
it('return a static value for now as only http1 is supported', () => {
it('return the correct value for http/1.0 requests', () => {
const request = hapiMocks.createRequest({
raw: {
req: {
httpVersion: '1.0',
},
},
});
const kibanaRequest = CoreKibanaRequest.from(request);
expect(kibanaRequest.protocol).toEqual('http1');
});
it('return the correct value for http/1.1 requests', () => {
const request = hapiMocks.createRequest({
raw: {
req: {
httpVersion: '1.1',
},
},
});
const kibanaRequest = CoreKibanaRequest.from(request);
expect(kibanaRequest.protocol).toEqual('http1');
});
it('return the correct value for http/2 requests', () => {
const request = hapiMocks.createRequest({
raw: {
req: {
@ -227,7 +251,7 @@ describe('CoreKibanaRequest', () => {
});
const kibanaRequest = CoreKibanaRequest.from(request);
expect(kibanaRequest.protocol).toEqual('http1');
expect(kibanaRequest.protocol).toEqual('http2');
});
});

7
packages/core/http/core-http-router-server-internal/src/request.ts
View file

@ -173,8 +173,7 @@ export class CoreKibanaRequest<
});
this.httpVersion = isRealReq ? request.raw.req.httpVersion : '1.0';
// hardcoded for now as only supporting http1
this.protocol = 'http1';
this.protocol = getProtocolFromHttpVersion(this.httpVersion);
this.route = deepFreeze(this.getRouteInfo(request));
this.socket = isRealReq
@ -374,3 +373,7 @@ function sanitizeRequest(req: Request): { query: unknown; params: unknown; body:
body: req.payload,
};
}
function getProtocolFromHttpVersion(httpVersion: string): HttpProtocol {
return httpVersion.split('.')[0] === '2' ? 'http2' : 'http1';
}

9
packages/core/http/core-http-router-server-internal/src/router.ts
View file

@ -35,6 +35,7 @@ import { HapiResponseAdapter } from './response_adapter';
import { wrapErrors } from './error_wrapper';
import { Method } from './versioned_router/types';
import { prepareRouteConfigValidation } from './util';
import { stripIllegalHttp2Headers } from './strip_illegal_http2_headers';
export type ContextEnhancer<
P,
@ -265,6 +266,14 @@ export class Router<Context extends RequestHandlerContextBase = RequestHandlerCo
try {
const kibanaResponse = await handler(kibanaRequest, kibanaResponseFactory);
if (kibanaRequest.protocol === 'http2' && kibanaResponse.options.headers) {
kibanaResponse.options.headers = stripIllegalHttp2Headers({
headers: kibanaResponse.options.headers,
isDev: this.options.isDev ?? false,
logger: this.log,
requestContext: `${request.route.method} ${request.route.path}`,
});
}
return hapiResponseAdapter.handle(kibanaResponse);
} catch (error) {
// capture error

114
packages/core/http/core-http-router-server-internal/src/strip_illegal_http2_headers.test.ts Normal file
View file

@ -0,0 +1,114 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { stripIllegalHttp2Headers } from './strip_illegal_http2_headers';
import { loggerMock, MockedLogger } from '@kbn/logging-mocks';
describe('stripIllegalHttp2Headers', () => {
let logger: MockedLogger;
beforeEach(() => {
logger = loggerMock.create();
});
it('removes illegal http2 headers', () => {
const headers = {
'x-foo': 'bar',
'x-hello': 'dolly',
connection: 'keep-alive',
'proxy-connection': 'keep-alive',
'keep-alive': 'true',
upgrade: 'probably',
'transfer-encoding': 'chunked',
'http2-settings': 'yeah',
};
const output = stripIllegalHttp2Headers({
headers,
isDev: false,
logger,
requestContext: 'requestContext',
});
expect(output).toEqual({
'x-foo': 'bar',
'x-hello': 'dolly',
});
});
it('ignores case when detecting headers', () => {
const headers = {
'x-foo': 'bar',
'x-hello': 'dolly',
Connection: 'keep-alive',
'Proxy-Connection': 'keep-alive',
'kEeP-AlIvE': 'true',
};
const output = stripIllegalHttp2Headers({
headers,
isDev: false,
logger,
requestContext: 'requestContext',
});
expect(output).toEqual({
'x-foo': 'bar',
'x-hello': 'dolly',
});
});
it('logs a warning about the illegal header when in dev mode', () => {
const headers = {
'x-foo': 'bar',
Connection: 'keep-alive',
};
stripIllegalHttp2Headers({
headers,
isDev: true,
logger,
requestContext: 'requestContext',
});
expect(logger.warn).toHaveBeenCalledTimes(1);
expect(logger.warn).toHaveBeenCalledWith(
`Handler for "requestContext" returned an illegal http2 header: Connection. Please check "request.protocol" in handlers before assigning connection headers`
);
});
it('does not log a warning about the illegal header when not in dev mode', () => {
const headers = {
'x-foo': 'bar',
Connection: 'keep-alive',
};
stripIllegalHttp2Headers({
headers,
isDev: false,
logger,
requestContext: 'requestContext',
});
expect(logger.warn).not.toHaveBeenCalled();
});
it('does not mutate the original headers', () => {
const headers = {
'x-foo': 'bar',
Connection: 'keep-alive',
};
stripIllegalHttp2Headers({
headers,
isDev: true,
logger,
requestContext: 'requestContext',
});
expect(headers).toEqual({
'x-foo': 'bar',
Connection: 'keep-alive',
});
});
});

49
packages/core/http/core-http-router-server-internal/src/strip_illegal_http2_headers.ts Normal file
View file

@ -0,0 +1,49 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import type { Logger } from '@kbn/logging';
import type { ResponseHeaders } from '@kbn/core-http-server';
// from https://github.com/nodejs/node/blob/v22.2.0/lib/internal/http2/util.js#L557
const ILLEGAL_HTTP2_CONNECTION_HEADERS = new Set([
'connection',
'proxy-connection',
'keep-alive',
'upgrade',
'transfer-encoding',
'http2-settings',
]);
/**
* Return a new version of the provided headers, with all illegal http2 headers removed.
* If `isDev` is `true`, will also log a warning if such header is encountered.
*/
export const stripIllegalHttp2Headers = ({
headers,
isDev,
logger,
requestContext,
}: {
headers: ResponseHeaders;
isDev: boolean;
logger: Logger;
requestContext: string;
}): ResponseHeaders => {
return Object.entries(headers).reduce((output, [headerName, headerValue]) => {
if (ILLEGAL_HTTP2_CONNECTION_HEADERS.has(headerName.toLowerCase())) {
if (isDev) {
logger.warn(
`Handler for "${requestContext}" returned an illegal http2 header: ${headerName}. Please check "request.protocol" in handlers before assigning connection headers`
);
}
} else {
output[headerName as keyof ResponseHeaders] = headerValue;
}
return output;
}, {} as ResponseHeaders);
};

3
packages/core/http/core-http-router-server-internal/tsconfig.json
View file

@ -17,7 +17,8 @@
"@kbn/hapi-mocks",
"@kbn/core-logging-server-mocks",
"@kbn/logging",
"@kbn/core-http-common"
"@kbn/core-http-common",
"@kbn/logging-mocks"
],
"exclude": [
"target/**/*",

4
packages/core/http/core-http-server-internal/src/__snapshots__/http_config.test.ts.snap generated
View file

@ -70,6 +70,9 @@ Object {
},
},
"host": "localhost",
"http2": Object {
"allowUnsecure": false,
},
"keepaliveTimeout": 120000,
"maxPayload": ByteSizeValue {
"valueInBytes": 1048576,
@ -80,6 +83,7 @@ Object {
},
"payloadTimeout": 20000,
"port": 5601,
"protocol": "http1",
"requestId": Object {
"allowFromAnyIp": false,
"ipAllowlist": Array [],

67
packages/core/http/core-http-server-internal/src/http_config.test.ts
View file

@ -571,6 +571,73 @@ describe('cdn', () => {
});
});
describe('http2 protocol', () => {
it('throws if http2 is enabled but TLS is not', () => {
expect(() =>
config.schema.validate({
protocol: 'http2',
ssl: {
enabled: false,
},
})
).toThrowErrorMatchingInlineSnapshot(
`"http2 requires TLS to be enabled. Use 'http2.allowUnsecure: true' to allow running http2 without a valid h2c setup"`
);
});
it('throws if http2 is enabled but TLS has no suitable versions', () => {
expect(() =>
config.schema.validate({
protocol: 'http2',
ssl: {
enabled: true,
supportedProtocols: ['TLSv1.1'],
certificate: '/path/to/certificate',
key: '/path/to/key',
},
})
).toThrowErrorMatchingInlineSnapshot(
`"http2 requires 'ssl.supportedProtocols' to include TLSv1.2 or TLSv1.3. Use 'http2.allowUnsecure: true' to allow running http2 without a valid h2c setup"`
);
});
it('does not throws if http2 is enabled and TLS is not if http2.allowUnsecure is true', () => {
expect(
config.schema.validate({
protocol: 'http2',
http2: {
allowUnsecure: true,
},
ssl: {
enabled: false,
},
})
).toEqual(
expect.objectContaining({
protocol: 'http2',
})
);
});
it('does not throws if supportedProtocols are not valid for h2c if http2.allowUnsecure is true', () => {
expect(
config.schema.validate({
protocol: 'http2',
http2: {
allowUnsecure: true,
},
ssl: {
enabled: true,
supportedProtocols: ['TLSv1.1'],
certificate: '/path/to/certificate',
key: '/path/to/key',
},
})
).toEqual(
expect.objectContaining({
protocol: 'http2',
})
);
});
});
describe('HttpConfig', () => {
it('converts customResponseHeaders to strings or arrays of strings', () => {
const httpSchema = config.schema;

44
packages/core/http/core-http-server-internal/src/http_config.ts
View file

@ -6,23 +6,21 @@
* Side Public License, v 1.
*/
import { EOL, hostname } from 'node:os';
import url, { URL } from 'node:url';
import type { Duration } from 'moment';
import { ByteSizeValue, offeringBasedSchema, schema, TypeOf } from '@kbn/config-schema';
import { IHttpConfig, SslConfig, sslSchema } from '@kbn/server-http-tools';
import { IHttpConfig, SslConfig, sslSchema, TLS_V1_2, TLS_V1_3 } from '@kbn/server-http-tools';
import type { ServiceConfigDescriptor } from '@kbn/core-base-server-internal';
import { uuidRegexp } from '@kbn/core-base-server-internal';
import type { ICspConfig, IExternalUrlConfig } from '@kbn/core-http-server';
import { hostname, EOL } from 'node:os';
import url, { URL } from 'node:url';
import type { Duration } from 'moment';
import type { HttpProtocol, ICspConfig, IExternalUrlConfig } from '@kbn/core-http-server';
import type { IHttpEluMonitorConfig } from '@kbn/core-http-server/src/elu_monitor';
import type { HandlerResolutionStrategy } from '@kbn/core-http-router-server-internal';
import { CspConfigType, CspConfig } from './csp';
import { CspConfig, CspConfigType } from './csp';
import { ExternalUrlConfig } from './external_url';
import {
securityResponseHeadersSchema,
parseRawSecurityResponseHeadersConfig,
securityResponseHeadersSchema,
} from './security_response_headers_config';
import { CdnConfig } from './cdn_config';
@ -123,6 +121,9 @@ const configSchema = schema.object(
}
},
}),
protocol: schema.oneOf([schema.literal('http1'), schema.literal('http2')], {
defaultValue: 'http1',
}),
host: schema.string({
defaultValue: 'localhost',
hostname: true,
@ -144,6 +145,9 @@ const configSchema = schema.object(
payloadTimeout: schema.number({
defaultValue: 20 * SECOND,
}),
http2: schema.object({
allowUnsecure: schema.boolean({ defaultValue: false }),
}),
compression: schema.object({
enabled: schema.boolean({ defaultValue: true }),
brotli: schema.object({
@ -259,6 +263,13 @@ const configSchema = schema.object(
return 'cannot use [compression.referrerWhitelist] when [compression.enabled] is set to false';
}
if (rawConfig.protocol === 'http2' && !rawConfig.http2.allowUnsecure) {
const err = ensureValidTLSConfigForH2C(rawConfig.ssl);
if (err) {
return err;
}
}
if (
rawConfig.ssl.enabled &&
rawConfig.ssl.redirectHttpFromPort !== undefined &&
@ -285,6 +296,7 @@ export const config: ServiceConfigDescriptor<HttpConfigType> = {
export class HttpConfig implements IHttpConfig {
public name: string;
public autoListen: boolean;
public protocol: HttpProtocol;
public host: string;
public keepaliveTimeout: number;
public socketTimeout: number;
@ -352,6 +364,7 @@ export class HttpConfig implements IHttpConfig {
);
this.maxPayload = rawHttpConfig.maxPayload;
this.name = rawHttpConfig.name;
this.protocol = rawHttpConfig.protocol;
this.basePath = rawHttpConfig.basePath;
this.publicBaseUrl = rawHttpConfig.publicBaseUrl;
this.keepaliveTimeout = rawHttpConfig.keepaliveTimeout;
@ -378,3 +391,16 @@ export class HttpConfig implements IHttpConfig {
const convertHeader = (entry: any): string => {
return typeof entry === 'object' ? JSON.stringify(entry) : String(entry);
};
const ensureValidTLSConfigForH2C = (tlsConfig: TypeOf<typeof sslSchema>): string | undefined => {
if (!tlsConfig.enabled) {
return `http2 requires TLS to be enabled. Use 'http2.allowUnsecure: true' to allow running http2 without a valid h2c setup`;
}
if (
!tlsConfig.supportedProtocols.includes(TLS_V1_2) &&
!tlsConfig.supportedProtocols.includes(TLS_V1_3)
) {
return `http2 requires 'ssl.supportedProtocols' to include ${TLS_V1_2} or ${TLS_V1_3}. Use 'http2.allowUnsecure: true' to allow running http2 without a valid h2c setup`;
}
return undefined;
};

3
packages/core/http/core-http-server/src/http_contract.ts
View file

@ -408,5 +408,6 @@ export interface HttpServerInfo {
* (Only supporting http1 for now)
*
* - http1: regroups all http/1.x protocols
* - http2: h2
*/
export type HttpProtocol = 'http1';
export type HttpProtocol = 'http1' | 'http2';

20
packages/kbn-cli-dev-mode/src/base_path_proxy_server.ts → packages/kbn-cli-dev-mode/src/base_path_proxy/http1.ts
View file

@ -11,28 +11,18 @@ import { Agent as HttpsAgent, ServerOptions as TlsOptions } from 'https';
import apm from 'elastic-apm-node';
import { Server, Request } from '@hapi/hapi';
import HapiProxy from '@hapi/h2o2';
import { sampleSize } from 'lodash';
import * as Rx from 'rxjs';
import { take } from 'rxjs';
import { ByteSizeValue } from '@kbn/config-schema';
import { createServer, getServerOptions } from '@kbn/server-http-tools';
import { DevConfig, HttpConfig } from './config';
import { Log } from './log';
import { DevConfig, HttpConfig } from '../config';
import { Log } from '../log';
import { getRandomBasePath } from './utils';
import type { BasePathProxyServer, BasePathProxyServerOptions } from './types';
const ONE_GIGABYTE = 1024 * 1024 * 1024;
const alphabet = 'abcdefghijklmnopqrztuvwxyz'.split('');
// Thank you, Spencer! :elasticheart:
const getRandomBasePath = () =>
Math.random() * 100 < 1 ? 'spalger' : sampleSize(alphabet, 3).join('');
export interface BasePathProxyServerOptions {
shouldRedirectFromOldBasePath: (path: string) => boolean;
delayUntil: () => Rx.Observable<void>;
}
export class BasePathProxyServer {
export class Http1BasePathProxyServer implements BasePathProxyServer {
private readonly httpConfig: HttpConfig;
private server?: Server;
private httpsAgent?: HttpsAgent;

183
packages/kbn-cli-dev-mode/src/base_path_proxy/http2.ts Normal file
View file

@ -0,0 +1,183 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import Url from 'url';
import { Agent as HttpsAgent, ServerOptions as TlsOptions } from 'https';
import http2, { Agent as Http2Agent, AutoRequestOptions } from 'http2-wrapper';
import http2Proxy from 'http2-proxy';
import { take } from 'rxjs';
import { getServerOptions, getServerTLSOptions } from '@kbn/server-http-tools';
import { DevConfig, HttpConfig } from '../config';
import { Log } from '../log';
import type { BasePathProxyServer, BasePathProxyServerOptions } from './types';
import { getRandomBasePath } from './utils';
export class Http2BasePathProxyServer implements BasePathProxyServer {
private readonly httpConfig: HttpConfig;
private server?: http2.Http2SecureServer;
private httpsAgent?: HttpsAgent;
constructor(
private readonly log: Log,
httpConfig: HttpConfig,
private readonly devConfig: DevConfig
) {
this.httpConfig = {
...httpConfig,
basePath: httpConfig.basePath ?? `/${getRandomBasePath()}`,
};
}
public get basePath() {
return this.httpConfig.basePath;
}
public get targetPort() {
return this.devConfig.basePathProxyTargetPort;
}
public get host() {
return this.httpConfig.host;
}
public get port() {
return this.httpConfig.port;
}
public async start(options: BasePathProxyServerOptions) {
const serverOptions = getServerOptions(this.httpConfig);
if (this.httpConfig.ssl.enabled) {
const tlsOptions = serverOptions.tls as TlsOptions;
this.httpsAgent = new HttpsAgent({
ca: tlsOptions.ca,
cert: tlsOptions.cert,
key: tlsOptions.key,
passphrase: tlsOptions.passphrase,
rejectUnauthorized: false,
});
}
await this.setupServer(options);
this.log.write(
`basepath proxy server running at ${Url.format({
protocol: this.httpConfig.ssl.enabled ? 'https' : 'http',
host: this.httpConfig.host,
pathname: this.httpConfig.basePath,
})}`
);
}
public async stop() {
if (this.server !== undefined) {
await new Promise<void>((resolve, reject) => {
this.server!.close((err) => {
if (err) {
reject(err);
} else {
resolve();
}
});
});
this.server = undefined;
}
if (this.httpsAgent !== undefined) {
this.httpsAgent.destroy();
this.httpsAgent = undefined;
}
}
private async setupServer({ delayUntil }: Readonly<BasePathProxyServerOptions>) {
const tlsOptions = getServerTLSOptions(this.httpConfig.ssl);
this.server = http2.createSecureServer({
...tlsOptions,
rejectUnauthorized: false,
allowHTTP1: true,
});
const server = this.server;
const http2Agent = new Http2Agent();
server.on('error', (e) => {
this.log.bad('error', `error initializing the base path server: ${e.message}`);
throw e;
});
server.listen(this.httpConfig.port, this.httpConfig.host, () => {
server.on('request', (inboundRequest, inboundResponse) => {
const requestPath = Url.parse(inboundRequest.url).path ?? '/';
if (requestPath === '/') {
// Always redirect from root URL to the URL with basepath.
inboundResponse.writeHead(302, {
location: this.httpConfig.basePath,
});
inboundResponse.end();
} else if (requestPath.startsWith(this.httpConfig.basePath!)) {
// Perform proxy request if requested path is within base path
http2Proxy.web(
inboundRequest,
inboundResponse,
{
protocol: 'https',
hostname: this.httpConfig.host,
port: this.devConfig.basePathProxyTargetPort,
onReq: async (request, options) => {
// Before we proxy request to a target port we may want to wait until some
// condition is met (e.g. until target listener is ready).
await delayUntil().pipe(take(1)).toPromise();
const proxyOptions = {
...options,
...tlsOptions,
rejectUnauthorized: false,
path: options.path,
agent: {
https: this.httpsAgent ?? false,
http2: http2Agent,
},
} as AutoRequestOptions;
const proxyReq = await http2.auto(proxyOptions, (proxyRes) => {
// `http2-proxy` doesn't automatically remove pseudo-headers
for (const name in proxyRes.headers) {
if (name.startsWith(':')) {
delete proxyRes.headers[name];
}
}
});
// `http2-proxy` waits for the `socket` event before calling `h2request.end()`
proxyReq.flushHeaders();
return proxyReq;
},
onRes: async (request, response, _proxyRes) => {
// wrong type - proxyRes is declared as Http.ServerResponse but is Http.IncomingMessage
const proxyRes = _proxyRes as unknown as http2.IncomingMessage;
response.setHeader('x-powered-by', 'kibana-base-path-server');
response.writeHead(proxyRes.statusCode!, proxyRes.headers);
proxyRes.pipe(response);
},
},
(err, req, res) => {
if (err) {
this.log.bad('warning', 'base path proxy: error forwarding request', err);
res.statusCode = (err as any).statusCode || 500;
res.end((err as any).stack ?? err.message);
}
}
);
}
});
});
}
}

33
packages/kbn-cli-dev-mode/src/base_path_proxy/index.ts Normal file
View file

@ -0,0 +1,33 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import type { Log } from '../log';
import type { CliDevConfig } from '../config';
import type { BasePathProxyServer } from './types';
import { Http1BasePathProxyServer } from './http1';
import { Http2BasePathProxyServer } from './http2';
export type { BasePathProxyServer, BasePathProxyServerOptions } from './types';
export { Http1BasePathProxyServer } from './http1';
export { Http2BasePathProxyServer } from './http2';
export const getBasePathProxyServer = ({
log,
httpConfig,
devConfig,
}: {
log: Log;
httpConfig: CliDevConfig['http'];
devConfig: CliDevConfig['dev'];
}): BasePathProxyServer => {
if (httpConfig.protocol === 'http2') {
return new Http2BasePathProxyServer(log, httpConfig, devConfig);
} else {
return new Http1BasePathProxyServer(log, httpConfig, devConfig);
}
};

24
packages/kbn-cli-dev-mode/src/base_path_proxy/types.ts Normal file
View file

@ -0,0 +1,24 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import type { Observable } from 'rxjs';
export interface BasePathProxyServer {
readonly basePath: string | undefined;
readonly targetPort: number;
readonly host: string;
readonly port: number;
start(options: BasePathProxyServerOptions): Promise<void>;
stop(): Promise<void>;
}
export interface BasePathProxyServerOptions {
shouldRedirectFromOldBasePath: (path: string) => boolean;
delayUntil: () => Observable<void>;
}

15
packages/kbn-cli-dev-mode/src/base_path_proxy/utils.ts Normal file
View file

@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { sampleSize } from 'lodash';
const alphabet = 'abcdefghijklmnopqrztuvwxyz'.split('');
// Thank you, Spencer! :elasticheart:
export const getRandomBasePath = () =>
Math.random() * 100 < 1 ? 'spalger' : sampleSize(alphabet, 3).join('');

20
packages/kbn-cli-dev-mode/src/cli_dev_mode.test.ts
View file

@ -29,8 +29,8 @@ const { Optimizer } = jest.requireMock('./optimizer');
jest.mock('./dev_server');
const { DevServer } = jest.requireMock('./dev_server');
jest.mock('./base_path_proxy_server');
const { BasePathProxyServer } = jest.requireMock('./base_path_proxy_server');
jest.mock('./base_path_proxy');
const { getBasePathProxyServer } = jest.requireMock('./base_path_proxy');
jest.mock('@kbn/ci-stats-reporter');
const { CiStatsReporter } = jest.requireMock('@kbn/ci-stats-reporter');
@ -47,7 +47,7 @@ let log: TestLog;
beforeEach(() => {
process.argv = ['node', './script', 'foo', 'bar', 'baz'];
log = new TestLog();
BasePathProxyServer.mockImplementation(() => mockBasePathProxy);
getBasePathProxyServer.mockImplementation(() => mockBasePathProxy);
});
afterEach(() => {
@ -142,7 +142,7 @@ it('passes correct args to sub-classes', () => {
]
`);
expect(BasePathProxyServer).not.toHaveBeenCalled();
expect(getBasePathProxyServer).not.toHaveBeenCalled();
expect(log.messages).toMatchInlineSnapshot(`Array []`);
});
@ -163,13 +163,15 @@ it('disables the watcher', () => {
it('enables the basePath proxy', () => {
new CliDevMode(createOptions({ cliArgs: { basePath: true } }));
expect(BasePathProxyServer).toHaveBeenCalledTimes(1);
expect(BasePathProxyServer.mock.calls[0]).toMatchInlineSnapshot(`
expect(getBasePathProxyServer).toHaveBeenCalledTimes(1);
expect(getBasePathProxyServer.mock.calls[0]).toMatchInlineSnapshot(`
Array [
<TestLog>,
Object {},
Object {
"basePathProxyTargetPort": 9000,
"devConfig": Object {
"basePathProxyTargetPort": 9000,
},
"httpConfig": Object {},
"log": <TestLog>,
},
]
`);

8
packages/kbn-cli-dev-mode/src/cli_dev_mode.ts
View file

@ -29,7 +29,7 @@ import { Log, CliLog } from './log';
import { Optimizer } from './optimizer';
import { DevServer } from './dev_server';
import { Watcher } from './watcher';
import { BasePathProxyServer } from './base_path_proxy_server';
import { getBasePathProxyServer, type BasePathProxyServer } from './base_path_proxy';
import { shouldRedirectFromOldBasePath } from './should_redirect_from_old_base_path';
import { CliDevConfig } from './config';
@ -110,7 +110,11 @@ export class CliDevMode {
this.log = log || new CliLog(!!cliArgs.silent);
if (cliArgs.basePath) {
this.basePathProxy = new BasePathProxyServer(this.log, config.http, config.dev);
this.basePathProxy = getBasePathProxyServer({
log: this.log,
devConfig: config.dev,
httpConfig: config.http,
});
}
this.watcher = new Watcher({

5
packages/kbn-cli-dev-mode/src/config/http_config.ts
View file

@ -12,6 +12,9 @@ import { Duration } from 'moment';
export const httpConfigSchema = schema.object(
{
protocol: schema.oneOf([schema.literal('http1'), schema.literal('http2')], {
defaultValue: 'http1',
}),
host: schema.string({
defaultValue: 'localhost',
hostname: true,
@ -49,6 +52,7 @@ export const httpConfigSchema = schema.object(
export type HttpConfigType = TypeOf<typeof httpConfigSchema>;
export class HttpConfig implements IHttpConfig {
protocol: 'http1' | 'http2';
basePath?: string;
host: string;
port: number;
@ -62,6 +66,7 @@ export class HttpConfig implements IHttpConfig {
restrictInternalApis: boolean;
constructor(rawConfig: HttpConfigType) {
this.protocol = rawConfig.protocol;
this.basePath = rawConfig.basePath;
this.host = rawConfig.host;
this.port = rawConfig.port;

21
packages/kbn-cli-dev-mode/src/integration_tests/base_path_proxy_server.test.ts → packages/kbn-cli-dev-mode/src/integration_tests/http1_base_path_proxy_server.test.ts
View file

@ -13,13 +13,13 @@ import supertest from 'supertest';
import { getServerOptions, createServer, type IHttpConfig } from '@kbn/server-http-tools';
import { ByteSizeValue } from '@kbn/config-schema';
import { BasePathProxyServer, BasePathProxyServerOptions } from '../base_path_proxy_server';
import { Http1BasePathProxyServer, BasePathProxyServerOptions } from '../base_path_proxy';
import { DevConfig } from '../config/dev_config';
import { TestLog } from '../log';
describe('BasePathProxyServer', () => {
describe('Http1BasePathProxyServer', () => {
let server: Server;
let proxyServer: BasePathProxyServer;
let proxyServer: Http1BasePathProxyServer;
let logger: TestLog;
let config: IHttpConfig;
let basePath: string;
@ -29,6 +29,7 @@ describe('BasePathProxyServer', () => {
logger = new TestLog();
config = {
protocol: 'http1',
host: '127.0.0.1',
port: 10012,
shutdownTimeout: moment.duration(30, 'seconds'),
@ -51,7 +52,7 @@ describe('BasePathProxyServer', () => {
// setup and start the proxy server
const proxyConfig: IHttpConfig = { ...config, port: 10013 };
const devConfig = new DevConfig({ basePathProxyTarget: config.port });
proxyServer = new BasePathProxyServer(logger, proxyConfig, devConfig);
proxyServer = new Http1BasePathProxyServer(logger, proxyConfig, devConfig);
const options: BasePathProxyServerOptions = {
shouldRedirectFromOldBasePath: () => true,
delayUntil: () => EMPTY,
@ -322,14 +323,18 @@ describe('BasePathProxyServer', () => {
});
describe('shouldRedirect', () => {
let proxyServerWithoutShouldRedirect: BasePathProxyServer;
let proxyServerWithoutShouldRedirect: Http1BasePathProxyServer;
let proxyWithoutShouldRedirectSupertest: supertest.Agent;
beforeEach(async () => {
// setup and start a proxy server which does not use "shouldRedirectFromOldBasePath"
const proxyConfig: IHttpConfig = { ...config, port: 10004 };
const devConfig = new DevConfig({ basePathProxyTarget: config.port });
proxyServerWithoutShouldRedirect = new BasePathProxyServer(logger, proxyConfig, devConfig);
proxyServerWithoutShouldRedirect = new Http1BasePathProxyServer(
logger,
proxyConfig,
devConfig
);
const options: Readonly<BasePathProxyServerOptions> = {
shouldRedirectFromOldBasePath: () => false, // Return false to not redirect
delayUntil: () => EMPTY,
@ -365,14 +370,14 @@ describe('BasePathProxyServer', () => {
});
describe('constructor option for sending in a custom basePath', () => {
let proxyServerWithFooBasePath: BasePathProxyServer;
let proxyServerWithFooBasePath: Http1BasePathProxyServer;
let proxyWithFooBasePath: supertest.Agent;
beforeEach(async () => {
// setup and start a proxy server which uses a basePath of "foo"
const proxyConfig = { ...config, port: 10004, basePath: '/foo' }; // <-- "foo" here in basePath
const devConfig = new DevConfig({ basePathProxyTarget: config.port });
proxyServerWithFooBasePath = new BasePathProxyServer(logger, proxyConfig, devConfig);
proxyServerWithFooBasePath = new Http1BasePathProxyServer(logger, proxyConfig, devConfig);
const options: Readonly<BasePathProxyServerOptions> = {
shouldRedirectFromOldBasePath: () => true,
delayUntil: () => EMPTY,

254
packages/kbn-cli-dev-mode/src/integration_tests/http2_base_path_proxy_server.test.ts Normal file
View file

@ -0,0 +1,254 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { readFileSync } from 'fs';
import { Server } from '@hapi/hapi';
import { EMPTY } from 'rxjs';
import moment from 'moment';
import supertest from 'supertest';
import { KBN_CERT_PATH, KBN_KEY_PATH } from '@kbn/dev-utils';
import { getServerOptions, createServer, type IHttpConfig } from '@kbn/server-http-tools';
import { ByteSizeValue } from '@kbn/config-schema';
import { Http2BasePathProxyServer, BasePathProxyServerOptions } from '../base_path_proxy';
import { DevConfig } from '../config/dev_config';
import { TestLog } from '../log';
describe('Http2BasePathProxyServer', () => {
let server: Server;
let proxyServer: Http2BasePathProxyServer;
let logger: TestLog;
let config: IHttpConfig;
let basePath: string;
let proxySupertest: supertest.Agent;
beforeAll(() => {
// required for the self-signed certificates used in testing
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
});
beforeEach(async () => {
logger = new TestLog();
config = {
protocol: 'http2',
host: '127.0.0.1',
port: 10012,
shutdownTimeout: moment.duration(30, 'seconds'),
keepaliveTimeout: 1000,
socketTimeout: 1000,
payloadTimeout: 1000,
cors: {
enabled: false,
allowCredentials: false,
allowOrigin: [],
},
ssl: {
enabled: true,
certificate: await readFileSync(KBN_CERT_PATH, 'utf-8'),
key: await readFileSync(KBN_KEY_PATH, 'utf-8'),
cipherSuites: ['TLS_AES_256_GCM_SHA384'],
},
maxPayload: new ByteSizeValue(1024),
restrictInternalApis: false,
};
const serverOptions = getServerOptions(config);
server = createServer(serverOptions);
// setup and start the proxy server
const proxyConfig: IHttpConfig = { ...config, port: 10013 };
const devConfig = new DevConfig({ basePathProxyTarget: config.port });
proxyServer = new Http2BasePathProxyServer(logger, proxyConfig, devConfig);
const options: BasePathProxyServerOptions = {
shouldRedirectFromOldBasePath: () => true,
delayUntil: () => EMPTY,
};
await proxyServer.start(options);
// set the base path or throw if for some unknown reason it is not setup
if (proxyServer.basePath == null) {
throw new Error('Invalid null base path, all tests will fail');
} else {
basePath = proxyServer.basePath;
}
proxySupertest = supertest(`https://127.0.0.1:${proxyConfig.port}`, { http2: true });
});
afterEach(async () => {
await server.stop();
await proxyServer.stop();
jest.clearAllMocks();
});
test('root URL will return a 302 redirect', async () => {
await proxySupertest.get('/').expect(302);
});
test('root URL will return a redirect location with exactly 3 characters that are a-z (or spalger)', async () => {
const res = await proxySupertest.get('/');
const location = res.header.location;
expect(location).toMatch(/^\/(spalger|[a-z]{3})$/);
});
test('forwards request with the correct path', async () => {
server.route({
method: 'GET',
path: `${basePath}/foo/{test}`,
handler: (request, h) => {
return h.response(request.params.test);
},
});
await server.start();
await proxySupertest
.get(`${basePath}/foo/some-string`)
.expect(200)
.then((res) => {
expect(res.text).toBe('some-string');
});
});
test('can serve http/1.x requests', async () => {
server.route({
method: 'GET',
path: `${basePath}/foo/{test}`,
handler: (request, h) => {
return h.response(request.params.test);
},
});
await server.start();
await proxySupertest
.get(`${basePath}/foo/some-string`)
.http2(false)
.expect(200)
.then((res) => {
expect(res.text).toBe('some-string');
});
});
test('forwards request with the correct query params', async () => {
server.route({
method: 'GET',
path: `${basePath}/foo/`,
handler: (request, h) => {
return h.response(request.query);
},
});
await server.start();
await proxySupertest
.get(`${basePath}/foo/?bar=test&quux=123`)
.expect(200)
.then((res) => {
expect(res.body).toEqual({ bar: 'test', quux: '123' });
});
});
test('forwards the request body', async () => {
server.route({
method: 'POST',
path: `${basePath}/foo/`,
handler: (request, h) => {
return h.response(request.payload);
},
});
await server.start();
await proxySupertest
.post(`${basePath}/foo/`)
.send({
bar: 'test',
baz: 123,
})
.expect(200)
.then((res) => {
expect(res.body).toEqual({ bar: 'test', baz: 123 });
});
});
test('returns the correct status code', async () => {
server.route({
method: 'GET',
path: `${basePath}/foo/`,
handler: (request, h) => {
return h.response({ foo: 'bar' }).code(417);
},
});
await server.start();
await proxySupertest
.get(`${basePath}/foo/`)
.expect(417)
.then((res) => {
expect(res.body).toEqual({ foo: 'bar' });
});
});
test('returns the response headers', async () => {
server.route({
method: 'GET',
path: `${basePath}/foo/`,
handler: (request, h) => {
return h.response({ foo: 'bar' }).header('foo', 'bar');
},
});
await server.start();
await proxySupertest
.get(`${basePath}/foo/`)
.expect(200)
.then((res) => {
expect(res.get('foo')).toEqual('bar');
});
});
test('forwards request cancellation', async () => {
let propagated = false;
let notifyRequestReceived: () => void;
const requestReceived = new Promise<void>((resolve) => {
notifyRequestReceived = resolve;
});
let notifyRequestAborted: () => void;
const requestAborted = new Promise<void>((resolve) => {
notifyRequestAborted = resolve;
});
server.route({
method: 'GET',
path: `${basePath}/foo/{test}`,
handler: async (request, h) => {
notifyRequestReceived();
request.raw.req.once('aborted', () => {
notifyRequestAborted();
propagated = true;
});
return await new Promise((resolve) => undefined);
},
});
await server.start();
const request = proxySupertest.get(`${basePath}/foo/some-string`).end();
await requestReceived;
request.abort();
await requestAborted;
expect(propagated).toEqual(true);
});
});

1
packages/kbn-cli-dev-mode/tsconfig.json
View file

@ -25,6 +25,7 @@
"@kbn/import-resolver",
"@kbn/picomatcher",
"@kbn/repo-packages",
"@kbn/dev-utils",
],
"exclude": [
"target/**/*",

1
packages/kbn-health-gateway-server/src/server/server_config.ts
View file

@ -70,6 +70,7 @@ export const config: ServiceConfigDescriptor<ServerConfigType> = {
};
export class ServerConfig implements IHttpConfig {
readonly protocol = 'http1';
host: string;
port: number;
maxPayload: ByteSizeValue;

2
packages/kbn-journeys/journey/journey_ftr_harness.ts
View file

@ -151,7 +151,7 @@ export class JourneyFtrHarness {
private async setupBrowserAndPage() {
const browser = await this.getBrowserInstance();
const browserContextArgs = this.auth.isCloud() ? {} : { bypassCSP: true };
this.context = await browser.newContext(browserContextArgs);
this.context = await browser.newContext({ ...browserContextArgs, ignoreHTTPSErrors: true });
if (this.journeyConfig.shouldAutoLogin()) {
const cookie = await this.auth.login();

10
packages/kbn-server-http-tools/index.ts
View file

@ -6,7 +6,13 @@
* Side Public License, v 1.
*/
export type { IHttpConfig, ISslConfig, ICorsConfig } from './src/types';
export type {
IHttpConfig,
ISslConfig,
ICorsConfig,
ServerProtocol,
ServerListener,
} from './src/types';
export { createServer } from './src/create_server';
export { defaultValidationErrorHandler } from './src/default_validation_error_handler';
export { getServerListener } from './src/get_listener';
@ -14,4 +20,4 @@ export { getServerOptions } from './src/get_server_options';
export { getServerTLSOptions } from './src/get_tls_options';
export { getRequestId } from './src/get_request_id';
export { setTlsConfig } from './src/set_tls_config';
export { sslSchema, SslConfig } from './src/ssl';
export { sslSchema, SslConfig, TLS_V1, TLS_V1_1, TLS_V1_2, TLS_V1_3 } from './src/ssl';

23
packages/kbn-server-http-tools/src/get_listener.test.mocks.ts
View file

@ -45,3 +45,26 @@ jest.doMock('https', () => {
createServer: createHttpsServerMock,
};
});
export const createHttp2SecureServerMock = jest.fn(() => {
return {
on: jest.fn(),
setTimeout: jest.fn(),
};
});
export const createHttp2UnsecureServerMock = jest.fn(() => {
return {
on: jest.fn(),
setTimeout: jest.fn(),
};
});
jest.doMock('http2', () => {
const actual = jest.requireActual('https');
return {
...actual,
createServer: createHttp2UnsecureServerMock,
createSecureServer: createHttp2SecureServerMock,
};
});

211
packages/kbn-server-http-tools/src/get_listener.test.ts
View file

@ -10,6 +10,8 @@ import {
getServerTLSOptionsMock,
createHttpServerMock,
createHttpsServerMock,
createHttp2UnsecureServerMock,
createHttp2SecureServerMock,
} from './get_listener.test.mocks';
import moment from 'moment';
import { ByteSizeValue } from '@kbn/config-schema';
@ -18,6 +20,7 @@ import { getServerListener } from './get_listener';
const createConfig = (parts: Partial<IHttpConfig>): IHttpConfig => ({
host: 'localhost',
protocol: 'http1',
port: 5601,
socketTimeout: 120000,
keepaliveTimeout: 120000,
@ -41,99 +44,195 @@ const createConfig = (parts: Partial<IHttpConfig>): IHttpConfig => ({
describe('getServerListener', () => {
beforeEach(() => {
getServerTLSOptionsMock.mockReset();
createHttpServerMock.mockClear();
createHttpsServerMock.mockClear();
createHttp2UnsecureServerMock.mockClear();
createHttp2SecureServerMock.mockClear();
});
describe('when TLS is enabled', () => {
it('calls getServerTLSOptions with the correct parameters', () => {
const config = createConfig({ ssl: { enabled: true } });
describe('When protocol is `http1`', () => {
describe('when TLS is enabled', () => {
it('calls getServerTLSOptions with the correct parameters', () => {
const config = createConfig({ ssl: { enabled: true } });
getServerListener(config);
getServerListener(config);
expect(getServerTLSOptionsMock).toHaveBeenCalledTimes(1);
expect(getServerTLSOptionsMock).toHaveBeenCalledWith(config.ssl);
});
expect(getServerTLSOptionsMock).toHaveBeenCalledTimes(1);
expect(getServerTLSOptionsMock).toHaveBeenCalledWith(config.ssl);
});
it('calls https.createServer with the correct parameters', () => {
const config = createConfig({ ssl: { enabled: true } });
it('calls https.createServer with the correct parameters', () => {
const config = createConfig({ ssl: { enabled: true } });
getServerTLSOptionsMock.mockReturnValue({ stub: true });
getServerTLSOptionsMock.mockReturnValue({ stub: true });
getServerListener(config);
getServerListener(config);
expect(createHttpsServerMock).toHaveBeenCalledTimes(1);
expect(createHttpsServerMock).toHaveBeenCalledWith({
stub: true,
keepAliveTimeout: config.keepaliveTimeout,
expect(createHttpsServerMock).toHaveBeenCalledTimes(1);
expect(createHttpsServerMock).toHaveBeenCalledWith({
stub: true,
keepAliveTimeout: config.keepaliveTimeout,
});
});
it('properly configures the listener', () => {
const config = createConfig({ ssl: { enabled: true } });
const server = getServerListener(config);
expect(server.setTimeout).toHaveBeenCalledTimes(1);
expect(server.setTimeout).toHaveBeenCalledWith(config.socketTimeout);
expect(server.on).toHaveBeenCalledTimes(2);
expect(server.on).toHaveBeenCalledWith('clientError', expect.any(Function));
expect(server.on).toHaveBeenCalledWith('timeout', expect.any(Function));
});
it('returns the https server', () => {
const config = createConfig({ ssl: { enabled: true } });
const server = getServerListener(config);
const expectedServer = createHttpsServerMock.mock.results[0].value;
expect(server).toBe(expectedServer);
});
});
it('properly configures the listener', () => {
const config = createConfig({ ssl: { enabled: true } });
const server = getServerListener(config);
describe('when TLS is disabled', () => {
it('does not call getServerTLSOptions', () => {
const config = createConfig({ ssl: { enabled: false } });
expect(server.setTimeout).toHaveBeenCalledTimes(1);
expect(server.setTimeout).toHaveBeenCalledWith(config.socketTimeout);
getServerListener(config);
expect(server.on).toHaveBeenCalledTimes(2);
expect(server.on).toHaveBeenCalledWith('clientError', expect.any(Function));
expect(server.on).toHaveBeenCalledWith('timeout', expect.any(Function));
});
expect(getServerTLSOptionsMock).not.toHaveBeenCalled();
});
it('returns the https server', () => {
const config = createConfig({ ssl: { enabled: true } });
it('calls http.createServer with the correct parameters', () => {
const config = createConfig({ ssl: { enabled: false } });
const server = getServerListener(config);
getServerTLSOptionsMock.mockReturnValue({ stub: true });
const expectedServer = createHttpsServerMock.mock.results[0].value;
getServerListener(config);
expect(server).toBe(expectedServer);
expect(createHttpServerMock).toHaveBeenCalledTimes(1);
expect(createHttpServerMock).toHaveBeenCalledWith({
keepAliveTimeout: config.keepaliveTimeout,
});
});
it('properly configures the listener', () => {
const config = createConfig({ ssl: { enabled: false } });
const server = getServerListener(config);
expect(server.setTimeout).toHaveBeenCalledTimes(1);
expect(server.setTimeout).toHaveBeenCalledWith(config.socketTimeout);
expect(server.on).toHaveBeenCalledTimes(2);
expect(server.on).toHaveBeenCalledWith('clientError', expect.any(Function));
expect(server.on).toHaveBeenCalledWith('timeout', expect.any(Function));
});
it('returns the http server', () => {
const config = createConfig({ ssl: { enabled: false } });
const server = getServerListener(config);
const expectedServer = createHttpServerMock.mock.results[0].value;
expect(server).toBe(expectedServer);
});
});
});
describe('when TLS is disabled', () => {
it('does not call getServerTLSOptions', () => {
const config = createConfig({ ssl: { enabled: false } });
describe('When protocol is `http2`', () => {
const createHttp2Config = (parts: Partial<IHttpConfig>) =>
createConfig({ ...parts, protocol: 'http2' });
getServerListener(config);
describe('when TLS is enabled', () => {
it('calls getServerTLSOptions with the correct parameters', () => {
const config = createHttp2Config({ ssl: { enabled: true } });
expect(getServerTLSOptionsMock).not.toHaveBeenCalled();
});
getServerListener(config);
it('calls http.createServer with the correct parameters', () => {
const config = createConfig({ ssl: { enabled: false } });
expect(getServerTLSOptionsMock).toHaveBeenCalledTimes(1);
expect(getServerTLSOptionsMock).toHaveBeenCalledWith(config.ssl);
});
getServerTLSOptionsMock.mockReturnValue({ stub: true });
it('calls http2.createSecureServer with the correct parameters', () => {
const config = createHttp2Config({ ssl: { enabled: true } });
getServerListener(config);
getServerTLSOptionsMock.mockReturnValue({ stub: true });
expect(createHttpServerMock).toHaveBeenCalledTimes(1);
expect(createHttpServerMock).toHaveBeenCalledWith({
keepAliveTimeout: config.keepaliveTimeout,
getServerListener(config);
expect(createHttp2SecureServerMock).toHaveBeenCalledTimes(1);
expect(createHttp2SecureServerMock).toHaveBeenCalledWith({
stub: true,
allowHTTP1: true,
});
});
it('properly configures the listener', () => {
const config = createHttp2Config({ ssl: { enabled: true } });
const server = getServerListener(config);
expect(server.setTimeout).toHaveBeenCalledTimes(1);
expect(server.setTimeout).toHaveBeenCalledWith(config.socketTimeout);
expect(server.on).not.toHaveBeenCalled();
});
it('returns the http2 secure server', () => {
const config = createHttp2Config({ ssl: { enabled: true } });
const server = getServerListener(config);
const expectedServer = createHttp2SecureServerMock.mock.results[0].value;
expect(server).toBe(expectedServer);
});
});
it('properly configures the listener', () => {
const config = createConfig({ ssl: { enabled: false } });
const server = getServerListener(config);
describe('when TLS is disabled', () => {
it('does not call getServerTLSOptions', () => {
const config = createHttp2Config({ ssl: { enabled: false } });
expect(server.setTimeout).toHaveBeenCalledTimes(1);
expect(server.setTimeout).toHaveBeenCalledWith(config.socketTimeout);
getServerListener(config);
expect(server.on).toHaveBeenCalledTimes(2);
expect(server.on).toHaveBeenCalledWith('clientError', expect.any(Function));
expect(server.on).toHaveBeenCalledWith('timeout', expect.any(Function));
});
expect(getServerTLSOptionsMock).not.toHaveBeenCalled();
});
it('returns the http server', () => {
const config = createConfig({ ssl: { enabled: false } });
it('calls http2.createServer with the correct parameters', () => {
const config = createHttp2Config({ ssl: { enabled: false } });
const server = getServerListener(config);
getServerTLSOptionsMock.mockReturnValue({ stub: true });
const expectedServer = createHttpServerMock.mock.results[0].value;
getServerListener(config);
expect(server).toBe(expectedServer);
expect(createHttp2UnsecureServerMock).toHaveBeenCalledTimes(1);
expect(createHttp2UnsecureServerMock).toHaveBeenCalledWith({});
});
it('properly configures the listener', () => {
const config = createHttp2Config({ ssl: { enabled: false } });
const server = getServerListener(config);
expect(server.setTimeout).toHaveBeenCalledTimes(1);
expect(server.setTimeout).toHaveBeenCalledWith(config.socketTimeout);
expect(server.on).not.toHaveBeenCalled();
});
it('returns the http2 unsecure server', () => {
const config = createHttp2Config({ ssl: { enabled: false } });
const server = getServerListener(config);
const expectedServer = createHttp2UnsecureServerMock.mock.results[0].value;
expect(server).toBe(expectedServer);
});
});
});
});

26
packages/kbn-server-http-tools/src/get_listener.ts
View file

@ -8,6 +8,7 @@
import http from 'http';
import https from 'https';
import http2 from 'http2';
import { getServerTLSOptions } from './get_tls_options';
import type { IHttpConfig, ServerListener } from './types';
@ -19,7 +20,10 @@ export function getServerListener(
config: IHttpConfig,
options: GetServerListenerOptions = {}
): ServerListener {
return configureHttp1Listener(config, options);
const useHTTP2 = config.protocol === 'http2';
return useHTTP2
? configureHttp2Listener(config, options)
: configureHttp1Listener(config, options);
}
const configureHttp1Listener = (
@ -52,3 +56,23 @@ const configureHttp1Listener = (
return listener;
};
const configureHttp2Listener = (
config: IHttpConfig,
{ configureTLS = true }: GetServerListenerOptions = {}
): ServerListener => {
const useTLS = configureTLS && config.ssl.enabled;
const tlsOptions = useTLS ? getServerTLSOptions(config.ssl) : undefined;
const listener = useTLS
? http2.createSecureServer({
...tlsOptions,
// allow ALPN negotiation fallback to HTTP/1.x
allowHTTP1: true,
})
: http2.createServer({});
listener.setTimeout(config.socketTimeout);
return listener;
};

1
packages/kbn-server-http-tools/src/get_server_options.test.ts
View file

@ -23,6 +23,7 @@ jest.mock('fs', () => {
const createConfig = (parts: Partial<IHttpConfig>): IHttpConfig => ({
host: 'localhost',
protocol: 'http1',
port: 5601,
socketTimeout: 120000,
keepaliveTimeout: 120000,

2
packages/kbn-server-http-tools/src/get_server_options.ts
View file

@ -24,10 +24,12 @@ export function getServerOptions(config: IHttpConfig, { configureTLS = true } =
headers: corsAllowedHeaders,
}
: false;
const options: ServerOptions = {
host: config.host,
port: config.port,
// manually configuring the listener
// @ts-expect-error HAPI types only define http1/https listener, not http2
listener: getServerListener(config, { configureTLS }),
// must set to true when manually passing a TLS listener, false otherwise
tls: configureTLS && config.ssl.enabled,

1
packages/kbn-server-http-tools/src/get_tls_options.test.ts
View file

@ -22,6 +22,7 @@ jest.mock('fs', () => {
const createConfig = (parts: Partial<IHttpConfig>): IHttpConfig => ({
host: 'localhost',
protocol: 'http1',
port: 5601,
socketTimeout: 120000,
keepaliveTimeout: 120000,

12
packages/kbn-server-http-tools/src/ssl/constants.ts Normal file
View file

@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export const TLS_V1 = 'TLSv1';
export const TLS_V1_1 = 'TLSv1.1';
export const TLS_V1_2 = 'TLSv1.2';
export const TLS_V1_3 = 'TLSv1.3';

1
packages/kbn-server-http-tools/src/ssl/index.ts
View file

@ -7,3 +7,4 @@
*/
export { SslConfig, sslSchema } from './ssl_config';
export { TLS_V1, TLS_V1_1, TLS_V1_2, TLS_V1_3 } from './constants';

19
packages/kbn-server-http-tools/src/ssl/ssl_config.ts
View file

@ -11,13 +11,14 @@ import { schema, TypeOf } from '@kbn/config-schema';
import { readPkcs12Keystore, readPkcs12Truststore } from '@kbn/crypto';
import { constants as cryptoConstants } from 'crypto';
import { readFileSync } from 'fs';
import { TLS_V1, TLS_V1_1, TLS_V1_2, TLS_V1_3 } from './constants';
const protocolMap = new Map<string, number>([
['TLSv1', cryptoConstants.SSL_OP_NO_TLSv1],
['TLSv1.1', cryptoConstants.SSL_OP_NO_TLSv1_1],
['TLSv1.2', cryptoConstants.SSL_OP_NO_TLSv1_2],
[TLS_V1, cryptoConstants.SSL_OP_NO_TLSv1],
[TLS_V1_1, cryptoConstants.SSL_OP_NO_TLSv1_1],
[TLS_V1_2, cryptoConstants.SSL_OP_NO_TLSv1_2],
// @ts-expect-error According to the docs SSL_OP_NO_TLSv1_3 should exist (https://nodejs.org/docs/latest-v12.x/api/crypto.html)
['TLSv1.3', cryptoConstants.SSL_OP_NO_TLSv1_3],
[TLS_V1_3, cryptoConstants.SSL_OP_NO_TLSv1_3],
]);
export const sslSchema = schema.object(
@ -45,12 +46,12 @@ export const sslSchema = schema.object(
redirectHttpFromPort: schema.maybe(schema.number()),
supportedProtocols: schema.arrayOf(
schema.oneOf([
schema.literal('TLSv1'),
schema.literal('TLSv1.1'),
schema.literal('TLSv1.2'),
schema.literal('TLSv1.3'),
schema.literal(TLS_V1),
schema.literal(TLS_V1_1),
schema.literal(TLS_V1_2),
schema.literal(TLS_V1_3),
]),
{ defaultValue: ['TLSv1.1', 'TLSv1.2', 'TLSv1.3'], minSize: 1 }
{ defaultValue: [TLS_V1_1, TLS_V1_2, TLS_V1_3], minSize: 1 }
),
clientAuthentication: schema.oneOf(
[schema.literal('none'), schema.literal('optional'), schema.literal('required')],

8
packages/kbn-server-http-tools/src/types.ts
View file

@ -8,8 +8,9 @@
import type { Server as HttpServer } from 'http';
import type { Server as HttpsServer } from 'https';
import { ByteSizeValue } from '@kbn/config-schema';
import type { Http2SecureServer, Http2Server } from 'http2';
import type { Duration } from 'moment';
import { ByteSizeValue } from '@kbn/config-schema';
/**
* Composite type of all possible kind of Listener types.
@ -17,9 +18,12 @@ import type { Duration } from 'moment';
* Unfortunately, there's no real common interface between all those concrete classes,
* as `net.Server` and `tls.Server` don't list all the APIs we're using (such as event binding)
*/
export type ServerListener = HttpServer | HttpsServer;
export type ServerListener = Http2Server | Http2SecureServer | HttpServer | HttpsServer;
export type ServerProtocol = 'http1' | 'http2';
export interface IHttpConfig {
protocol: ServerProtocol;
host: string;
port: number;
maxPayload: ByteSizeValue;

1
packages/kbn-test/src/kbn_client/kbn_client_requester.ts
View file

@ -96,6 +96,7 @@ export class KbnClientRequester {
Url.parse(options.url).protocol === 'https:'
? new Https.Agent({
ca: options.certificateAuthorities,
rejectUnauthorized: false,
})
: null;
}

69
src/cli/serve/serve.js
View file

@ -101,6 +101,12 @@ export function applyConfigOverrides(rawConfig, opts, extraCliOptions, keystoreC
const get = _.partial(_.get, rawConfig);
const has = _.partial(_.has, rawConfig);
function ensureNotDefined(path, command = '--ssl') {
if (has(path)) {
throw new Error(`Can't use ${command} when "${path}" configuration is already defined.`);
}
}
if (opts.oss) {
delete rawConfig.xpack;
}
@ -152,49 +158,59 @@ export function applyConfigOverrides(rawConfig, opts, extraCliOptions, keystoreC
}
}
if (opts.ssl) {
if (opts.http2) {
set('server.protocol', 'http2');
}
// HTTP TLS configuration
if (opts.ssl || opts.http2) {
// @kbn/dev-utils is part of devDependencies
// eslint-disable-next-line import/no-extraneous-dependencies
const { CA_CERT_PATH, KBN_KEY_PATH, KBN_CERT_PATH } = require('@kbn/dev-utils');
const customElasticsearchHosts = opts.elasticsearch
? opts.elasticsearch.split(',')
: [].concat(get('elasticsearch.hosts') || []);
function ensureNotDefined(path) {
if (has(path)) {
throw new Error(`Can't use --ssl when "${path}" configuration is already defined.`);
}
}
ensureNotDefined('server.ssl.certificate');
ensureNotDefined('server.ssl.key');
ensureNotDefined('server.ssl.keystore.path');
ensureNotDefined('server.ssl.truststore.path');
ensureNotDefined('server.ssl.certificateAuthorities');
ensureNotDefined('elasticsearch.ssl.certificateAuthorities');
const elasticsearchHosts = (
(customElasticsearchHosts.length > 0 && customElasticsearchHosts) || [
'https://localhost:9200',
]
).map((hostUrl) => {
const parsedUrl = url.parse(hostUrl);
if (parsedUrl.hostname !== 'localhost') {
throw new Error(
`Hostname "${parsedUrl.hostname}" can't be used with --ssl. Must be "localhost" to work with certificates.`
);
}
return `https://localhost:${parsedUrl.port}`;
});
set('server.ssl.enabled', true);
set('server.ssl.certificate', KBN_CERT_PATH);
set('server.ssl.key', KBN_KEY_PATH);
set('server.ssl.certificateAuthorities', CA_CERT_PATH);
set('elasticsearch.hosts', elasticsearchHosts);
set('elasticsearch.ssl.certificateAuthorities', CA_CERT_PATH);
}
}
// Kib/ES encryption
if (opts.ssl) {
// @kbn/dev-utils is part of devDependencies
// eslint-disable-next-line import/no-extraneous-dependencies
const { CA_CERT_PATH } = require('@kbn/dev-utils');
const customElasticsearchHosts = opts.elasticsearch
? opts.elasticsearch.split(',')
: [].concat(get('elasticsearch.hosts') || []);
ensureNotDefined('elasticsearch.ssl.certificateAuthorities');
const elasticsearchHosts = (
(customElasticsearchHosts.length > 0 && customElasticsearchHosts) || [
'https://localhost:9200',
]
).map((hostUrl) => {
const parsedUrl = url.parse(hostUrl);
if (parsedUrl.hostname !== 'localhost') {
throw new Error(
`Hostname "${parsedUrl.hostname}" can't be used with --ssl. Must be "localhost" to work with certificates.`
);
}
return `https://localhost:${parsedUrl.port}`;
});
set('elasticsearch.hosts', elasticsearchHosts);
set('elasticsearch.ssl.certificateAuthorities', CA_CERT_PATH);
}
if (opts.elasticsearch) set('elasticsearch.hosts', opts.elasticsearch.split(','));
if (opts.port) set('server.port', opts.port);
if (opts.host) set('server.host', opts.host);
@ -262,6 +278,7 @@ export default function (program) {
command
.option('--dev', 'Run the server with development mode defaults')
.option('--ssl', 'Run the dev server using HTTPS')
.option('--http2', 'Run the dev server using HTTP2 with TLS')
.option('--dist', 'Use production assets from kbn/optimizer')
.option(
'--no-base-path',

1
src/core/server/index.ts
View file

@ -232,6 +232,7 @@ export type {
HttpServiceStart,
RawRequest,
FakeRawRequest,
HttpProtocol,
} from '@kbn/core-http-server';
export type { IExternalUrlPolicy } from '@kbn/core-http-common';

214
src/core/server/integration_tests/http/http2_protocol.test.ts Normal file
View file

@ -0,0 +1,214 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { Server } from 'http';
import supertest from 'supertest';
import { of } from 'rxjs';
import { KBN_CERT_PATH, KBN_KEY_PATH } from '@kbn/dev-utils';
import { Router } from '@kbn/core-http-router-server-internal';
import {
HttpServer,
HttpConfig,
config as httpConfig,
cspConfig,
externalUrlConfig,
} from '@kbn/core-http-server-internal';
import { mockCoreContext } from '@kbn/core-base-server-mocks';
import type { Logger } from '@kbn/logging';
const CSP_CONFIG = cspConfig.schema.validate({});
const EXTERNAL_URL_CONFIG = externalUrlConfig.schema.validate({});
describe('Http2 - Smoke tests', () => {
let server: HttpServer;
let config: HttpConfig;
let logger: Logger;
let coreContext: ReturnType<typeof mockCoreContext.create>;
let innerServerListener: Server;
const enhanceWithContext = (fn: (...args: any[]) => any) => fn.bind(null, {});
beforeAll(() => {
// required for the self-signed certificates used in testing
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
});
beforeEach(() => {
coreContext = mockCoreContext.create();
logger = coreContext.logger.get();
const rawConfig = httpConfig.schema.validate({
name: 'kibana',
protocol: 'http2',
host: '127.0.0.1',
port: 10002,
ssl: {
enabled: true,
certificate: KBN_CERT_PATH,
key: KBN_KEY_PATH,
cipherSuites: ['TLS_AES_256_GCM_SHA384'],
redirectHttpFromPort: 10003,
},
shutdownTimeout: '5s',
});
config = new HttpConfig(rawConfig, CSP_CONFIG, EXTERNAL_URL_CONFIG);
server = new HttpServer(coreContext, 'tests', of(config.shutdownTimeout));
});
afterEach(async () => {
await server?.stop();
});
describe('Basic tests against all supported methods', () => {
beforeEach(async () => {
const { registerRouter, server: innerServer } = await server.setup({ config$: of(config) });
innerServerListener = innerServer.listener;
const router = new Router('', logger, enhanceWithContext, {
isDev: false,
versionedRouterOptions: {
defaultHandlerResolutionStrategy: 'oldest',
},
});
router.post({ path: '/', validate: false }, async (context, req, res) => {
return res.ok({
body: { protocol: req.protocol, httpVersion: req.httpVersion },
});
});
router.get({ path: '/', validate: false }, async (context, req, res) => {
return res.ok({
body: { protocol: req.protocol, httpVersion: req.httpVersion },
});
});
router.put({ path: '/', validate: false }, async (context, req, res) => {
return res.ok({
body: { protocol: req.protocol, httpVersion: req.httpVersion },
});
});
router.delete({ path: '/', validate: false }, async (context, req, res) => {
return res.ok({
body: { protocol: req.protocol, httpVersion: req.httpVersion },
});
});
registerRouter(router);
await server.start();
});
describe('POST', () => {
it('should respond to POST endpoint for an HTTP/2 request', async () => {
const response = await supertest(innerServerListener).post('/').http2();
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http2', httpVersion: '2.0' });
});
it('should respond to POST endpoint for an HTTP/1.x request', async () => {
const response = await supertest(innerServerListener).post('/');
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http1', httpVersion: '1.1' });
});
});
describe('GET', () => {
it('should respond to GET endpoint for an HTTP/2 request', async () => {
const response = await supertest(innerServerListener).get('/').http2();
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http2', httpVersion: '2.0' });
});
it('should respond to GET endpoint for an HTTP/1.x request', async () => {
const response = await supertest(innerServerListener).get('/');
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http1', httpVersion: '1.1' });
});
});
describe('DELETE', () => {
it('should respond to DELETE endpoint for an HTTP/2 request', async () => {
const response = await supertest(innerServerListener).delete('/').http2();
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http2', httpVersion: '2.0' });
});
it('should respond to DELETE endpoint for an HTTP/1.x request', async () => {
const response = await supertest(innerServerListener).delete('/');
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http1', httpVersion: '1.1' });
});
});
describe('PUT', () => {
it('should respond to PUT endpoint for an HTTP/2 request', async () => {
const response = await supertest(innerServerListener).put('/').http2();
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http2', httpVersion: '2.0' });
});
it('should respond to PUT endpoint for an HTTP/1.x request', async () => {
const response = await supertest(innerServerListener).put('/');
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http1', httpVersion: '1.1' });
});
});
});
describe('HTTP2-specific behaviors', () => {
beforeEach(async () => {
const { registerRouter, server: innerServer } = await server.setup({ config$: of(config) });
innerServerListener = innerServer.listener;
const router = new Router('', logger, enhanceWithContext, {
isDev: false,
versionedRouterOptions: {
defaultHandlerResolutionStrategy: 'oldest',
},
});
router.get({ path: '/illegal_headers', validate: false }, async (context, req, res) => {
return res.ok({
headers: {
connection: 'close',
},
body: { protocol: req.protocol },
});
});
registerRouter(router);
await server.start();
});
describe('illegal http2 headers', () => {
it('should strip illegal http2 headers without causing errors when serving HTTP/2', async () => {
const response = await supertest(innerServerListener).get('/illegal_headers').http2();
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http2' });
expect(response.header).toEqual(expect.not.objectContaining({ connection: 'close' }));
});
it('should keep illegal http2 headers when serving HTTP/1.x', async () => {
const response = await supertest(innerServerListener).get('/illegal_headers');
expect(response.status).toBe(200);
expect(response.body).toEqual({ protocol: 'http1' });
expect(response.header).toEqual(expect.objectContaining({ connection: 'close' }));
});
});
});
});

1
src/core/server/integration_tests/http/set_tls_config.test.ts
View file

@ -74,6 +74,7 @@ describe('setTlsConfig', () => {
name: 'kibana',
host: '127.0.0.1',
port: 10002,
protocol: 'http1',
ssl: {
enabled: true,
certificate: ES_CERT_PATH,

2
src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker
View file

@ -155,11 +155,13 @@ kibana_vars=(
server.customResponseHeaders
server.defaultRoute
server.host
server.http2.allowUnsecure
server.keepAliveTimeout
server.maxPayload
server.maxPayloadBytes
server.name
server.port
server.protocol
server.publicBaseUrl
server.requestId.allowFromAnyIp
server.requestId.ipAllowlist

21
src/plugins/bfetch/server/plugin.ts
View file

@ -18,6 +18,7 @@ import {
RequestHandler,
KibanaResponseFactory,
AnalyticsServiceStart,
HttpProtocol,
} from '@kbn/core/server';
import { map$ } from '@kbn/std';
@ -65,11 +66,19 @@ export interface BfetchServerSetup {
// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface BfetchServerStart {}
const streamingHeaders = {
'Content-Type': 'application/x-ndjson',
Connection: 'keep-alive',
'Transfer-Encoding': 'chunked',
'X-Accel-Buffering': 'no',
const getStreamingHeaders = (protocol: HttpProtocol): Record<string, string> => {
if (protocol === 'http2') {
return {
'Content-Type': 'application/x-ndjson',
'X-Accel-Buffering': 'no',
};
}
return {
'Content-Type': 'application/x-ndjson',
Connection: 'keep-alive',
'Transfer-Encoding': 'chunked',
'X-Accel-Buffering': 'no',
};
};
interface Query {
@ -144,7 +153,7 @@ export class BfetchServerPlugin
const data = request.body;
const compress = request.query.compress;
return response.ok({
headers: streamingHeaders,
headers: getStreamingHeaders(request.protocol),
body: createStream(
handlerInstance.getResponseStream(data),
logger,

37
src/setup_node_env/exit_on_warning.js
View file

@ -53,6 +53,28 @@ var IGNORE_WARNINGS = [
message:
'The URL https://github.com:crypto-browserify/browserify-rsa.git is invalid. Future versions of Node.js will throw an error.',
},
// supertest in HTTP2 mode uses 0.0.0.0 as the server's name
{
name: 'DeprecationWarning',
code: 'DEP0123',
message:
'Setting the TLS ServerName to an IP address is not permitted by RFC 6066. This will be ignored in a future version.',
},
{
// emitted whenever a header not supported by http2 is set. it's not actionable for the end user.
// HAPI sets a connection: close header - see https://github.com/hapijs/hapi/issues/3830
name: 'UnsupportedWarning',
messageContains:
'header is not valid, the value will be dropped from the header and will never be in use.',
},
// We have to enabled NODE_TLS_REJECT_UNAUTHORIZED for FTR testing
// when http2 is enabled to accept dev self-signed certificates
{
ftrOnly: true,
name: 'Warning',
message:
"Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.",
},
];
if (process.noProcessWarnings !== true) {
@ -68,7 +90,6 @@ if (process.noProcessWarnings !== true) {
console.error();
console.error('Terminating process...');
}
process.exit(1);
});
@ -87,10 +108,22 @@ if (process.noProcessWarnings !== true) {
function shouldIgnore(warn) {
warn = parseWarn(warn);
return IGNORE_WARNINGS.some(function ({ name, code, message, file, line, col }) {
return IGNORE_WARNINGS.some(function ({
name,
code,
message,
messageContains,
file,
line,
col,
ftrOnly,
}) {
if (ftrOnly && !process.env.IS_FTR_RUNNER) return false;
if (name && name !== warn.name) return false;
if (code && code !== warn.code) return false;
if (message && message !== warn.message) return false;
if (messageContains && !warn.message.includes(messageContains)) return false;
if (file && !warn.frames[0].file.endsWith(file)) return false;
if (line && line !== warn.frames[0].line) return false;
if (col && col !== warn.frames[0].col) return false;

20
test/api_integration/services/supertest.ts
View file

@ -10,13 +10,27 @@ import { systemIndicesSuperuser } from '@kbn/test';
import { format as formatUrl } from 'url';
import supertest from 'supertest';
import supertest, { AgentOptions } from 'supertest';
import { FtrProviderContext } from '../../functional/ftr_provider_context';
export function KibanaSupertestProvider({ getService }: FtrProviderContext): supertest.Agent {
const config = getService('config');
const kibanaServerUrl = formatUrl(config.get('servers.kibana'));
return supertest(kibanaServerUrl);
const kibanaServerConfig = config.get('servers.kibana');
const kibanaServerUrl = formatUrl(kibanaServerConfig);
const options: AgentOptions = {};
if (kibanaServerConfig.certificateAuthorities) {
options.ca = kibanaServerConfig.certificateAuthorities;
options.rejectUnauthorized = false;
}
const serverArgs = config.get('kbnTestServer.serverArgs', []) as string[];
const http2Enabled = serverArgs.includes('--server.protocol=http2');
if (http2Enabled) {
options.http2 = true;
}
return supertest(kibanaServerUrl, options);
}
export function ElasticsearchSupertestProvider({

85
test/common/configure_http2.ts Normal file
View file

@ -0,0 +1,85 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { readFileSync } from 'fs';
import { CA_CERT_PATH, KBN_CERT_PATH, KBN_KEY_PATH } from '@kbn/dev-utils';
type ConfigType = Record<string, any>;
/**
* Enables HTTP2 by adding/changing the appropriate config settings
*
* Important: this must be used on "final" (non-reused) configs, otherwise
* the overrides from the children configs could remove the overrides
* done in that helper.
*/
export const configureHTTP2 = (config: ConfigType): ConfigType => {
// Add env flag to avoid terminating on NODE_TLS_REJECT_UNAUTHORIZED warning
process.env.IS_FTR_RUNNER = 'true';
// tell native node agents to trust unsafe certificates
// this is ugly, but unfortunately required, as some libraries (such as supertest)
// have no real alternatives to accept self-signed certs
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';
// tell webdriver browser to accept self-signed certificates
config.browser.acceptInsecureCerts = true;
// change the configured kibana server to run on https with the dev CA
config.servers.kibana = {
...config.servers.kibana,
protocol: 'https',
certificateAuthorities: [readFileSync(CA_CERT_PATH, 'utf-8')],
};
const serverArgs = config.kbnTestServer.serverArgs;
// enable http2 on the kibana server
addOrReplaceKbnServerArg(serverArgs, 'server.protocol', () => 'http2');
// enable and configure TLS on the kibana server
addOrReplaceKbnServerArg(serverArgs, 'server.ssl.enabled', () => 'true');
addOrReplaceKbnServerArg(serverArgs, 'server.ssl.key', () => KBN_KEY_PATH);
addOrReplaceKbnServerArg(serverArgs, 'server.ssl.certificate', () => KBN_CERT_PATH);
addOrReplaceKbnServerArg(serverArgs, 'server.ssl.certificateAuthorities', () => CA_CERT_PATH);
// replace the newsfeed test plugin url to use https
addOrReplaceKbnServerArg(serverArgs, 'newsfeed.service.urlRoot', (oldValue) => {
if (!oldValue || !oldValue.includes(config.servers.kibana.hostname)) {
return undefined;
}
return oldValue.replaceAll('http', 'https');
});
return config;
};
/**
* Set or replace given `arg` in the provided serverArgs list, using the provided replacer function
*/
const addOrReplaceKbnServerArg = (
serverArgs: string[],
argName: string,
replacer: (value: string | undefined) => string | undefined
) => {
const argPrefix = `--${argName}=`;
const argIndex = serverArgs.findIndex((value) => value.startsWith(argPrefix));
if (argIndex === -1) {
const newArgValue = replacer(undefined);
if (newArgValue !== undefined) {
serverArgs.push(`${argPrefix}${newArgValue}`);
}
} else {
const currentArgValue = serverArgs[argIndex].substring(argPrefix.length);
const newArgValue = replacer(currentArgValue);
if (newArgValue !== undefined) {
serverArgs[argIndex] = `${argPrefix}${newArgValue}`;
} else {
serverArgs.splice(argIndex, 1);
}
}
};

12
test/common/services/deployment.ts
View file

@ -7,6 +7,7 @@
*/
import { get } from 'lodash';
import { Agent } from 'https';
import fetch from 'node-fetch';
import { getUrl } from '@kbn/test';
@ -33,12 +34,23 @@ export class DeploymentService extends FtrService {
const baseUrl = this.getHostPort();
const username = this.config.get('servers.kibana.username');
const password = this.config.get('servers.kibana.password');
const protocol = this.config.get('servers.kibana.protocol');
let agent: Agent | undefined;
if (protocol === 'https') {
agent = new Agent({
// required for self-signed certificates used for HTTPS FTR testing
rejectUnauthorized: false,
});
}
const response = await fetch(baseUrl + '/api/stats?extended', {
method: 'get',
headers: {
'Content-Type': 'application/json',
Authorization: 'Basic ' + Buffer.from(username + ':' + password).toString('base64'),
},
agent,
});
const data = await response.json();
return get(data, 'usage.cloud.is_cloud_enabled', false);

5
test/functional/apps/console/config.ts
View file

@ -7,12 +7,13 @@
*/
import { FtrConfigProviderContext } from '@kbn/test';
import { configureHTTP2 } from '../../../common/configure_http2';
export default async function ({ readConfigFile }: FtrConfigProviderContext) {
const functionalConfig = await readConfigFile(require.resolve('../../config.base.js'));
return {
return configureHTTP2({
...functionalConfig.getAll(),
testFiles: [require.resolve('.')],
};
});
}

5
test/functional/apps/home/config.ts
View file

@ -7,12 +7,13 @@
*/
import { FtrConfigProviderContext } from '@kbn/test';
import { configureHTTP2 } from '../../../common/configure_http2';
export default async function ({ readConfigFile }: FtrConfigProviderContext) {
const functionalConfig = await readConfigFile(require.resolve('../../config.base.js'));
return {
return configureHTTP2({
...functionalConfig.getAll(),
testFiles: [require.resolve('.')],
};
});
}

20
test/functional/services/supertest.ts
View file

@ -8,11 +8,25 @@
import { format as formatUrl } from 'url';
import supertest from 'supertest';
import supertest, { AgentOptions } from 'supertest';
import { FtrProviderContext } from '../ftr_provider_context';
export function KibanaSupertestProvider({ getService }: FtrProviderContext) {
const config = getService('config');
const kibanaServerUrl = formatUrl(config.get('servers.kibana'));
return supertest(kibanaServerUrl);
const kibanaServerConfig = config.get('servers.kibana');
const kibanaServerUrl = formatUrl(kibanaServerConfig);
const options: AgentOptions = {};
if (kibanaServerConfig.certificateAuthorities) {
options.ca = kibanaServerConfig.certificateAuthorities;
options.rejectUnauthorized = false;
}
const serverArgs = config.get('kbnTestServer.serverArgs', []) as string[];
const http2Enabled = serverArgs.includes('--server.protocol=http2');
if (http2Enabled) {
options.http2 = true;
}
return supertest(kibanaServerUrl, options);
}

11
x-pack/test/common/services/spaces.ts
View file

@ -7,6 +7,7 @@
import type { Space } from '@kbn/spaces-plugin/common';
import Axios from 'axios';
import Https from 'https';
import { format as formatUrl } from 'url';
import util from 'util';
import { FtrProviderContext } from '../ftr_provider_context';
@ -16,11 +17,21 @@ export function SpacesServiceProvider({ getService }: FtrProviderContext) {
const config = getService('config');
const url = formatUrl(config.get('servers.kibana'));
const certificateAuthorities = config.get('servers.kibana.certificateAuthorities');
const httpsAgent: Https.Agent | undefined = certificateAuthorities
? new Https.Agent({
ca: certificateAuthorities,
// required for self-signed certificates used for HTTPS FTR testing
rejectUnauthorized: false,
})
: undefined;
const axios = Axios.create({
headers: { 'kbn-xsrf': 'x-pack/ftr/services/spaces/space' },
baseURL: url,
maxRedirects: 0,
validateStatus: () => true, // we do our own validation below and throw better error messages
httpsAgent,
});
return new (class SpacesService {

38
yarn.lock
View file

@ -19258,6 +19258,11 @@ http2-client@^1.2.5:
resolved "https://registry.yarnpkg.com/http2-client/-/http2-client-1.3.5.tgz#20c9dc909e3cc98284dd20af2432c524086df181"
integrity sha512-EC2utToWl4RKfs5zd36Mxq7nzHHBuomZboI0yYL6Y0RmBgT7Sgkq4rQ0ezFTYoIsSs7Tm9SJe+o2FcAg6GBhGA==
http2-proxy@^5.0.53:
version "5.0.53"
resolved "https://registry.yarnpkg.com/http2-proxy/-/http2-proxy-5.0.53.tgz#fc6cb07d2bb977a388ebeec4449557f2011e5a1f"
integrity sha512-k9OUKrPWau/YeViJGv5peEFgSGPE2n8CDyk/G3f+JfaaJzbFMPAK5PJTd99QYSUvgUwVBGNbZJCY/BEb+kUZNQ==
http2-wrapper@^1.0.0-beta.5.2:
version "1.0.0-beta.5.2"
resolved "https://registry.yarnpkg.com/http2-wrapper/-/http2-wrapper-1.0.0-beta.5.2.tgz#8b923deb90144aea65cf834b016a340fc98556f3"
@ -19266,7 +19271,7 @@ http2-wrapper@^1.0.0-beta.5.2:
quick-lru "^5.1.1"
resolve-alpn "^1.0.0"
http2-wrapper@^2.1.10:
http2-wrapper@^2.1.10, http2-wrapper@^2.2.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/http2-wrapper/-/http2-wrapper-2.2.1.tgz#310968153dcdedb160d8b72114363ef5fce1f64a"
integrity sha512-V5nVw1PAOgfI3Lmeaj2Exmeg7fenjhRUgz1lPSezy1CuhPYbgQtbQj4jZfEAEMlaL+vupsvhjqCyjzob0yxsmQ==
@ -28874,7 +28879,7 @@ string-replace-loader@^2.2.0:
loader-utils "^1.2.3"
schema-utils "^1.0.0"
"string-width-cjs@npm:string-width@^4.2.0", "string-width@^1.0.2 || 2 || 3 || 4", string-width@^4.0.0, string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.2, string-width@^4.2.3:
"string-width-cjs@npm:string-width@^4.2.0":
version "4.2.3"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
@ -28892,6 +28897,15 @@ string-width@^1.0.1:
is-fullwidth-code-point "^1.0.0"
strip-ansi "^3.0.0"
"string-width@^1.0.2 || 2 || 3 || 4", string-width@^4.0.0, string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.2, string-width@^4.2.3:
version "4.2.3"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
dependencies:
emoji-regex "^8.0.0"
is-fullwidth-code-point "^3.0.0"
strip-ansi "^6.0.1"
string-width@^5.0.1, string-width@^5.1.2:
version "5.1.2"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-5.1.2.tgz#14f8daec6d81e7221d2a357e668cab73bdbca794"
@ -29001,7 +29015,7 @@ stringify-object@^3.2.1:
is-obj "^1.0.1"
is-regexp "^1.0.0"
"strip-ansi-cjs@npm:strip-ansi@^6.0.1", strip-ansi@^6.0.0, strip-ansi@^6.0.1:
"strip-ansi-cjs@npm:strip-ansi@^6.0.1":
version "6.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
@ -29015,6 +29029,13 @@ strip-ansi@^3.0.0, strip-ansi@^3.0.1:
dependencies:
ansi-regex "^2.0.0"
strip-ansi@^6.0.0, strip-ansi@^6.0.1:
version "6.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
dependencies:
ansi-regex "^5.0.1"
strip-ansi@^7.0.1, strip-ansi@^7.1.0:
version "7.1.0"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-7.1.0.tgz#d5b6568ca689d8561370b0707685d22434faff45"
@ -31882,7 +31903,7 @@ workerpool@6.2.1:
resolved "https://registry.yarnpkg.com/workerpool/-/workerpool-6.2.1.tgz#46fc150c17d826b86a008e5a4508656777e9c343"
integrity sha512-ILEIE97kDZvF9Wb9f6h5aXK4swSlKGUcOEGiIYb2OOu/IrDU9iwj0fD//SsA6E5ibwJxpEvhullJY4Sl4GcpAw==
"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0", wrap-ansi@^7.0.0:
"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0":
version "7.0.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
@ -31908,6 +31929,15 @@ wrap-ansi@^6.2.0:
string-width "^4.1.0"
strip-ansi "^6.0.0"
wrap-ansi@^7.0.0:
version "7.0.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
dependencies:
ansi-styles "^4.0.0"
string-width "^4.1.0"
strip-ansi "^6.0.0"
wrap-ansi@^8.1.0:
version "8.1.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-8.1.0.tgz#56dc22368ee570face1b49819975d9b9a5ead214"