kibana/packages/kbn-server-http-tools
Pierre Gayvallet dea26c6450
Add http2 support for Kibana server (#183465)
## Summary

Part of https://github.com/elastic/kibana/issues/7104

Add support for `http2` to the Kibana server. `http2` can be enabled by
setting `server.protocol: http2` in the Kibana config file.

*Note: by default, enabling `http2` requires a valid `h2c`
configuration, meaning that it can only run over HTTPS with TLS1.2+*

```yaml
## kibana.yaml
server.protocol: http2
server.ssl.enabled: true
server.ssl.key: path/to/key
server.ssl.certificate: path/my/cerf
```

## What is this PR doing

### Add HTTP2 support for the Kibana server

#### - Plug http2 to the Kibana server 

Even if HAPI was never officially updated to really support HTTP2,
node's `http`/`https`/`http2` modules are compatible enough to be able
to just instantiate an http2 server/listener and provide it to HAPI "as
a plain https listener". There were some tweaks to do (mostly silencing
a few warnings that HAPI was causing by sending http2-illegal headers
such as `Connection`), but overall, it went smoothly.

#### - Add config validation

By default, Kibana will require a valid `h2c` configuration to accept
enabling `http2`. It means that TLS must be enabled and that TLS1.2+
should at least be in the list of supported SSL protocols
(`server.ssl.supportedProtocols`). Note that default value of this
setting includes TLS1.2 and 1.3.

#### - Add escape hatch to run `h2` without `h2c`

In some situations, it may be required to enable http2 without a valid
`h2c` configuration. Kibana supports it, by setting
`server.http2.allowUnsecure` to `true`.

(*Note, however, that if http2 is enabled without TLS, ALPN protocol
negotiation won't work, meaning that most http2 agents/clients will fail
connecting unless they're explictly configured to use http2.*)

### Add documentation about this new feature

#### - Update the user-facing doc about this new `server.protocol`
setting

Update the user-facing Kibana settings documentation to include this
`http.protocol` setting (and refer to `server.http2.allowUnsecure`)

**Note: this setting, and this feature, are considered as experimental**

### Adapt our dev tooling to support running Kibana with http2 enabled

#### - Add a `--http2` flag to the dev CLI

Enabling this flag will add the proper configuration settings to run
Kibana with `http2` enabled in an (almost) valid `h2c` configutation.

*Note: when using this flag, even if listening on the same port, the
Kibana server will be accessible over https, meaning that you need to
use https in your browser to access it. Aka `http://localhost:5601`
won't work, you need to use `https://localhost:5601`. Also, we're using
the self-signed dev certificates, meaning that you must go though the
scary warning of your browser*

#### - Implement an http2-compatible base-path proxy

The current base path proxy is based on `hapi` and `hapi/h2o2`. I tried
for a bunch hours trying to hack around to make it work with http2
proxying, but ultimately gave up and implemented a new version from
scratch.

Note that with some additional efforts, this new http2 basepath proxy
could probably fully replace the existing one and be used for both http1
and http2 traffic, but it's an optimization / refactoring that did not
feel required for this PR.

### Adapt the FTR to run suites against http2

#### - Add support to run FTR test suite against an h2c-enabled Kibana

Note that with ALPN, clients using http1 should be (and are) able to
communicate with http2 Kibana, given h2c/alpn allows protocol
negitiation. So adapting our FTR tooling was not really about making it
work with http2 (which worked out of the box), but making it work with
**the self signed certifcates we use for https on dev mode**

Note that I'm not a big fan of what I had to do, however, realistically
this was the only possible approach if we want to run arbitrary test
suites with TLS/HTTP2 enabled without massively changing our FTR setup.

Operations and QA, feel free to chime in there, as this is your
territory.

#### - Change some FTR test suites to run against an HTTP2-enabled
server

I added a quick `configureHTTP2` helper function to take any "final" FTR
suite config and mutate it to enable `http2`. I then enabled it on a few
suites locally, to make sure the suites were passing correctly.

I kept two suites running with http2 enabled:
- the `console` oss functional tests
- the `home` oss functional tests

We could possibly enable it for more, but we need to figure out what
kind of strategy we want on that matter (see below)

## What is this pull request NOT doing

#### - Making sure everything works when HTTP2 is enabled

I navigated the applications quite a bit, and did not see anything
broken, however I obviously wasn't able to do a full coverage. Also, the
self-signed certificate was a huge pain to detect issues really caused
by http2 compared to issues because the local setup isn't valid `h2c`.

In theory though (famous last words) anything not doing http/1.1
specific hacks such as bfetch should work fine with http2, given that
even if using non-http2 clients, ALPN should just allow to fallback to
http/1.x (this part was tested)

#### - Enabling HTTP2 by default

PR isn't doing it for obvious reasons. 

#### - Enabling HTTP2 for all FTR suites

First of all, it's not that easy, because it requires adapting various
parts of the config (and even some var env...), and we don't have any
proper way to override config "at the end". For instance, if you add the
http2 config on a top level config (e.g. the oss functional one that is
reuse by the whole world - learned the hard way), it won't work because
higher-level configs redefined (and override) the `browser` part of the
config, loosing the settings added to run the browser in insecure mode.

Secondly, I'm not sure we really need to run that many suites with http2
enabled. I learned working on that PR that we only have like one suite
where https is enabled for the Kibana server, and I feel like it could
be fine to have the same for http2. In theory it's just a protocol
change, unless parts of our apps (e.g. bfetch) are doing things that are
specific to http/1.1, switching to http2 should be an implementation
detail.

But I'd love to get @elastic/kibana-operations and @elastic/appex-qa
opinion on that one, given they have more expertise than I do on that
area.

- Running performances tests

We should absolutely run perf testing between http/1.1 over https and
http/2, to make sure that it goes into the right directly (at least in
term of user perceived speed), but I did not do it in the scope of this
PR (and @dmlemeshko is on PTO so... 😅)

## Release Note

Add support for `http2` to the Kibana server. `http2` can be enabled by
setting `server.protocol: http2` in the Kibana config file.

Note: by default, enabling `http2` requires a valid `h2c` configuration,
meaning that it can only run over HTTPS with TLS1.2+

Please refer to the Kibana config documentation for more details.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-06-03 09:34:13 +02:00
..
src Add http2 support for Kibana server (#183465) 2024-06-03 09:34:13 +02:00
index.ts Add http2 support for Kibana server (#183465) 2024-06-03 09:34:13 +02:00
jest.config.js Remove core->cli dependency (#95145) 2021-03-30 13:39:32 +02:00
kibana.jsonc Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
package.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
README.md Remove core->cli dependency (#95145) 2021-03-30 13:39:32 +02:00
tsconfig.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00

@kbn/http-tools

Http utilities for core and the basepath server