[6.x] Support PKCS#12 encoded certificates (#17261) (#17546)

* [6.x] Backport Support PKCS#12 Certificates

docs/setup/production.asciidoc

@@ -54,6 +54,15 @@ server.ssl.key: /path/to/your/server.key
 server.ssl.certificate: /path/to/your/server.crt
 ----
 
+Alternatively, you can specify a PKCS#12 encoded certificate with the `server.ssl.keystore.path` property in `kibana.yml`:
+
+[source,text]
+----
+# SSL for outgoing requests from the Kibana Server (PKCS#12 formatted)
+server.ssl.enabled: true
+server.ssl.keystore.path: /path/to/your/server.p12
+----
+
 If you are using X-Pack Security or a proxy that provides an HTTPS endpoint for Elasticsearch,
 you can configure Kibana to access Elasticsearch via HTTPS so communications between
 the Kibana server and Elasticsearch are encrypted.

docs/setup/settings.asciidoc

@@ -23,8 +23,13 @@ To send *no* client-side headers, set this value to [] (an empty list).
 `elasticsearch.requestTimeout:`:: *Default: 30000* Time in milliseconds to wait for responses from the back end or
 Elasticsearch. This value must be a positive integer.
 `elasticsearch.shardTimeout:`:: *Default: 30000* Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
+`elasticsearch.ssl.keystore.path`:: Optional setting that provides the path to the PKCS#12-format SSL Certificate and Key file. This file is used to verify the identity of Kibana
+to Elasticsearch. Either this, or `elasticsearch.ssl.certificate`/`elasticsearch.ssl.key` pair is required when `xpack.ssl.verification_mode` in Elasticsearch is set to either 
+`certificate` or `full`. Specifying both `elasticsearch.ssl.keystore.path` and `elasticsearch.ssl.certificate` is not allowed.
 `elasticsearch.ssl.certificate:` and `elasticsearch.ssl.key:`:: Optional settings that provide the paths to the PEM-format SSL
-certificate and key files. These files are used to verify the identity of Kibana to Elasticsearch and are required when `xpack.ssl.verification_mode` in Elasticsearch is set to either `certificate` or `full`.
+certificate and key files. These files are used to verify the identity of Kibana to Elasticsearch.
+Either this, or `elasticsearch.ssl.keystore.path` is required when `xpack.ssl.verification_mode` in Elasticsearch is set to either `certificate` or `full`. 
+Specifying both `elasticsearch.ssl.certificate` and `elasticsearch.ssl.keystore.path` is not allowed.
 `elasticsearch.ssl.certificateAuthorities:`:: Optional setting that enables you to specify a list of paths to the PEM file for the certificate
 authority for your Elasticsearch instance.
 `elasticsearch.ssl.keyPassphrase:`:: The passphrase that will be used to decrypt the private key. This value is optional as the key may not be encrypted.
@@ -117,6 +122,8 @@ By turning this off, only the layers that are configured here will be included.
 `server.port:`:: *Default: 5601* Kibana is served by a back end server. This setting specifies the port to use.
 `server.ssl.enabled:`:: *Default: "false"* Enables SSL for outgoing requests from the Kibana server to the browser. When set to `true`, `server.ssl.certificate` and `server.ssl.key` are required
 `server.ssl.certificate:` and `server.ssl.key:`:: Paths to the PEM-format SSL certificate and SSL key files, respectively.
+`server.ssl.keystore.path`:: Path to the PKCS#12 encoded SSL certificate and key. This is an alternative to setting `server.ssl.certificate` and `server.ssl.key`.
+`server.ssl.keystore.password`:: The password that will be used to decrypt the private key within the keystore. This value is optional as the key may not be encrypted.
 `server.ssl.certificateAuthorities:`:: List of paths to PEM encoded certificate files that should be trusted.
 `server.ssl.cipherSuites:`:: *Default: ECDHE-RSA-AES128-GCM-SHA256, ECDHE-ECDSA-AES128-GCM-SHA256, ECDHE-RSA-AES256-GCM-SHA384, ECDHE-ECDSA-AES256-GCM-SHA384, DHE-RSA-AES128-GCM-SHA256, ECDHE-RSA-AES128-SHA256, DHE-RSA-AES128-SHA256, ECDHE-RSA-AES256-SHA384, DHE-RSA-AES256-SHA384, ECDHE-RSA-AES256-SHA256, DHE-RSA-AES256-SHA256, HIGH,!aNULL, !eNULL, !EXPORT, !DES, !RC4, !MD5, !PSK, !SRP, !CAMELLIA*. Details on the format, and the valid options, are available via the [OpenSSL cipher list format documentation](https://www.openssl.org/docs/man1.0.2/apps/ciphers.html#CIPHER-LIST-FORMAT)
 `server.ssl.keyPassphrase:`:: The passphrase that will be used to decrypt the private key. This value is optional as the key may not be encrypted.

src/cli/cluster/__tests__/base_path_proxy.js

@@ -0,0 +1,18 @@
+import expect from 'expect.js';
+import { set } from 'lodash';
+import BasePathProxy from '../base_path_proxy';
+
+describe('CLI Cluster Manager', function () {
+  describe('base_path_proxy constructor', function () {
+    it('should throw an error when both server.ssl.keystore.path and server.ssl.certificate are specified', function () {
+      const settings = {};
+      set(settings, 'server.ssl.keystore.path', '/cert.p12');
+      set(settings, 'server.ssl.certificate', './cert.crt');
+      set(settings, 'server.ssl.key', './cert.key');
+
+      expect(() => new BasePathProxy(null, settings)).to.throwError(
+        `Invalid Configuration: please specify either "server.ssl.keystore.path" or "server.ssl.certificate", not both.`
+      );
+    });
+  });
+});

src/cli/cluster/base_path_proxy.js

@@ -26,13 +26,28 @@ export default class BasePathProxy {
 
     const sslEnabled = config.get('server.ssl.enabled');
     if (sslEnabled) {
-      this.proxyAgent = new HttpsAgent({
-        key: readFileSync(config.get('server.ssl.key')),
-        passphrase: config.get('server.ssl.keyPassphrase'),
-        cert: readFileSync(config.get('server.ssl.certificate')),
-        ca: map(config.get('server.ssl.certificateAuthorities'), readFileSync),
+      const agentOptions = {
+        ca: map(config.get('server.ssl.certificateAuthorities'), (certAuthority) => readFileSync(certAuthority)),
         rejectUnauthorized: false
-      });
+      };
+
+      const keystoreConfig = config.get('server.ssl.keystore.path');
+      const pemConfig = config.get('server.ssl.certificate');
+
+      if (keystoreConfig && pemConfig) {
+        throw new Error(`Invalid Configuration: please specify either "server.ssl.keystore.path" or "server.ssl.certificate", not both.`);
+      }
+
+      if (keystoreConfig) {
+        agentOptions.pfx = readFileSync(keystoreConfig);
+        agentOptions.passphrase = config.get('server.ssl.keystore.password');
+      } else {
+        agentOptions.key = readFileSync(config.get('server.ssl.key'));
+        agentOptions.cert = readFileSync(pemConfig);
+        agentOptions.passphrase = config.get('server.ssl.keyPassphrase');
+      }
+
+      this.proxyAgent = new HttpsAgent(agentOptions);
     }
 
     if (!this.basePath) {

src/cli/serve/serve.js

@@ -45,7 +45,7 @@ function readServerSettings(opts, extraCliOptions) {
       set('server.ssl.enabled', true);
     }
 
-    if (opts.ssl && !has('server.ssl.certificate') && !has('server.ssl.key')) {
+    if (opts.ssl && !has('server.ssl.keystore.path') && !has('server.ssl.certificate') && !has('server.ssl.key')) {
       set('server.ssl.certificate', DEV_SSL_CERT_PATH);
       set('server.ssl.key', DEV_SSL_KEY_PATH);
     }

src/core_plugins/console/server/elasticsearch_proxy_config.js

@@ -37,7 +37,10 @@ const createAgent = (server) => {
   }
 
   // Add client certificate and key if required by elasticsearch
-  if (config.get('elasticsearch.ssl.certificate') && config.get('elasticsearch.ssl.key')) {
+  if (config.get('elasticsearch.ssl.keystore.path')) {
+    agentOptions.pfx = readFile(config.get('elasticsearch.ssl.keystore.path'));
+    agentOptions.passphrase = config.get('elasticsearch.ssl.keystore.password');
+  } else if (config.get('elasticsearch.ssl.certificate') && config.get('elasticsearch.ssl.key')) {
     agentOptions.cert = readFile(config.get('elasticsearch.ssl.certificate'));
     agentOptions.key = readFile(config.get('elasticsearch.ssl.key'));
     agentOptions.passphrase = config.get('elasticsearch.ssl.keyPassphrase');

src/core_plugins/elasticsearch/index.js

@@ -16,46 +16,52 @@ export default function (kibana) {
   return new kibana.Plugin({
     require: ['kibana'],
     config(Joi) {
-      const { array, boolean, number, object, string, ref } = Joi;
-
-      const sslSchema = object({
-        verificationMode: string().valid('none', 'certificate', 'full').default('full'),
-        certificateAuthorities: array().single().items(string()),
-        certificate: string(),
-        key: string(),
-        keyPassphrase: string()
+      const sslSchema = Joi.object({
+        verificationMode: Joi.string().valid('none', 'certificate', 'full').default('full'),
+        certificateAuthorities: Joi.array().single().items(Joi.string()),
+        certificate: Joi.string(),
+        key: Joi.when('certificate', {
+          is: Joi.exist(),
+          then: Joi.string().required(),
+          otherwise: Joi.string().forbidden()
+        }),
+        keystore: Joi.object({
+          path: Joi.string(),
+          password: Joi.string()
+        }).default(),
+        keyPassphrase: Joi.string()
       }).default();
 
-      return object({
-        enabled: boolean().default(true),
-        url: string().uri({ scheme: ['http', 'https'] }).default('http://localhost:9200'),
-        preserveHost: boolean().default(true),
-        username: string(),
-        password: string(),
-        shardTimeout: number().default(30000),
-        requestTimeout: number().default(30000),
-        requestHeadersWhitelist: array().items().single().default(DEFAULT_REQUEST_HEADERS),
-        customHeaders: object().default({}),
-        pingTimeout: number().default(ref('requestTimeout')),
-        startupTimeout: number().default(5000),
-        logQueries: boolean().default(false),
+      return Joi.object({
+        enabled: Joi.boolean().default(true),
+        url: Joi.string().uri({ scheme: ['http', 'https'] }).default('http://localhost:9200'),
+        preserveHost: Joi.boolean().default(true),
+        username: Joi.string(),
+        password: Joi.string(),
+        shardTimeout: Joi.number().default(30000),
+        requestTimeout: Joi.number().default(30000),
+        requestHeadersWhitelist: Joi.array().items().single().default(DEFAULT_REQUEST_HEADERS),
+        customHeaders: Joi.object().default({}),
+        pingTimeout: Joi.number().default(Joi.ref('requestTimeout')),
+        startupTimeout: Joi.number().default(5000),
+        logQueries: Joi.boolean().default(false),
         ssl: sslSchema,
         apiVersion: Joi.string().default('master'),
-        healthCheck: object({
-          delay: number().default(2500)
+        healthCheck: Joi.object({
+          delay: Joi.number().default(2500)
         }).default(),
-        tribe: object({
-          url: string().uri({ scheme: ['http', 'https'] }),
-          preserveHost: boolean().default(true),
-          username: string(),
-          password: string(),
-          shardTimeout: number().default(0),
-          requestTimeout: number().default(30000),
-          requestHeadersWhitelist: array().items().single().default(DEFAULT_REQUEST_HEADERS),
-          customHeaders: object().default({}),
-          pingTimeout: number().default(ref('requestTimeout')),
-          startupTimeout: number().default(5000),
-          logQueries: boolean().default(false),
+        tribe: Joi.object({
+          url: Joi.string().uri({ scheme: ['http', 'https'] }),
+          preserveHost: Joi.boolean().default(true),
+          username: Joi.string(),
+          password: Joi.string(),
+          shardTimeout: Joi.number().default(0),
+          requestTimeout: Joi.number().default(30000),
+          requestHeadersWhitelist: Joi.array().items().single().default(DEFAULT_REQUEST_HEADERS),
+          customHeaders: Joi.object().default({}),
+          pingTimeout: Joi.number().default(Joi.ref('requestTimeout')),
+          startupTimeout: Joi.number().default(5000),
+          logQueries: Joi.boolean().default(false),
           ssl: sslSchema,
           apiVersion: Joi.string().default('master'),
         }).default()

src/core_plugins/elasticsearch/lib/__tests__/fixtures/cert.pfx

@@ -0,0 +1 @@
+test pfx

src/core_plugins/elasticsearch/lib/__tests__/parse_config.js

@@ -10,7 +10,8 @@ describe('plugins/elasticsearch', function () {
         serverConfig = {
           url: 'https://localhost:9200',
           ssl: {
-            verificationMode: 'full'
+            verificationMode: 'full',
+            keystore: {}
           }
         };
       });
@@ -86,6 +87,25 @@ describe('plugins/elasticsearch', function () {
         const config = parseConfig(serverConfig);
         expect(config.ssl.passphrase).to.be('secret');
       });
+
+      it(`sets pfx when a PKCS#12 certificate bundle is specified`, function () {
+        serverConfig.ssl.keystore.path = __dirname + '/fixtures/cert.pfx';
+        serverConfig.ssl.keystore.password = 'secret';
+
+        const config = parseConfig(serverConfig);
+        expect(Buffer.isBuffer(config.ssl.pfx)).to.be(true);
+        expect(config.ssl.pfx.toString('utf-8')).to.be('test pfx\n');
+        expect(config.ssl.passphrase).to.be('secret');
+      });
+
+      it('throws an error when both pfx and certificate are specified', function () {
+        serverConfig.ssl.certificate = __dirname + '/fixtures/cert.crt';
+        serverConfig.ssl.keystore.path = __dirname + '/fixtures/cert.pfx';
+
+        expect(() => parseConfig(serverConfig)).to.throwError(
+          `Invalid Configuration: please specify either "elasticsearch.ssl.keystore.path" or "elasticsearch.ssl.certificate", not both.`
+        );
+      });
     });
   });
 });

src/core_plugins/elasticsearch/lib/parse_config.js

@@ -5,6 +5,7 @@ import { readFileSync } from 'fs';
 import Bluebird from 'bluebird';
 
 const readFile = (file) => readFileSync(file, 'utf8');
+const readBinaryFile = (file) => readFileSync(file);
 
 export function parseConfig(serverConfig = {}) {
   const config = {
@@ -56,8 +57,20 @@ export function parseConfig(serverConfig = {}) {
   }
 
   // Add client certificate and key if required by elasticsearch
-  if (get(serverConfig, 'ssl.certificate') && get(serverConfig, 'ssl.key')) {
-    config.ssl.cert = readFile(serverConfig.ssl.certificate);
+  const keystoreConfig = get(serverConfig, 'ssl.keystore.path');
+  const pemConfig = get(serverConfig, 'ssl.certificate');
+
+  if (keystoreConfig && pemConfig) {
+    throw new Error(
+      `Invalid Configuration: please specify either "elasticsearch.ssl.keystore.path" or "elasticsearch.ssl.certificate", not both.`
+    );
+  }
+
+  if (keystoreConfig) {
+    config.ssl.pfx = readBinaryFile(keystoreConfig);
+    config.ssl.passphrase = get(serverConfig, 'ssl.keystore.password');
+  } else if (pemConfig && get(serverConfig, 'ssl.key')) {
+    config.ssl.cert = readFile(pemConfig);
     config.ssl.key = readFile(serverConfig.ssl.key);
     config.ssl.passphrase = serverConfig.ssl.keyPassphrase;
   }

src/server/config/config.js

@@ -134,6 +134,7 @@ export class Config {
           const child = schema._inner.children[i];
           // If the child is an object recurse through it's children and return
           // true if there's a match
+
           if (child.schema._type === 'object') {
             if (has(key, child.schema, path.concat([child.key]))) return true;
           // if the child matches, return true

src/server/config/schema.js

@@ -5,6 +5,25 @@ import os from 'os';
 import { fromRoot } from '../../utils';
 import { getData } from '../path';
 
+const sslSchema = Joi.object({
+  enabled: Joi.boolean().optional().default(false),
+  redirectHttpFromPort: Joi.number().default(),
+  keystore: Joi.object({
+    path: Joi.string(),
+    password: Joi.string()
+  }).default(),
+  certificate: Joi.string(),
+  key: Joi.when('certificate', {
+    is: Joi.exist(),
+    then: Joi.string().required(),
+    otherwise: Joi.string().forbidden()
+  }),
+  keyPassphrase: Joi.string(),
+  certificateAuthorities: Joi.array().single().items(Joi.string()).default([]),
+  supportedProtocols: Joi.array().items(Joi.string().valid('TLSv1', 'TLSv1.1', 'TLSv1.2')).default([]),
+  cipherSuites: Joi.array().items(Joi.string()).default(cryptoConstants.defaultCoreCipherList.split(':'))
+});
+
 export default () => Joi.object({
   pkg: Joi.object({
     version: Joi.string().default(Joi.ref('$version')),
@@ -59,22 +78,7 @@ export default () => Joi.object({
       otherwise: Joi.default(false),
     }),
     customResponseHeaders: Joi.object().unknown(true).default({}),
-    ssl: Joi.object({
-      enabled: Joi.boolean().default(false),
-      redirectHttpFromPort: Joi.number(),
-      certificate: Joi.string().when('enabled', {
-        is: true,
-        then: Joi.required(),
-      }),
-      key: Joi.string().when('enabled', {
-        is: true,
-        then: Joi.required()
-      }),
-      keyPassphrase: Joi.string(),
-      certificateAuthorities: Joi.array().single().items(Joi.string()).default([]),
-      supportedProtocols: Joi.array().items(Joi.string().valid('TLSv1', 'TLSv1.1', 'TLSv1.2')),
-      cipherSuites: Joi.array().items(Joi.string()).default(cryptoConstants.defaultCoreCipherList.split(':'))
-    }).default(),
+    ssl: sslSchema.default(),
     cors: Joi.when('$dev', {
       is: true,
       then: Joi.object().default({

src/server/config/schema.test.js

@@ -119,16 +119,6 @@ describe('Config schema', function () {
           const { error } = validate(config);
           expect(error).toBe(null);
         });
-
-        it('is required when ssl is enabled', function () {
-          const config = {};
-          set(config, 'server.ssl.enabled', true);
-          set(config, 'server.ssl.key', '/path.key');
-          const { error } = validate(config);
-          expect(error).toBeInstanceOf(Object);
-          expect(error).toHaveProperty('details');
-          expect(error.details[0]).toHaveProperty('path', 'server.ssl.certificate');
-        });
       });
 
       describe('key', function () {
@@ -150,6 +140,40 @@ describe('Config schema', function () {
         });
       });
 
+      describe('keystore.path', function () {
+        it('isn\'t required when ssl isn\'t enabled', function () {
+          const config = {};
+          set(config, 'server.ssl.enabled', false);
+          const { error } = validate(config);
+          expect(error).toBe(null);
+        });
+
+        it('is allowed when ssl is enabled, and a certificate is not specified', function () {
+          const config = {};
+          set(config, 'server.ssl.enabled', true);
+          set(config, 'server.ssl.keystore.path', '/path.p12');
+          const { error } = validate(config);
+          expect(error).toBe(null);
+        });
+      });
+
+      describe('keystore.password', function () {
+        it('isn\'t required when ssl isn\'t enabled', function () {
+          const config = {};
+          set(config, 'server.ssl.enabled', false);
+          const { error } = validate(config);
+          expect(error).toBe(null);
+        });
+
+        it('is allowed when ssl is enabled, and a certificate is not specified', function () {
+          const config = {};
+          set(config, 'server.ssl.enabled', true);
+          set(config, 'server.ssl.keystore.password', 'secret');
+          const { error } = validate(config);
+          expect(error).toBe(null);
+        });
+      });
+
       describe('keyPassphrase', function () {
         it('is a possible config value', function () {
           const config = {};

src/server/config/transform_deprecations.js

@@ -7,9 +7,13 @@ const serverSslEnabled = (settings, log) => {
   const has = partial(_.has, settings);
   const set = partial(_.set, settings);
 
-  if (!has('server.ssl.enabled') && has('server.ssl.certificate') && has('server.ssl.key')) {
+  const hasPkcs12Cert = has('server.ssl.keystore.path');
+  const hasPemCert = has('server.ssl.certificate') && has('server.ssl.key');
+
+  if (!has('server.ssl.enabled') && (hasPkcs12Cert || hasPemCert)) {
     set('server.ssl.enabled', true);
-    log('Enabling ssl by only specifying server.ssl.certificate and server.ssl.key is deprecated. Please set server.ssl.enabled to true');
+    log('Enabling ssl by only specifying server.ssl.keystore.path or server.ssl.certificate/key is deprecated. '
+      + 'Please set server.ssl.enabled to true');
   }
 };
 

src/server/config/transform_deprecations.test.js

@@ -18,6 +18,21 @@ describe('server/config', function () {
         expect(result.server.ssl.enabled).toBe(true);
       });
 
+      it('sets enabled to true when keystore.path is set', function () {
+        const settings = {
+          server: {
+            ssl: {
+              keystore: {
+                path: '/server.pfx'
+              }
+            }
+          }
+        };
+
+        const result = transformDeprecations(settings);
+        expect(result.server.ssl.enabled).toBe(true);
+      });
+
       it('logs a message when automatically setting enabled to true', function () {
         const settings = {
           server: {

src/server/http/setup_connection.js

@@ -32,14 +32,28 @@ export function setupConnection(server, config) {
     return;
   }
 
+  const tlsOptions = {};
+  const keystoreConfig = config.get('server.ssl.keystore.path');
+  const pemConfig = config.get('server.ssl.certificate');
+
+  if (keystoreConfig && pemConfig) {
+    throw new Error(`Invalid Configuration: please specify either "server.ssl.keystore.path" or "server.ssl.certificate", not both.`);
+  }
+
+  if (keystoreConfig) {
+    tlsOptions.pfx = readFileSync(keystoreConfig);
+    tlsOptions.passphrase = config.get('server.ssl.keystore.password');
+  } else {
+    tlsOptions.key = readFileSync(config.get('server.ssl.key'));
+    tlsOptions.cert = readFileSync(pemConfig);
+    tlsOptions.passphrase = config.get('server.ssl.keyPassphrase');
+  }
+
   const connection = server.connection({
     ...connectionOptions,
     tls: {
-      key: readFileSync(config.get('server.ssl.key')),
-      cert: readFileSync(config.get('server.ssl.certificate')),
+      ...tlsOptions,
       ca: config.get('server.ssl.certificateAuthorities').map(ca => readFileSync(ca, 'utf8')),
-      passphrase: config.get('server.ssl.keyPassphrase'),
-
       ciphers: config.get('server.ssl.cipherSuites').join(':'),
       // We use the server's cipher order rather than the client's to prevent the BEAST attack
       honorCipherOrder: true,