Adding RBAC Phase 1 Docs (#21178)

* Beginning to work on the role management APIs. Added docs for GET

* Adding PUT docs

* Adding PUT details

* Adding delete docs

* Fixing linking

* Adding Kibana privileges section

* Fixing dashboard only mode docs

* Fixing a few more references to managing roles

* Beginning to work on authorization docs, might be moving some to
stack-docs

* Collapsing authorization description in the kibana privileges page

* Adding audit logging section

* Revising the language on the Kibana role management section

* Splitting back out the auth/privileges and adding legacy fallback
details

* Revising language around impact of disabling security

* Changing Kibana to {kib} and Elasticsearch to {es}

* Beginning to work on developer centric docs

* Fixing some formatting, adding some diagrams

* Adding note about the role management APIs

* Adding overview, fixing small syntax issues

* Fixing chunk name for transitioning to application privileges

* Adjusting tone for the authorization introduction

* Changing the tone and structure of the RBAC docs

* Deleting blog stuff after refactoring

* Addressing first round of peer review comments

* Fixing endpoints links

* Peer review suggested edits

* Addressing other PR feedback

docs/api.asciidoc

@@ -27,11 +27,13 @@ entirely.
 [float]
 == APIs
 
+* <<role-management-api>>
 * <<saved-objects-api>>
 * <<logstash-configuration-management-api>>
 * <<url-shortening-api>>
 --
 
+include::api/role-management.asciidoc[]
 include::api/saved-objects.asciidoc[]
 include::api/logstash-configuration-management.asciidoc[]
 include::api/url-shortening.asciidoc[]

docs/api/role-management.asciidoc

@@ -0,0 +1,18 @@
+[role="xpack"]
+[[role-management-api]]
+== Kibana Role Management API
+
+experimental[This API is *experimental* and may be changed or removed completely in a future release. The underlying mechanism of enforcing role based access control is stable, but the APIs for managing the roles are currently experimental.]
+
+The role management API allows people to manage roles that grant <<kibana-privileges>>.
+It is *not* supported to do so using the
+{ref}/security-api-roles.html[{es} Role Management APIs], and doing 
+so will likely cause {kib}'s authorization to behave unexpectedly. 
+
+* <<role-management-api-put>>
+* <<role-management-api-get>>
+* <<role-management-api-delete>>
+
+include::role-management/put.asciidoc[]
+include::role-management/get.asciidoc[]
+include::role-management/delete.asciidoc[]

docs/api/role-management/delete.asciidoc

@@ -0,0 +1,24 @@
+[[role-management-api-delete]]
+=== Delete role
+
+experimental[This API is experimental and may be changed or removed completely in a future release. Although the underlying mechanism of enforcing role-based access control is stable, the APIs for managing the roles are currently experimental.]
+
+==== Authorization
+
+To use this API, you must have at least the `manage_security` cluster privilege.
+
+==== Request
+
+To delete a role, submit a DELETE request to the `/api/security/role/<rolename>`
+endpoint:
+
+[source,js]
+--------------------------------------------------
+DELETE /api/security/role/my_admin_role
+--------------------------------------------------
+// KIBANA
+
+==== Response
+
+If the role is successfully deleted, the response code is `204`; otherwise, the response
+code is 404.

docs/api/role-management/get.asciidoc

@@ -0,0 +1,111 @@
+[[role-management-api-get]]
+=== Get Role
+
+experimental[This API is experimental and may be changed or removed completely in a future release. Although the underlying mechanism of enforcing role-based access control is stable, the APIs for managing the roles are currently experimental.]
+
+Retrieves all {kib} roles, or a specific role.
+
+==== Authorization
+
+To use this API, you must have at least the `manage_security` cluster privilege.
+
+==== Get all {kib} roles
+
+===== Request
+
+To retrieve all roles, issue a GET request to the
+/api/security/role endpoint.
+
+[source,js]
+--------------------------------------------------
+GET /api/security/role
+--------------------------------------------------
+// KIBANA
+
+===== Response
+
+A successful call returns a response code of `200` and a response body containing a JSON 
+representation of the roles.
+
+[source,js]
+--------------------------------------------------
+[
+  {
+    "name": "my_kibana_role",
+    "metadata" : {
+      "version" : 1
+    },
+    "transient_metadata": {
+      "enabled": true
+    },
+    "elasticsearch": {
+      "indices": [ ],
+      "cluster": [ ],
+      "run_as": [ ]
+    },
+    "kibana": [ {
+        "privileges": [ "all" ]
+    } ],
+  },
+  {
+    "name": "my_admin_role",
+    "metadata" : {
+      "version" : 1
+    },
+    "transient_metadata": {
+      "enabled": true
+    },
+    "elasticsearch": {
+      "cluster" : [ "all" ],
+      "indices" : [ {
+        "names" : [ "index1", "index2" ],
+        "privileges" : [ "all" ],
+        "field_security" : {
+          "grant" : [ "title", "body" ]
+        },
+        "query" : "{\"match\": {\"title\": \"foo\"}}"
+      } ],
+    },
+    "kibana": [ ]
+  }
+]
+--------------------------------------------------
+
+==== Get a specific role
+
+===== Request
+
+To retrieve a specific role, issue a GET request to
+the `/api/security/role/<rolename>` endpoint:
+
+[source,js]
+--------------------------------------------------
+GET /api/security/role/my_kibana_role
+--------------------------------------------------
+// KIBANA
+
+===== Response
+
+A successful call returns a response code of `200` and a response body containing a JSON 
+representation of the role.
+
+[source,js]
+--------------------------------------------------
+{
+  "name": "my_kibana_role",
+  "metadata" : {
+    "version" : 1
+  },
+  "transient_metadata": {
+    "enabled": true
+  },
+  "elasticsearch": {
+    "cluster": [ ],
+    "indices": [ ],
+    "run_as": [ ]
+  },
+  "kibana": [ {
+      "privileges": [ "all" ]
+  } ],
+}
+--------------------------------------------------

docs/api/role-management/put.asciidoc

@@ -0,0 +1,64 @@
+[[role-management-api-put]]
+=== Create or Update Role
+
+experimental[This API is experimental and may be changed or removed completely in a future release. Although the underlying mechanism of enforcing role-based access control is stable, the APIs for managing the roles are currently experimental.]
+
+Creates a new {kib} role or updates the attributes of an existing role. {kib} roles are stored in the
+{es} native realm.
+
+==== Authorization
+
+To use this API, you must have at least the `manage_security` cluster privilege.
+
+==== Request
+
+To create or update a role, issue a PUT request to the 
+`/api/security/role/<rolename>` endpoint.
+
+[source,js]
+--------------------------------------------------
+PUT /api/security/role/my_kibana_role
+--------------------------------------------------
+
+==== Request Body
+
+The following parameters can be specified in the body of a PUT request to add or update a role:
+
+`metadata`:: (object) Optional meta-data. Within the `metadata` object, keys
+that begin with `_` are reserved for system usage.
+
+`elasticsearch`:: (object) Optional {es} cluster and index privileges, valid keys are 
+`cluster`, `indices` and `run_as`. For more information, see {xpack-ref}/defining-roles.html[Defining Roles].
+
+`kibana`:: (list) A list of objects that specify the <<kibana-privileges>>.
+
+===== Example
+
+[source,js]
+--------------------------------------------------
+PUT /api/security/role/my_kibana_role
+{
+  "metadata" : {
+    "version" : 1
+  },
+  "elasticsearch": {
+    "cluster" : [ "all" ],
+    "indices" : [ {
+      "names" : [ "index1", "index2" ],
+      "privileges" : [ "all" ],
+      "field_security" : {
+        "grant" : [ "title", "body" ]
+      },
+      "query" : "{\"match\": {\"title\": \"foo\"}}"
+    } ],
+  },
+  "kibana": [ {
+      "privileges": [ "all" ]
+  } ],
+}
+--------------------------------------------------
+// KIBANA
+
+==== Response
+
+A successful call returns a response code of `204` and no response body.

docs/dashboard.asciidoc

@@ -119,4 +119,4 @@ TIP: You can create a link to a dashboard by title by doing this: +
 
 TIP: When sharing a link to a dashboard snapshot, use the *Short URL*. Snapshot
 URLs are long and can be problematic for Internet Explorer and other
-tools.  To create a short URL, you must have write access to `.kibana`.
+tools.  To create a short URL, you must have write access to {kib}.

docs/development.asciidoc

@@ -15,4 +15,7 @@ include::development/core-development.asciidoc[]
 
 include::development/plugin-development.asciidoc[]
 
+include::development/security/index.asciidoc[]
+
 include::development/pr-review.asciidoc[]
+

docs/development/security/index.asciidoc

@@ -0,0 +1,12 @@
+[[development-security]]
+== Security
+
+Kibana has generally been able to implement security transparently to core and plugin developers, and this largely remains the case. {kib} on two methods that the <<development-elasticsearch, elasticsearch plugin>>'s `Cluster` provides: `callWithRequest` and `callWithInternalUser`. 
+
+`callWithRequest` executes requests against Elasticsearch using the authentication credentials of the Kibana end-user. So, if you log into Kibana with the user of `foo` when `callWithRequest` is used, {kib} execute the request against Elasticsearch as the user `foo`. Historically, `callWithRequest` has been used extensively to perform actions that are initiated at the request of Kibana end-users.
+
+`callWithInternalUser` executes requests against Elasticsearch using the internal Kibana server user, and has historically been used for performing actions that aren't initiated by Kibana end users; for example, creating the initial `.kibana` index or performing health checks against Elasticsearch.
+
+However, with the changes that role-based access control (RBAC) introduces, this is no longer cut and dry. {kib} now requires all access to the `.kibana` index goes through the `SavedObjectsClient`. This used to be a best practice, as the `SavedObjectsClient` was responsible for translating the documents stored in Elasticsearch to and from Saved Objects, but RBAC is now taking advantage of this abstraction to implement access control and determine when to use `callWithRequest` versus `callWithInternalUser`.
+
+include::rbac.asciidoc[]

docs/development/security/rbac.asciidoc

@@ -0,0 +1,173 @@
+[[development-security-rbac]]
+=== Role-based access control
+
+Role-based access control (RBAC) in {kib} relies upon the {ref}/security-api-privileges.html[privilege APIs] that Elasticsearch exposes. This {kib} to define the privileges that {kib} wishes to grant to users, assign them to the relevant users using roles, and then authorize the user to perform a specific action. This is handled within a secured instance of the `SavedObjectsClient` and available transparently to consumers when using `request.getSavedObjectsClient()` or `savedObjects.getScopedSavedObjectsClient()`.
+
+[[development-rbac-privileges]]
+==== {kib} Privileges
+
+When {kib} first starts up, it executes the following `POST` request against {es}. This synchronizes the definition of the privileges with various `actions` which are later used to authorize a user:
+
+[source,js]
+----------------------------------
+POST /_xpack/security/privilege
+Content-Type: application/json
+Authorization: Basic kibana changeme
+
+{
+   "kibana-.kibana":{
+       "all":{
+           "application":"kibana-.kibana",
+           "name":"all",
+           "actions":[
+               "version:7.0.0-alpha1-SNAPSHOT",
+               "action:login",
+               "action:*"
+           ],
+           "metadata":{}
+       },
+       "read":{
+           "application":"kibana-.kibana",
+           "name":"read",
+           "actions":[
+               "version:7.0.0-alpha1-SNAPSHOT",
+               "action:login",
+               "action:saved_objects/dashboard/get",
+               "action:saved_objects/dashboard/bulk_get",
+               "action:saved_objects/dashboard/find",
+               ...
+           ],"metadata":{}}
+   }
+}
+----------------------------------
+
+[NOTE]
+==============================================
+
+The application is created by concatenating the prefix of `kibana-` with the value of `kibana.index` from the `kibana.yml`, so different {kib} tenants are isolated from one another.
+
+==============================================
+
+[[development-rbac-assigning-privileges]]
+==== Assigning {kib} Privileges
+
+{kib} privileges are assigned to specific roles using the `applications` element. For example, the following role assigns the <<kibana-privileges-all, all>> privilege at `*` `resources` (which will in the future be used to secure spaces) to the default {kib} `application`:
+
+[source,js]
+----------------------------------
+"new_kibana_user": {
+   "applications": [
+     {
+       "application": "kibana-.kibana",
+       "privileges": [
+         "all"
+       ],
+       "resources": [
+         "*"
+       ]
+     }
+   ]
+ }
+----------------------------------
+
+Roles that grant <<kibana-privileges>> should be managed using the <<role-management-api>> or the *Management* / *Security* / *Roles* page, not directly using the {es} {ref}/security-api-roles.html[Role Management API]. This role can then be assigned to users using {es}'s {ref}/security-api-users.html[User Management APIs].
+
+[[development-rbac-authorization]]
+==== Authorization
+
+The {es} {ref}/security-api-privileges.html#security-api-privileges[has privileges API]determines whether the user is authorized to perform a specific action:
+
+[source,js]
+----------------------------------
+POST /_xpack/security/user/_has_privileges
+Content-Type: application/json
+Authorization: Basic foo_read_only_user password
+
+{
+   "applications":[
+       {
+           "application":"kibana-.kibana",
+           "resources":["*"],
+           "privileges":[
+             "action:saved_objects/dashboard/save",
+           ]
+       }
+   ]
+}
+----------------------------------
+
+{es} checks if the user is granted a specific action. If the user is assigned a role that grants a privilege, {es} uses the <<development-rbac-privileges, {kib} privileges>> definition to associate this with the actions, which makes authorizing users more intuitive and flexible programatically.
+
+Once we have authorized the user to perform a specific action, we can execute the request using `callWithInternalUser`.
+
+[[development-rbac-legacy-fallback]]
+==== Legacy Fallback
+
+Users have existign roles that rely on index privileges to the `.kibana` index. The legacy fallback uses the `callWithRequest` method when the user doesn't have and application privileges. This relies on the user have index privileges on `.kibana`. The legacy fallback will be available until 7.0.
+
+Within the secured instance of the `SavedObjectsClient` the `_has_privileges` check determines if the user has any index privileges on the `.kibana` index:
+
+[source,js]
+----------------------------------
+POST /_xpack/security/user/_has_privileges
+Content-Type: application/json
+Authorization: Basic foo_legacy_user password
+
+{
+   "applications":[
+       {
+           "application":"kibana-.kibana",
+           "resources":["*"],
+           "privileges":[
+             "action:saved_objects/dashboard/save"
+           ]
+       }
+   ],
+   "index": [
+       {
+           "names": ".kibana",
+           "privileges": ["create", "delete", "read", "view_index_metadata"]
+       }
+   ]
+}
+----------------------------------
+
+Here is an example response if the user does not have application privileges, but does have privileges on the `.kibana` index:
+
+[source,js]
+----------------------------------
+{
+  "username": "foo_legacy_user",
+  "has_all_requested": false,
+  "cluster": {},
+  "index": {
+    ".kibana": {
+      "read": true,
+      "view_index_metadata": true,
+      "create": true,
+      "delete": true
+    }
+  },
+  "application": {
+    "kibana-.kibana": {
+      "*": {
+        "action:saved_objects/dashboard/save": false
+      }
+    }
+  }
+}
+----------------------------------
+
+{kib} automatically detects that the request could be executed against `.kibana` using `callWithRequest` and does so.
+
+When the user first logs into {kib}, if they have no application privileges and will have to rely on the legacy fallback, {kib} logs a deprecation warning similar to the following:
+
+[source,js]
+----------------------------------
+${username} relies on index privileges on the {kib} index. This is deprecated and will be removed in {kib} 7.0
+----------------------------------
+
+[[development-rbac-reserved-roles]]
+==== Reserved roles
+
+Ideally, the `kibana_user` and `kibana_dashboard_only_user` roles should only use application privileges, and no longer have index privileges on the `.kibana` index. However, making this switch forces the user to incur downtime if Elasticsearch is upgraded to >= 6.4, and {kib} is running < 6.4. To mitigate this downtime, for the 6.x releases the `kibana_user` and `kibana_dashbord_only_user` roles have both application privileges and index privileges. When {kib} is running >= 6.4 it uses the application privileges to authorize the user, but when {kib} is running < 6.4 {kib} relies on the direct index privileges. 

docs/management/dashboard_only_mode/advanced_configuration.asciidoc

@@ -3,7 +3,7 @@
 === Advanced Configuration for Dashboard Only Mode
 
 If {security} is enabled, Kibana has a built-in `kibana_dashboard_only_user` 
-role that grants read only access to the `.kibana` index. This role is sufficient
+role that grants read-only access to {kib}. This role is sufficient
 for most use cases.  However, if your setup requires a custom {kib} index, you can create
 your own roles and tag them as *Dashboard only mode*.
 
@@ -15,8 +15,8 @@ Here you can add as many roles as you like.
 image:management/dashboard_only_mode/images/advanced_dashboard_mode_role_setup.png["Advanced dashboard mode role setup"]
 
 By default, a *dashboard only mode* user doesn't have access to any data indices.
-To grant read only access to your custom {kib}
-index, you must assign `view_index_metadata` and `read` privileges.
+To grant read-only access to your custom {kib} instance,
+you must assign the read <<kibana-privileges, Kibana privilege>>.
 These privileges are available under *Management > Security > Roles*.
 
 For more information on roles and privileges, see {xpack-ref}/authorization.html[User Authorization].

docs/management/dashboard_only_mode/index.asciidoc

@@ -18,10 +18,10 @@ that grant the user appropriate data access. For information on roles
 and privileges, see {xpack-ref}/authorization.html[User Authorization].
 
 The `kibana_dashboard_only_user` role is 
-preconfigured with read only permissions on the `.kibana` index.
+preconfigured with read-only permissions to {kib}.
 
 IMPORTANT: If you assign users the `kibana_dashboard_only_user` role, along with a role 
-with write permissions on the `.kibana` index, they *will* have write access, 
+with write permissions to {kib}, they *will* have write access, 
 even though the controls remain hidden in the {kib} UI.
 
 IMPORTANT: If you also assign users the reserved `superuser` role, they will be able to see 

docs/security/audit-logging.asciidoc

@@ -0,0 +1,36 @@
+[role="xpack"]
+[[xpack-security-audit-logging]]
+===  Audit Logging
+
+You can enable auditing to keep track of security-related events such as
+authorization success and failures. Logging these events enables you
+to monitor {kib} for suspicious activity and provides evidence in the
+event of an attack.
+
+Use the {kib} audit logs in conjunction with {es}'s
+audit logging to get a holistic view of all security related events. 
+{kib} defers to {es}'s security model for authentication, data
+index authorization, and features that are driven by cluster-wide privileges.
+For more information on enabling audit logging in {es}, see
+{ref}/auditing.html[Auditing Security Events].
+
+[IMPORTANT]
+============================================================================
+Audit logs are **disabled** by default. To enable this functionality, you
+must set `xpack.security.audit.enabled` to `true` in `kibana.yml`.
+============================================================================
+
+Audit logging uses the standard {kib} logging output, which can be configured
+in the `kibana.yml` and is discussed in <<settings>>.
+
+==== Audit event types
+
+When you are auditing security events, each request can generate 
+multiple audit events. The following is a list of the events that can be generated:
+
+|======
+| `saved_objects_authorization_success`    | Logged when a user is authorized to access a saved
+                                             objects when using a role with <<kibana-privileges>>
+| `saved_objects_authorization_failure`    | Logged when a user isn't authorized to access a saved
+                                             objects when using a role with <<kibana-privileges>>
+|======

docs/security/authorization/index.asciidoc

@@ -0,0 +1,31 @@
+[role="xpack"]
+[[xpack-security-authorization]]
+=== Authorization
+
+Authorizing users to use {kib} in most configurations is as simple as assigning the user
+either the `kibana_user` or `kibana_dashboard_only_user` reserved role. If you're running 
+a single tenant of {kib} against your {es} cluster, this is sufficient and no other 
+action is required.
+
+==== Multi-tenant {kib}
+
+When running multiple tenants of {kib}, and changing the `kibana.index` in your `kibana.yml`, you 
+must create custom roles that authorize the user for that specific tenant. You can use
+either the *Management / Security / Roles* page in {kib} or the <<role-management-api>>
+to assign a specific <<kibana-privileges, Kibana privilege>> at that tenant. After creating the
+custom role, you should assign this role to the user(s) that you wish to have access.
+
+==== Legacy roles
+
+Prior to {kib} 6.4, {kib} users required index privileges to the `kibana.index`
+in {es}. This approach is deprecated starting in 6.4, and you will need to switch to using
+<<kibana-privileges>> before 7.0. When a user logs into {kib} and they're using
+a legacy role, the following is logged to your {kib} logs:
+
+[source,js]
+----------------------------------
+<username> relies on index privileges on the Kibana index. This is deprecated and will be removed in Kibana 7.0
+----------------------------------
+
+To disable legacy roles from being authorized in {kib}, set `xpack.security.authorization.legacyFallback` to `false`
+in your `kibana.yml`.

docs/security/authorization/kibana-privileges.asciidoc

@@ -0,0 +1,15 @@
+[role="xpack"]
+[[kibana-privileges]]
+=== Kibana privileges
+
+This section lists the Kibana privileges that you can assign to a role.
+
+[horizontal]
+[[kibana-privileges-all]]
+`all`::
+All Kibana privileges, can read, write and delete saved searches, dashboards, visualizations,
+short URLs, Timelion sheets, graph workspaces, index patterns and advanced settings.
+
+`read`::
+Can read saved searches, dashboards, visualizations, short URLs, Timelion sheets, graph workspaces,
+index patterns, and advanced settings.

docs/security/index.asciidoc

@@ -20,6 +20,19 @@ authentication and built-in users, see
 [float]
 === Roles
 
-You can manage roles on the *Management* / *Security* / *Roles* page.  For more
-information, see
-{xpack-ref}/authorization.html[Configuring Role-based Access Control].
+You can manage roles on the *Management* / *Security* / *Roles* page, or use 
+{kib}'s <<role-management-api>>. For more information on configuring roles for
+{kib} see <<xpack-security-authorization, {kib} Authorization>>.
+
+For a more holistic overview of configuring roles for the entire stack,
+see {xpack-ref}/authorization.html[Configuring Role-based Access Control].
+
+[NOTE]
+============================================================================
+Managing roles that grant <<kibana-privileges>> using {es}'s {ref}
+/security-api-roles.html[Role Management APIs] is not supported. Doing so will likely
+cause Kibana's authorization to behave unexpectedly.
+============================================================================
+
+include::authorization/index.asciidoc[]
+include::authorization/kibana-privileges.asciidoc[]

docs/security/securing-kibana.asciidoc

@@ -124,4 +124,5 @@ NOTE: This must be a user who has been assigned the `kibana_user` role.
 
 include::authentication/index.asciidoc[]
 include::securing-communications/index.asciidoc[]
+include::audit-logging.asciidoc[]
 include::{kib-repo-dir}/settings/security-settings.asciidoc[]

docs/settings/security-settings.asciidoc

@@ -14,11 +14,19 @@ It is enabled by default.
 `xpack.security.enabled`::
 Set to `true` (default) to enable {security}. +
 +
-If set to `false` in `kibana.yml`, the user and role management options are
-hidden in this {kib} instance. If `xpack.security.enabled` is set to `true` in
-`elasticsearch.yml`, however, you can still use the {security} APIs. To disable
-{security} entirely, see the
-{ref}/security-settings.html[{es} Security Settings].
+Do not set this to `false`. To disable {security} entirely, see
+{ref}/security-settings.html[{es} Security Settings]. +
++
+If set to `false` in `kibana.yml`, the login form, user and role management screens, and 
+authorization using <<kibana-privileges>> are disabled. +
++
+`xpack.security.audit.enabled`::
+Set to `true` to enable audit logging for security events. This is set to `false` by default.
+For more details see <<xpack-security-audit-logging>>.
+
+`xpack.security.authorization.legacyFallback`::
+Set to `true` (default) to enable the legacy fallback. See <<xpack-security-authorization>>
+for more details.
 
 [float]
 [[security-ui-settings]]