Security Entity Analytics - Privileged user monitoring - Privileged access detection support (#224008)

# Overview

This pull request adds capabilities associated with the [Privileged
Access Detection (PAD) anomaly integration
package](https://www.elastic.co/docs/reference/integrations/pad) as a
first-class-citizen into the Entity Analytics Privileged User Monitoring
feature.

# How to test

- Pull this branch into your local machine
- Ensure that the security experimental flag
`privilegeMonitoringEnabled` has been enabled, for example by setting
`xpack.securitySolution.enableExperimental:
[privilegeMonitoringEnabled]` in your `config/kibana.dev.yml` file
- Start Elasticsearch and Kibana
- From the
[security-documents-generator](https://github.com/elastic/security-documents-generator)
repository, run the following command: `yarn start
privileged-user-monitoring` (ensuring your config is pointing to your
locally running Elastic cluster). This will load "source" events that
are anomalous in nature. You can run this command more than once if
desired to upload more than 10 users.
- Grab the CSV file that the generator created in its console output.
- Open the Entity analytics page and navigate to dashboards (by clicking
"Go to dashboards")
- Add the privileged users from the previous step to the privileged
users index using the CSV File Upload option.
> [!NOTE]  
> Any errors regarding risk scoring are unrelated to this PR, and are
being resolved separately
- You should see a panel that says "Enable Privileged access detection".
Click "Install", and you'll meet a loading state.
- Once complete, you shouldn't see any results. That's because, even
though we **install** the ML jobs by default, we don't **run** them by
default.
- Click "ML Job Settings", and note that only `pad` jobs should be
displaying in this callout. Feel free to test this callout's links and
filtering options.
- Click "Run job" next to the job called
`pad_linux_high_count_privileged_process_events_by_user`. This is the
job for which we have anomaly data.
- Click away from the callout. **You still shouldn't see data.** That's
because there just aren't any anomalies "today".
- Change the global date filter at the top of the screen to "Last 30
days".
- You should see something similar to this:

<img width="1441" alt="Screenshot 2025-06-16 at 12 50 25 AM"
src="https://github.com/user-attachments/assets/2b3f11f2-f45d-4716-bb8e-79d2b585aa3e"
/>

- Congrats for making it this far! Some things to play around with:
- Click around on the anomaly filters (i.e., click 25-50), and notice
that the results will change.
- Ensure that the ordering of the users is based on the **highest single
anomaly score in any visible bucket**. Meaning, if user `samwise` has
only one anomaly, but its score is 99, and user `frodo` has dozens of
anomalies, but no higher than, say, 80, `samwise` will be on the top of
the list.
  - Click the user names to open the appropriate user flyout
- Change the global time filter to change ranges, and ensure the data
shows up appropriately. The buckets will try to roughly show 30 buckets
total, but will have a maximum of 3 hours, meaning if your range is too
small, it may show fewer than 30 buckets. This is intentional behavior,
as the PAD jobs have an anomaly job window of 3 hours.
- Click the button that says "View all in Anomaly Explorer", which uses
the currently selected global time range, and compare results
> [!WARNING]  
> Remember that the users in the privileged user monitoring table are
only those that you designated as privileged users in a previous step.
In contrast, the Anomaly Explorer page shows **all** users.
Additionally, note that there may be very slight differences between the
swimlanes, because ES|QL calculates the bucket dates slightly
differently than the Anomaly Explorer. **This should not affect the
results themselves, only the buckets that an individual anomaly score
might fall in.** You might see a single anomaly fall into one visual
bucket instead of another, but the date should be correct.

# Helpful hints

If you'd like an easy way to "reset" the Privileged Access Detection
package and delete its associated jobs (and anomaly data), so that you
can redo the onboarding flow, you can run the below commands (changing
any credentials as necessary):

```shell
curl "http://localhost:5601/api/fleet/epm/packages/pad/0.5.0" \
  -X 'DELETE' \
  -H 'elastic-api-version: 2023-10-31' \
  -H 'kbn-xsrf:true' \
  --user elastic:changeme

curl "http://localhost:5601/internal/ml/jobs/delete_jobs" \
  -H 'kbn-xsrf:true' \
  --user elastic:changeme \
  -X 'POST' \
  -H 'elastic-api-version: 1' \
  -H 'x-elastic-internal-origin:kibana' \
  -H 'Content-Type: application/json' \
  --data-raw '{
    "jobIds": [
      "pad_linux_high_count_privileged_process_events_by_user",
      "pad_linux_high_median_process_command_line_entropy_by_user",
      "pad_linux_rare_process_executed_by_user",
      "pad_okta_high_sum_concurrent_sessions_by_user",
      "pad_okta_rare_host_name_by_user",
      "pad_okta_rare_region_name_by_user",
      "pad_okta_rare_source_ip_by_user",
      "pad_okta_spike_in_group_application_assignment_changes",
      "pad_okta_spike_in_group_lifecycle_changes",
      "pad_okta_spike_in_group_membership_changes",
      "pad_okta_spike_in_group_privilege_changes",
      "pad_okta_spike_in_user_lifecycle_management_changes",
      "pad_windows_high_count_group_management_events",
      "pad_windows_high_count_special_logon_events",
      "pad_windows_high_count_special_privilege_use_events",
      "pad_windows_high_count_user_account_management_events",
      "pad_windows_rare_device_by_user",
      "pad_windows_rare_group_name_by_user",
      "pad_windows_rare_privilege_assigned_to_user",
      "pad_windows_rare_region_name_by_user",
      "pad_windows_rare_source_ip_by_user"
    ],
    "deleteUserAnnotations": true,
    "deleteAlertingRules": false
  }
'
```

# What's left?
- This PR does not implement "proactive" permissions checks. Instead,
relevant permission issues will cause requests/queries to fail, which
are appropriately shown in the UI for troubleshooting. This proactive
check will be tackled in [this (private) followup
issue](https://github.com/elastic/security-team/issues/12822).
- There is not yet a way to "upgrade" the package directly from the
Privileged User Monitoring screen. This would be a nice addition later
on, instead of requiring users to navigate to the integration page. This
behavior will be tackled in [this (private) followup
issue](https://github.com/elastic/security-team/issues/12823).

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

oas_docs/output/kibana.serverless.yaml

@@ -11317,6 +11317,72 @@ paths:
       summary: List all monitored users
       tags:
         - Security Entity Analytics API
+  /api/entity_analytics/privileged_user_monitoring/pad/install:
+    post:
+      operationId: InstallPrivilegedAccessDetectionPackage
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  message:
+                    type: string
+                required:
+                  - message
+          description: Successful response
+      summary: Installs the privileged access detection package for the Entity Analytics privileged user monitoring experience
+      tags:
+        - Security Entity Analytics API
+  /api/entity_analytics/privileged_user_monitoring/pad/status:
+    get:
+      operationId: GetPrivilegedAccessDetectionPackageStatus
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  jobs:
+                    items:
+                      type: object
+                      properties:
+                        description:
+                          type: string
+                        job_id:
+                          type: string
+                        state:
+                          enum:
+                            - closing
+                            - closed
+                            - opened
+                            - failed
+                            - opening
+                          type: string
+                      required:
+                        - job_id
+                        - state
+                    type: array
+                  ml_module_setup_status:
+                    enum:
+                      - complete
+                      - incomplete
+                    type: string
+                  package_installation_status:
+                    enum:
+                      - complete
+                      - incomplete
+                    type: string
+                required:
+                  - package_installation_status
+                  - ml_module_setup_status
+                  - jobs
+          description: Privileged access detection status retrieved
+      summary: Gets the status of the privileged access detection package for the Entity Analytics privileged user monitoring experience
+      tags:
+        - Security Entity Analytics API
   /api/entity_store/enable:
     post:
       operationId: InitEntityStore

oas_docs/output/kibana.yaml

@@ -13476,6 +13476,72 @@ paths:
       summary: List all monitored users
       tags:
         - Security Entity Analytics API
+  /api/entity_analytics/privileged_user_monitoring/pad/install:
+    post:
+      operationId: InstallPrivilegedAccessDetectionPackage
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  message:
+                    type: string
+                required:
+                  - message
+          description: Successful response
+      summary: Installs the privileged access detection package for the Entity Analytics privileged user monitoring experience
+      tags:
+        - Security Entity Analytics API
+  /api/entity_analytics/privileged_user_monitoring/pad/status:
+    get:
+      operationId: GetPrivilegedAccessDetectionPackageStatus
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  jobs:
+                    items:
+                      type: object
+                      properties:
+                        description:
+                          type: string
+                        job_id:
+                          type: string
+                        state:
+                          enum:
+                            - closing
+                            - closed
+                            - opened
+                            - failed
+                            - opening
+                          type: string
+                      required:
+                        - job_id
+                        - state
+                    type: array
+                  ml_module_setup_status:
+                    enum:
+                      - complete
+                      - incomplete
+                    type: string
+                  package_installation_status:
+                    enum:
+                      - complete
+                      - incomplete
+                    type: string
+                required:
+                  - package_installation_status
+                  - ml_module_setup_status
+                  - jobs
+          description: Privileged access detection status retrieved
+      summary: Gets the status of the privileged access detection package for the Entity Analytics privileged user monitoring experience
+      tags:
+        - Security Entity Analytics API
   /api/entity_store/enable:
     post:
       operationId: InitEntityStore