mirror of
https://github.com/elastic/kibana.git
synced 2025-06-27 18:51:07 -04:00
[Entity Analytics][Privmon] CSV upload (#221798)
## Summary This PR adds the ability to upload a CSV file with privileged users from the Entity Analytics pages ## Changes ### Backend - Added (or completed) the upload CSV route: `/api/entity_analytics/monitoring/users/_csv` - Added shared utilities for batching with Node streams - Added bulk processing actions for the upload - Parsing users from CSV - Soft delete for omitted users - Batch upsert via the bulk API - Added a check for installing all required privmon resources ### Frontend - File uploader components - File validation logic - Updated EA privmon page to account for the new flow - Added managing users panels - open upload flow (same as asset criticality) ## Screen recording https://github.com/user-attachments/assets/7956f1cf-49e0-4430-8c23-7d6178a15342 ## How to test #### Prerequisite Make sure you have a CSV file with usernames Check [here](https://gist.github.com/tiansivive/0be2f09e1bb380fdde6609a131e929ed) for a little helper script Create a few copies where some of the users are deleted, in order to test soft delete 1. Start up kibana and ES 2. Navigate to Security > Entity Analytics > Privilege User Monitoring 3. Select the `File` option to add data 4. Add one of the CSV files to the open modal and upload 5. Repeat but now upload one of files with the omitted users Alternatively, testing only the backend only is possible by directly hitting the API wit curl ``` curl -u elastic:changeme \ -X POST "http://localhost:5601/api/entity_analytics/monitoring/users/_csv" \ -H "kbn-xsrf: true" \ -F "file=@test.csv;type=text/csv" ``` #### Verifying Easiest way is to use the dev tools to `_search` the privmon users index with: ``` GET .entity_analytics.monitoring.users-default/_search ``` Look for number of hits and/or use `query` to search for omitted users. ## Remaining work - [x] API integration tests - [ ] Batching logic unit tests - [ ] E2E tests? --------- Co-authored-by: machadoum <pablo.nevesmachado@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
parent
5e28a4200f
commit
a8a7574c66
55 changed files with 3065 additions and 389 deletions
|
@ -11146,58 +11146,48 @@ paths:
|
|||
- Security Entity Analytics API
|
||||
/api/entity_analytics/monitoring/users/_csv:
|
||||
post:
|
||||
operationId: BulkUploadUsersCSV
|
||||
operationId: PrivmonBulkUploadUsersCSV
|
||||
requestBody:
|
||||
content:
|
||||
text/csv:
|
||||
schema:
|
||||
type: string
|
||||
description: CSV file containing users to upsert
|
||||
required: true
|
||||
responses:
|
||||
'200':
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
upserted_count:
|
||||
type: integer
|
||||
description: Successful response
|
||||
summary: Upsert multiple monitored users via CSV upload
|
||||
tags:
|
||||
- Security Entity Analytics API
|
||||
/api/entity_analytics/monitoring/users/_json:
|
||||
post:
|
||||
operationId: BulkUploadUsersJSON
|
||||
requestBody:
|
||||
content:
|
||||
text/json:
|
||||
multipart/form-data:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
users:
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
is_monitored:
|
||||
type: boolean
|
||||
user_name:
|
||||
type: string
|
||||
type: array
|
||||
description: JSON file containing users to upsert
|
||||
required: true
|
||||
file:
|
||||
description: The CSV file to upload.
|
||||
format: binary
|
||||
type: string
|
||||
required:
|
||||
- file
|
||||
responses:
|
||||
'200':
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
example:
|
||||
errors:
|
||||
- index: 1
|
||||
message: Invalid monitored field
|
||||
username: john.doe
|
||||
stats:
|
||||
failed: 1
|
||||
successful: 1
|
||||
total: 2
|
||||
type: object
|
||||
properties:
|
||||
upserted_count:
|
||||
type: integer
|
||||
description: Successful response
|
||||
summary: Upsert multiple monitored users via JSON upload
|
||||
errors:
|
||||
items:
|
||||
$ref: '#/components/schemas/Security_Entity_Analytics_API_PrivmonUserCsvUploadErrorItem'
|
||||
type: array
|
||||
stats:
|
||||
$ref: '#/components/schemas/Security_Entity_Analytics_API_PrivmonUserCsvUploadStats'
|
||||
required:
|
||||
- errors
|
||||
- stats
|
||||
description: Bulk upload successful
|
||||
'413':
|
||||
description: File too large
|
||||
summary: Upsert multiple monitored users via CSV upload
|
||||
tags:
|
||||
- Security Entity Analytics API
|
||||
/api/entity_analytics/monitoring/users/{id}:
|
||||
|
@ -66988,6 +66978,34 @@ components:
|
|||
required:
|
||||
- type
|
||||
- status
|
||||
Security_Entity_Analytics_API_PrivmonUserCsvUploadErrorItem:
|
||||
type: object
|
||||
properties:
|
||||
index:
|
||||
nullable: true
|
||||
type: integer
|
||||
message:
|
||||
type: string
|
||||
username:
|
||||
nullable: true
|
||||
type: string
|
||||
required:
|
||||
- message
|
||||
- index
|
||||
- username
|
||||
Security_Entity_Analytics_API_PrivmonUserCsvUploadStats:
|
||||
type: object
|
||||
properties:
|
||||
failed:
|
||||
type: integer
|
||||
successful:
|
||||
type: integer
|
||||
total:
|
||||
type: integer
|
||||
required:
|
||||
- successful
|
||||
- failed
|
||||
- total
|
||||
Security_Entity_Analytics_API_RiskEngineScheduleNowErrorResponse:
|
||||
type: object
|
||||
properties:
|
||||
|
|
|
@ -13305,58 +13305,48 @@ paths:
|
|||
- Security Entity Analytics API
|
||||
/api/entity_analytics/monitoring/users/_csv:
|
||||
post:
|
||||
operationId: BulkUploadUsersCSV
|
||||
operationId: PrivmonBulkUploadUsersCSV
|
||||
requestBody:
|
||||
content:
|
||||
text/csv:
|
||||
schema:
|
||||
type: string
|
||||
description: CSV file containing users to upsert
|
||||
required: true
|
||||
responses:
|
||||
'200':
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
upserted_count:
|
||||
type: integer
|
||||
description: Successful response
|
||||
summary: Upsert multiple monitored users via CSV upload
|
||||
tags:
|
||||
- Security Entity Analytics API
|
||||
/api/entity_analytics/monitoring/users/_json:
|
||||
post:
|
||||
operationId: BulkUploadUsersJSON
|
||||
requestBody:
|
||||
content:
|
||||
text/json:
|
||||
multipart/form-data:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
users:
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
is_monitored:
|
||||
type: boolean
|
||||
user_name:
|
||||
type: string
|
||||
type: array
|
||||
description: JSON file containing users to upsert
|
||||
required: true
|
||||
file:
|
||||
description: The CSV file to upload.
|
||||
format: binary
|
||||
type: string
|
||||
required:
|
||||
- file
|
||||
responses:
|
||||
'200':
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
example:
|
||||
errors:
|
||||
- index: 1
|
||||
message: Invalid monitored field
|
||||
username: john.doe
|
||||
stats:
|
||||
failed: 1
|
||||
successful: 1
|
||||
total: 2
|
||||
type: object
|
||||
properties:
|
||||
upserted_count:
|
||||
type: integer
|
||||
description: Successful response
|
||||
summary: Upsert multiple monitored users via JSON upload
|
||||
errors:
|
||||
items:
|
||||
$ref: '#/components/schemas/Security_Entity_Analytics_API_PrivmonUserCsvUploadErrorItem'
|
||||
type: array
|
||||
stats:
|
||||
$ref: '#/components/schemas/Security_Entity_Analytics_API_PrivmonUserCsvUploadStats'
|
||||
required:
|
||||
- errors
|
||||
- stats
|
||||
description: Bulk upload successful
|
||||
'413':
|
||||
description: File too large
|
||||
summary: Upsert multiple monitored users via CSV upload
|
||||
tags:
|
||||
- Security Entity Analytics API
|
||||
/api/entity_analytics/monitoring/users/{id}:
|
||||
|
@ -76488,6 +76478,34 @@ components:
|
|||
required:
|
||||
- type
|
||||
- status
|
||||
Security_Entity_Analytics_API_PrivmonUserCsvUploadErrorItem:
|
||||
type: object
|
||||
properties:
|
||||
index:
|
||||
nullable: true
|
||||
type: integer
|
||||
message:
|
||||
type: string
|
||||
username:
|
||||
nullable: true
|
||||
type: string
|
||||
required:
|
||||
- message
|
||||
- index
|
||||
- username
|
||||
Security_Entity_Analytics_API_PrivmonUserCsvUploadStats:
|
||||
type: object
|
||||
properties:
|
||||
failed:
|
||||
type: integer
|
||||
successful:
|
||||
type: integer
|
||||
total:
|
||||
type: integer
|
||||
required:
|
||||
- successful
|
||||
- failed
|
||||
- total
|
||||
Security_Entity_Analytics_API_RiskEngineScheduleNowErrorResponse:
|
||||
type: object
|
||||
properties:
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue