kibana/test
Kibana Machine f0ec68a365
[8.x] Add support for GeoIP processor databases in Ingest Pipelines (#190830) (#196420)
# Backport

This will backport the following commits from `main` to `8.x`:
- [Add support for GeoIP processor databases in Ingest Pipelines
(#190830)](https://github.com/elastic/kibana/pull/190830)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Yulia
Čech","email":"6585477+yuliacech@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-15T17:58:43Z","message":"Add
support for GeoIP processor databases in Ingest Pipelines
(#190830)\n\nFixes
https://github.com/elastic/kibana/issues/190818\r\n\r\n##
Summary\r\n\r\nElasticsearch has added support for GeoIP, enabling the
use of paid\r\nGeoIP databases from MaxMind/IPInfo for more accurate and
granular\r\ngeolocation data. As such we should add support to ingest
pipelines UI\r\nfor making this available to the user.\r\n\r\n\r\n* If
the user doesn't have enough privileges, the \"Manage
Pipelines\"\r\nlink and UI won't show.\r\n* Users can add two types of
databases through the UI: MaxMind and\r\nIPinfo. Database names are
predefined by ES, and the user cannot enter\r\ntheir own.\r\n* Certain
types of databases (local and web) can be configured through\r\nES, and
these will appear in the UI, but they cannot be deleted as they\r\nare
read-only.\r\n* When configuring a `IP location` processor, the database
field will\r\ndisplay a list of available and configured databases that
the user can\r\nselect. It also allows for free-text input if the user
wants to\r\nconfigure a database that does not yet exist.\r\n* The new
IP location processor is essentially a clone of the GeoIP\r\nprocessor,
which we are moving away from due to copyright issues.\r\nHowever, it
was decided that GeoIP will remain as is for backward\r\ncompatibility,
and all new work will only be added to IP location
going\r\nforward.\r\n* I left a few mocks in the
`server/routes/api/geoip_database/list.ts `\r\nto try `local/web`
types\r\n\r\n## Release note\r\nThe Ingest Pipelines app now supports
adding and managing databases for\r\nthe GeoIP processor. Additionally,
the pipeline creation flow now\r\nincludes support for the IP Location
processor.\r\n\r\n<details>\r\n<summary>Screenshots</summary>\r\n\r\n![Screenshot
2024-10-07 at 09
36\r\n31](https://github.com/user-attachments/assets/60d438cc-6658-4475-bd27-036c7d13d496)\r\n![Screenshot
2024-10-07 at 09
38\r\n58](https://github.com/user-attachments/assets/7c08e94f-b35c-4e78-a204-1fb456d88181)\r\n![Screenshot
2024-10-07 at 09
47\r\n08](https://github.com/user-attachments/assets/2baca0bd-811d-4dd5-9eb6-9b3f41579249)\r\n![Screenshot
2024-10-07 at 09
47\r\n20](https://github.com/user-attachments/assets/74d8664c-8c73-41f3-8cd5-e0670f3ada77)\r\n![Screenshot
2024-10-07 at 09
48\r\n19](https://github.com/user-attachments/assets/9fb4c186-6224-404c-a8d6-5c44c14da951)\r\n![Screenshot
2024-10-07 at 09
48\r\n25](https://github.com/user-attachments/assets/07e4909d-2613-45aa-918b-11a189e14f6f)\r\n\r\n\r\n</details>\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>\r\nCo-authored-by: Ignacio
Rivas <rivasign@gmail.com>\r\nCo-authored-by: Elena Stoeva
<elenastoeva99@gmail.com>\r\nCo-authored-by: Elena Stoeva
<59341489+ElenaStoeva@users.noreply.github.com>\r\nCo-authored-by:
Matthew Kime
<matt@mattki.me>","sha":"302ac0d336feb861522c9ca3f3c271e172b86ae9","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Kibana
Management","v9.0.0","Feature:Ingest Node
Pipelines","release_note:feature","backport:prev-minor","ci:cloud-deploy"],"title":"Add
support for GeoIP processor databases in Ingest
Pipelines","number":190830,"url":"https://github.com/elastic/kibana/pull/190830","mergeCommit":{"message":"Add
support for GeoIP processor databases in Ingest Pipelines
(#190830)\n\nFixes
https://github.com/elastic/kibana/issues/190818\r\n\r\n##
Summary\r\n\r\nElasticsearch has added support for GeoIP, enabling the
use of paid\r\nGeoIP databases from MaxMind/IPInfo for more accurate and
granular\r\ngeolocation data. As such we should add support to ingest
pipelines UI\r\nfor making this available to the user.\r\n\r\n\r\n* If
the user doesn't have enough privileges, the \"Manage
Pipelines\"\r\nlink and UI won't show.\r\n* Users can add two types of
databases through the UI: MaxMind and\r\nIPinfo. Database names are
predefined by ES, and the user cannot enter\r\ntheir own.\r\n* Certain
types of databases (local and web) can be configured through\r\nES, and
these will appear in the UI, but they cannot be deleted as they\r\nare
read-only.\r\n* When configuring a `IP location` processor, the database
field will\r\ndisplay a list of available and configured databases that
the user can\r\nselect. It also allows for free-text input if the user
wants to\r\nconfigure a database that does not yet exist.\r\n* The new
IP location processor is essentially a clone of the GeoIP\r\nprocessor,
which we are moving away from due to copyright issues.\r\nHowever, it
was decided that GeoIP will remain as is for backward\r\ncompatibility,
and all new work will only be added to IP location
going\r\nforward.\r\n* I left a few mocks in the
`server/routes/api/geoip_database/list.ts `\r\nto try `local/web`
types\r\n\r\n## Release note\r\nThe Ingest Pipelines app now supports
adding and managing databases for\r\nthe GeoIP processor. Additionally,
the pipeline creation flow now\r\nincludes support for the IP Location
processor.\r\n\r\n<details>\r\n<summary>Screenshots</summary>\r\n\r\n![Screenshot
2024-10-07 at 09
36\r\n31](https://github.com/user-attachments/assets/60d438cc-6658-4475-bd27-036c7d13d496)\r\n![Screenshot
2024-10-07 at 09
38\r\n58](https://github.com/user-attachments/assets/7c08e94f-b35c-4e78-a204-1fb456d88181)\r\n![Screenshot
2024-10-07 at 09
47\r\n08](https://github.com/user-attachments/assets/2baca0bd-811d-4dd5-9eb6-9b3f41579249)\r\n![Screenshot
2024-10-07 at 09
47\r\n20](https://github.com/user-attachments/assets/74d8664c-8c73-41f3-8cd5-e0670f3ada77)\r\n![Screenshot
2024-10-07 at 09
48\r\n19](https://github.com/user-attachments/assets/9fb4c186-6224-404c-a8d6-5c44c14da951)\r\n![Screenshot
2024-10-07 at 09
48\r\n25](https://github.com/user-attachments/assets/07e4909d-2613-45aa-918b-11a189e14f6f)\r\n\r\n\r\n</details>\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>\r\nCo-authored-by: Ignacio
Rivas <rivasign@gmail.com>\r\nCo-authored-by: Elena Stoeva
<elenastoeva99@gmail.com>\r\nCo-authored-by: Elena Stoeva
<59341489+ElenaStoeva@users.noreply.github.com>\r\nCo-authored-by:
Matthew Kime
<matt@mattki.me>","sha":"302ac0d336feb861522c9ca3f3c271e172b86ae9"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/190830","number":190830,"mergeCommit":{"message":"Add
support for GeoIP processor databases in Ingest Pipelines
(#190830)\n\nFixes
https://github.com/elastic/kibana/issues/190818\r\n\r\n##
Summary\r\n\r\nElasticsearch has added support for GeoIP, enabling the
use of paid\r\nGeoIP databases from MaxMind/IPInfo for more accurate and
granular\r\ngeolocation data. As such we should add support to ingest
pipelines UI\r\nfor making this available to the user.\r\n\r\n\r\n* If
the user doesn't have enough privileges, the \"Manage
Pipelines\"\r\nlink and UI won't show.\r\n* Users can add two types of
databases through the UI: MaxMind and\r\nIPinfo. Database names are
predefined by ES, and the user cannot enter\r\ntheir own.\r\n* Certain
types of databases (local and web) can be configured through\r\nES, and
these will appear in the UI, but they cannot be deleted as they\r\nare
read-only.\r\n* When configuring a `IP location` processor, the database
field will\r\ndisplay a list of available and configured databases that
the user can\r\nselect. It also allows for free-text input if the user
wants to\r\nconfigure a database that does not yet exist.\r\n* The new
IP location processor is essentially a clone of the GeoIP\r\nprocessor,
which we are moving away from due to copyright issues.\r\nHowever, it
was decided that GeoIP will remain as is for backward\r\ncompatibility,
and all new work will only be added to IP location
going\r\nforward.\r\n* I left a few mocks in the
`server/routes/api/geoip_database/list.ts `\r\nto try `local/web`
types\r\n\r\n## Release note\r\nThe Ingest Pipelines app now supports
adding and managing databases for\r\nthe GeoIP processor. Additionally,
the pipeline creation flow now\r\nincludes support for the IP Location
processor.\r\n\r\n<details>\r\n<summary>Screenshots</summary>\r\n\r\n![Screenshot
2024-10-07 at 09
36\r\n31](https://github.com/user-attachments/assets/60d438cc-6658-4475-bd27-036c7d13d496)\r\n![Screenshot
2024-10-07 at 09
38\r\n58](https://github.com/user-attachments/assets/7c08e94f-b35c-4e78-a204-1fb456d88181)\r\n![Screenshot
2024-10-07 at 09
47\r\n08](https://github.com/user-attachments/assets/2baca0bd-811d-4dd5-9eb6-9b3f41579249)\r\n![Screenshot
2024-10-07 at 09
47\r\n20](https://github.com/user-attachments/assets/74d8664c-8c73-41f3-8cd5-e0670f3ada77)\r\n![Screenshot
2024-10-07 at 09
48\r\n19](https://github.com/user-attachments/assets/9fb4c186-6224-404c-a8d6-5c44c14da951)\r\n![Screenshot
2024-10-07 at 09
48\r\n25](https://github.com/user-attachments/assets/07e4909d-2613-45aa-918b-11a189e14f6f)\r\n\r\n\r\n</details>\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>\r\nCo-authored-by: Ignacio
Rivas <rivasign@gmail.com>\r\nCo-authored-by: Elena Stoeva
<elenastoeva99@gmail.com>\r\nCo-authored-by: Elena Stoeva
<59341489+ElenaStoeva@users.noreply.github.com>\r\nCo-authored-by:
Matthew Kime
<matt@mattki.me>","sha":"302ac0d336feb861522c9ca3f3c271e172b86ae9"}}]}]
BACKPORT-->

Co-authored-by: Elena Stoeva <elenastoeva99@gmail.com>
2024-10-15 17:06:15 -05:00
..
accessibility [8.x] [Console] UX Improvements for phase 2 (#190698) (#193169) 2024-09-18 01:22:58 -05:00
analytics [http] api_integration tests handle internal route restriction (#192407) 2024-09-12 09:23:10 +02:00
api_integration [8.x] Update dependency @types/lodash to ^4.17.10 (main) (#194739) (#196234) 2024-10-15 04:11:15 -05:00
common Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
examples [Dashboard] Remove library and drilldown notifications (#190797) 2024-09-09 14:11:15 -07:00
functional [8.x] [Stateful sidenav] Feedback button (#195751) (#195933) 2024-10-11 11:16:50 -05:00
harden Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
health_gateway Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
interactive_setup_api_integration FIPS FTR Overrides and test skips (#192053) 2024-09-10 13:17:40 -04:00
interactive_setup_functional [8.x] chore(security, eslint): apply platform security-specific rules to all exclusively owned paths. (#195711) (#196010) 2024-10-12 11:19:03 -04:00
interpreter_functional Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
node_roles_functional Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
package [CI / FIPS] Convert smoke tests to use FIPS agent (#186858) 2024-06-26 12:55:51 -07:00
plugin_functional [8.x] Add support for GeoIP processor databases in Ingest Pipelines (#190830) (#196420) 2024-10-15 17:06:15 -05:00
scripts [ci] Remove jenkins config (#173745) 2023-12-21 12:09:43 -06:00
server_integration [http] api_integration tests handle internal route restriction (#192407) 2024-09-12 09:23:10 +02:00
ui_capabilities/newsfeed_err Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana.jsonc [CODEOWNERS] fix appex-qa ownership (#189602) 2024-07-31 14:20:47 +02:00
tsconfig.json [8.x] [LogsUI] Add UI setting to hide Logs Stream and dashboard panel option (#194519) (#195542) 2024-10-09 12:05:38 +02:00