mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 07:37:19 -04:00
155 lines
4.3 KiB
Text
155 lines
4.3 KiB
Text
[[infer-service-google-vertex-ai]]
|
|
=== Google Vertex AI {infer} service
|
|
|
|
Creates an {infer} endpoint to perform an {infer} task with the `googlevertexai` service.
|
|
|
|
|
|
[discrete]
|
|
[[infer-service-google-vertex-ai-api-request]]
|
|
==== {api-request-title}
|
|
|
|
`PUT /_inference/<task_type>/<inference_id>`
|
|
|
|
[discrete]
|
|
[[infer-service-google-vertex-ai-path-params]]
|
|
==== {api-path-parms-title}
|
|
|
|
`<inference_id>`::
|
|
(Required, string)
|
|
include::inference-shared.asciidoc[tag=inference-id]
|
|
|
|
`<task_type>`::
|
|
(Required, string)
|
|
include::inference-shared.asciidoc[tag=task-type]
|
|
+
|
|
--
|
|
Available task types:
|
|
|
|
* `rerank`
|
|
* `text_embedding`.
|
|
--
|
|
|
|
[discrete]
|
|
[[infer-service-google-vertex-ai-api-request-body]]
|
|
==== {api-request-body-title}
|
|
|
|
`chunking_settings`::
|
|
(Optional, object)
|
|
include::inference-shared.asciidoc[tag=chunking-settings]
|
|
|
|
`max_chunking_size`:::
|
|
(Optional, integer)
|
|
include::inference-shared.asciidoc[tag=chunking-settings-max-chunking-size]
|
|
|
|
`overlap`:::
|
|
(Optional, integer)
|
|
include::inference-shared.asciidoc[tag=chunking-settings-overlap]
|
|
|
|
`sentence_overlap`:::
|
|
(Optional, integer)
|
|
include::inference-shared.asciidoc[tag=chunking-settings-sentence-overlap]
|
|
|
|
`strategy`:::
|
|
(Optional, string)
|
|
include::inference-shared.asciidoc[tag=chunking-settings-strategy]
|
|
|
|
`service`::
|
|
(Required, string)
|
|
The type of service supported for the specified task type. In this case,
|
|
`googlevertexai`.
|
|
|
|
`service_settings`::
|
|
(Required, object)
|
|
include::inference-shared.asciidoc[tag=service-settings]
|
|
+
|
|
--
|
|
These settings are specific to the `googlevertexai` service.
|
|
--
|
|
|
|
`service_account_json`:::
|
|
(Required, string)
|
|
A valid service account in json format for the Google Vertex AI API.
|
|
|
|
`model_id`:::
|
|
(Required, string)
|
|
The name of the model to use for the {infer} task.
|
|
You can find the supported models at https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text-embeddings-api[Text embeddings API].
|
|
|
|
`location`:::
|
|
(Required, string)
|
|
The name of the location to use for the {infer} task.
|
|
You find the supported locations at https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations[Generative AI on Vertex AI locations].
|
|
|
|
`project_id`:::
|
|
(Required, string)
|
|
The name of the project to use for the {infer} task.
|
|
|
|
`rate_limit`:::
|
|
(Optional, object)
|
|
By default, the `googlevertexai` service sets the number of requests allowed per minute to `30.000`.
|
|
This helps to minimize the number of rate limit errors returned from Google Vertex AI.
|
|
To modify this, set the `requests_per_minute` setting of this object in your service settings:
|
|
+
|
|
--
|
|
include::inference-shared.asciidoc[tag=request-per-minute-example]
|
|
|
|
More information about the rate limits for Google Vertex AI can be found in the https://cloud.google.com/vertex-ai/docs/quotas[Google Vertex AI Quotas docs].
|
|
--
|
|
|
|
`task_settings`::
|
|
(Optional, object)
|
|
include::inference-shared.asciidoc[tag=task-settings]
|
|
+
|
|
.`task_settings` for the `rerank` task type
|
|
[%collapsible%closed]
|
|
=====
|
|
`top_n`:::
|
|
(optional, boolean)
|
|
Specifies the number of the top n documents, which should be returned.
|
|
=====
|
|
+
|
|
.`task_settings` for the `text_embedding` task type
|
|
[%collapsible%closed]
|
|
=====
|
|
`auto_truncate`:::
|
|
(optional, boolean)
|
|
Specifies if the API truncates inputs longer than the maximum token length automatically.
|
|
=====
|
|
|
|
[discrete]
|
|
[[inference-example-google-vertex-ai]]
|
|
==== Google Vertex AI service example
|
|
|
|
The following example shows how to create an {infer} endpoint called
|
|
`google_vertex_ai_embeddings` to perform a `text_embedding` task type.
|
|
|
|
[source,console]
|
|
------------------------------------------------------------
|
|
PUT _inference/text_embedding/google_vertex_ai_embeddings
|
|
{
|
|
"service": "googlevertexai",
|
|
"service_settings": {
|
|
"service_account_json": "<service_account_json>",
|
|
"model_id": "<model_id>",
|
|
"location": "<location>",
|
|
"project_id": "<project_id>"
|
|
}
|
|
}
|
|
------------------------------------------------------------
|
|
// TEST[skip:TBD]
|
|
|
|
The next example shows how to create an {infer} endpoint called
|
|
`google_vertex_ai_rerank` to perform a `rerank` task type.
|
|
|
|
[source,console]
|
|
------------------------------------------------------------
|
|
PUT _inference/rerank/google_vertex_ai_rerank
|
|
{
|
|
"service": "googlevertexai",
|
|
"service_settings": {
|
|
"service_account_json": "<service_account_json>",
|
|
"project_id": "<project_id>"
|
|
}
|
|
}
|
|
------------------------------------------------------------
|
|
// TEST[skip:TBD]
|