elasticsearch/docs/reference/indices
Keith Massey 37125f265d
Adding index_template_substitutions to the simulate ingest API (#114128) (#114374)
This adds support for a new `index_template_substitutions` field to the
body of an ingest simulate API request. These substitutions can be used
to change the pipeline(s) used for ingest, or to change the mappings
used for validation. It is similar to the
`component_template_substitutions` added in #113276. Here is an example
that shows both of those usages working together:

```
## First, add a couple of pipelines that set a field to a boolean:
PUT /_ingest/pipeline/foo-pipeline?pretty
{
  "processors": [
    {
      "set": {
        "field": "foo",
        "value": true
      }
    }
  ]
}

PUT /_ingest/pipeline/bar-pipeline?pretty
{
  "processors": [
    {
      "set": {
        "field": "bar",
        "value": true
      }
    }
  ]
}

## Now, create three component templates. One provides a mapping enforces that the only field is "foo"
## and that field is a keyword. The next is similar, but adds a `bar` field. The final one provides a setting
## that makes "foo-pipeline" the default pipeline.
## Remember that the "foo-pipeline" sets the "foo" field to a boolean, so using both of these templates
## together would cause a validation exception. These could be in the same template, but are provided
## separately just so that later we can show how multiple templates can be overridden.
PUT _component_template/mappings_template
{
  "template": {
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "foo": {
          "type": "keyword"
        }
      }
    }
  }
}

PUT _component_template/mappings_template_with_bar
{
    "template": {
      "mappings": {
        "dynamic": "strict",
        "properties": {
          "foo": {
            "type": "keyword"
          },
          "bar": {
            "type": "boolean"
          }
        }
      }
    }
}

PUT _component_template/settings_template
{
  "template": {
    "settings": {
      "index": {
        "default_pipeline": "foo-pipeline"
      }
    }
  }
}

## Here we create an index template  pulling in both of the component templates above
PUT _index_template/template_1
{
  "index_patterns": ["foo*"],
  "composed_of": ["mappings_template", "settings_template"]
}

## We can index a document here to create the index, or not. Either way the simulate call ought to work the same
POST foo-1/_doc
{
  "foo": "FOO"
}

## This will not blow up with validation exceptions because the substitute "index_template_substitutions"
## uses `mappings_template_with_bar`, which adds the bar field.
## And the bar-pipeline is executed rather than the foo-pipeline because the substitute
## "index_template_substitutions" uses a substitute `settings_template`, so the value of "foo"
## does not get set to an invalid type.
POST _ingest/_simulate?pretty&index=foo-1
{
  "docs": [
    {
      "_id": "asdf",
      "_source": {
        "foo": "foo",
        "bar": "bar"
      }
    }
  ],
  "component_template_substitutions": {
    "settings_template": {
      "template": {
        "settings": {
          "index": {
            "default_pipeline": "bar-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "template_1": {
      "index_patterns": ["foo*"],
      "composed_of": ["mappings_template_with_bar", "settings_template"]
    }
  }
}
```
2024-10-09 13:04:23 +11:00
..
apis [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
add-alias.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
alias-exists.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
aliases.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
analyze.asciidoc [DOCS] Add security privileges to index API docs (#68071) 2021-01-28 08:53:10 -05:00
clearcache.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
clone-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
close.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
create-data-stream.asciidoc [DOCS] Refactor data stream setup tutorial (#71074) 2021-03-31 17:28:55 -04:00
create-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
dangling-index-delete.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
dangling-index-import.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
dangling-indices-list.asciidoc [DOCS] Add security privileges to index API docs (#68071) 2021-01-28 08:53:10 -05:00
data-stream-stats.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
delete-alias.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
delete-component-template.asciidoc [DOCS] Align docs to implementation for timeout parameters (#108593) 2024-05-16 13:05:39 +10:00
delete-data-stream.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
delete-index-template-v1.asciidoc [DOCS] Align docs to implementation for timeout parameters (#108593) 2024-05-16 13:05:39 +10:00
delete-index-template.asciidoc [DOCS] Align docs to implementation for timeout parameters (#108593) 2024-05-16 13:05:39 +10:00
delete-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
diskusage.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
downsample-data-stream.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
field-usage-stats.asciidoc [DOCS] Align docs to implementation for timeout parameters (#108593) 2024-05-16 13:05:39 +10:00
flush.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
forcemerge.asciidoc Correct force merge disk space requirements (#111066) 2024-07-19 11:31:05 +03:00
get-alias.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
get-component-template.asciidoc Call out monitor privilege for index and component templates (#106970) 2024-04-01 15:52:27 -06:00
get-data-stream.asciidoc Add 'verbose' flag retrieving maximum_timestamp for get data stream API (#112303) 2024-08-31 03:18:15 +10:00
get-field-mapping.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
get-index-template-v1.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
get-index-template.asciidoc Call out monitor privilege for index and component templates (#106970) 2024-04-01 15:52:27 -06:00
get-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
get-mapping.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
get-settings.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
ignore-missing-component-templates.asciidoc Add ignore_missing_component_templates config option (#92436) 2023-01-31 08:40:29 -07:00
index-mgmt.asciidoc [DOCS] Adds enrich policies and data retention for Index Management (#100922) 2023-10-19 07:24:52 -07:00
index-template-exists-v1.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
index-templates.asciidoc Use deprecation logger for CLDR date format specifiers (#112917) 2024-09-19 15:50:37 +01:00
indices-exists.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
migrate-to-data-stream.asciidoc [DOCS] Update anchor and add redirect for aliases (#77349) 2021-09-07 09:42:42 -04:00
open-close.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
put-component-template.asciidoc Adding component template substitutions to the simulate ingest API (#113276) (#113567) 2024-09-26 07:32:13 +10:00
put-index-template-v1.asciidoc Use deprecation logger for CLDR date format specifiers (#112917) 2024-09-19 15:50:37 +01:00
put-index-template.asciidoc Adding index_template_substitutions to the simulate ingest API (#114128) (#114374) 2024-10-09 13:04:23 +11:00
put-mapping.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
recovery.asciidoc Round up shard allocation / recovery / relocation concepts (#109943) 2024-07-25 14:44:57 -04:00
refresh.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
resolve-cluster.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
resolve.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
rollover-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
segments.asciidoc Add per-field KNN vector format to Index Segments API (#107216) 2024-04-23 10:12:50 +02:00
shard-stores.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
shrink-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
simulate-index.asciidoc Start with data stream lifecycle documentation (#95326) 2023-06-28 16:18:05 +03:00
simulate-multi-component-templates.asciidoc Simulate template api to include settings from IndexSettingProvider (#82609) 2022-01-17 10:22:22 +01:00
simulate-template.asciidoc Use deprecation logger for CLDR date format specifiers (#112917) 2024-09-19 15:50:37 +01:00
split-index.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
stats.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00
synced-flush.asciidoc Goodbye and thank you synced flush! (#50882) 2020-01-16 09:43:07 -05:00
update-settings.asciidoc [DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413) 2024-04-17 14:37:07 +02:00