We will deprecate the `_source.mode` mapping level configuration
in favor of the index-level `index.mapping.source.mode` setting.
As a result, we go through the documentation and update it to reflect
the introduction of the setting.
(cherry picked from commit f6a1e36d6b)
* Adds example plugin for custom ingest processor
Adds an example for creating a plugin with a simple custom ingest
processor. The example processor repeats the value of an expected filed
in a document, or ignores it if the expected field does not exist.
Closes#111539
This PR adds synthetic source support for annotated_text fields. Existing implementation for text is reused including test infrastructure so the majority of the change is moving and making things accessible.
Contributes to #106460, #78744.
Problem: The current install docs contain several `ifeval` statements that hide commands and code snippets based on the branch's release state. These statements make the Asciidoc hard to read and maintain. It also makes doc changes difficult to preview.
Solution: Remove `ifeval` statements that hide commands or code snippets. Leave in any `ifeval` statements used to add warnings.
Allows users of the HDFS repository plugin to configure the replication factor for files created by the repository implementation at repository creation time.
---------
Co-authored-by: James Baiera <james.baiera@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
C샤프 in English is C#, not C++.
(C++ in Korean is C플플, C쁠쁠 or C플러스플러스.)
The translation doesn't make sense, so I changed C++ to C#.
It might be true that the writer used C샤프 as just an independent example, regardless C++, but I think it is better to align them for better understanding.
If the recovery node bandwidth settings exist,
then the default value for max snapshot speed will
be infinite, and the speed will be rate limited
by the recovery rate limit as well.
Fixes#57023
This PR updates the URLs for several references that are being
used in the plugin document.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The installation docs for azure discovery suggest installing
Elasticsearch, and then verifying it is running. However, installation
packages do not generally automatically start an installed service. For
example, with systemd, once a package is installed it must be enabled
and/or started explicitly. This commit removes wording about verifying
Elasticsearch is running, as well as adjusts later docs about restarting
it.
The build_flavor was previously removed since it is no longer relevant;
only the default distribution now exists. However, the removal of build
flavor included removing it from the version information on the info
response for the root path. This API is supposed to be stable, so
removing that key was a compatibility break. This commit adds the
build_flavor back to that API, hardcoded to `default`. Additionally, a
test is added to ensure the key exists going forward, until it can be
properly deprecated.
closes#88318
The ingest attachment processor is currently available as a plugin. This
commit moves the processor to the default distribution so it is always
available.
The plugin author docs refer to a few example plugins, yet these are
just a few of the existing examples. While an overhaul is needed of
these docs, this commit slightly improves the situation by making it
clear that these are not the only examples, and linking to the entire
examples directory. Additionally, the wording is tweaked to remove the
superfluous use of "java plugin" because there is no longer a non java
plugin as there was in 1.x.
Tika 1.x is end of life as of later this year. This change updates the
AttachmentProcessor to use tika 2. The goal was to keep the
functionality as close as possible, just with upgraded tika. The tests
have been slightly modified because of a small change in tika
functionality -- as of 2.4.0 it now adds an extra newline to the output
for every embedded attachment in a document. Also as part of this I have
broken apart the tika-parsers into individual dependencies. The reason
is that we are considering breaking this plugin apart, and want to know
exactly which parsers we pull in.
The default distribution is the only remaining build flavor, and has been for
quite a while now. This commit removes flavor from the internal Build
class. It keeps rest api compat for nodes info for now by hardcoding
`default`.
With https://github.com/elastic/elasticsearch/pull/81870, the Azure, GCS, and S3 repository types have separate, dedicated pages in the Elasticsearch guide. For consistency, this PR creates separate pages for the shared file system, read-only URL, and source-only repository types.
Related changes:
- Adds redirects to the plugins docs
- Fixes a few breaking changes that refer to the Azure, GCS, and S3 repositories as plugins.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Closes#81652.
Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
Until now, we have been extracted a few number of fields from the binary files sent to the ingest attachment plugin:
* `content`,
* `title`,
* `author`,
* `keywords`,
* `date`,
* `content_type`,
* `content_length`,
* `language`.
Tika has a list of more standard properties which can be extracted:
* `modified`,
* `format`,
* `identifier`,
* `contributor`,
* `coverage`,
* `modifier`,
* `creator_tool`,
* `publisher`,
* `relation`,
* `rights`,
* `source`,
* `type`,
* `description`,
* `print_date`,
* `metadata_date`,
* `latitude`,
* `longitude`,
* `altitude`,
* `rating`,
* `comments`
This commit exposes those new fields.
Related to #22339.
Co-authored-by: Keith Massey <keith.massey@elastic.co>
Makes several changes to consolidate snapshot and backup-related docs.
Highlights:
* Adds info about supported ESS snapshot repository types
* Adds docs for Kibana's Snapshot and Restore feature
* Combines tutorial pages related to taking and managing snapshots
* Consolidates explanations of the snapshot process
* Incorporates SLM into the snapshot tutorial
* Removes duplicate "back up a cluster" pages
Keeping the binary as a field within the document might consume a lot of resources. It is highly recommended to remove that field from the document, which can be done by adding a `remove` processor in th pipeline.
This commit allows an easier way of doing this by adding a new option `remove_binary`.
When set to `true`, it removes automatically the field at the end of the processor execution.
It defaults to `false` to keep the existing behavior and not introduce any breaking change.
Today it's not clear how to tell Elasticsearch to trust an S3-compatible
repository that presents a certificate issued by a private or
nonstandard CA. This commit expands the docs to say how.
Supersedes #65034
Relates #77081
Co-authored-by: Joost De Cock <joost@decock.org>
Co-authored-by: Joost De Cock <joost@decock.org>
* Add note on cleanup of S3 multipart uploads
Adds docs describing Elasticsearch's behaviour around leaking multipart uploads
and what to do about it.
Closes#44955
* instance -> example
The quota aware filesystem was added as a means of allowing
Elasticsearch to track the used space of the underlying filesystem in
virtualized environments. However, the need for it was due to a bug in a
much earlier version of Elasticsearch that always found the underlying
mount and checked it directely for usage. That bug has already been
fixed, so the there is no longer a need for this plugin. This commit
removes the plugin. We should consider separately whether there is still
a need for bootstrap plugins.
closes#70309
SimpleFS is deprecated and will be removed in Lucene 9. This commit
deprecates SimpleFS in 7.x and uses NIOFS for SimpleFS in Elasticsearch
7.15 or later as it offers superior or equivalent performance to
SimpleFS.
There are some changes in GCP. The service accounts are now their own button and not a child under the IAM/permissions anymore.
Co-authored-by: Philipp Kahr <philipp.kahr@elastic.co>
Azure Storage accounts offer several storage services including Blob Storage, Table Storage, File Storage, and Storage Queues. The intro page for this plugin should specify which type is used for elasticsearch snapshots. This info is necessary for pricing at very least.
Co-authored-by: joshschmitter <45405518+joshschmitter@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
