This PR updates the URLs for several references that are being
used in the plugin document.
Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
The installation docs for azure discovery suggest installing
Elasticsearch, and then verifying it is running. However, installation
packages do not generally automatically start an installed service. For
example, with systemd, once a package is installed it must be enabled
and/or started explicitly. This commit removes wording about verifying
Elasticsearch is running, as well as adjusts later docs about restarting
it.
The build_flavor was previously removed since it is no longer relevant;
only the default distribution now exists. However, the removal of build
flavor included removing it from the version information on the info
response for the root path. This API is supposed to be stable, so
removing that key was a compatibility break. This commit adds the
build_flavor back to that API, hardcoded to `default`. Additionally, a
test is added to ensure the key exists going forward, until it can be
properly deprecated.
closes#88318
The ingest attachment processor is currently available as a plugin. This
commit moves the processor to the default distribution so it is always
available.
The plugin author docs refer to a few example plugins, yet these are
just a few of the existing examples. While an overhaul is needed of
these docs, this commit slightly improves the situation by making it
clear that these are not the only examples, and linking to the entire
examples directory. Additionally, the wording is tweaked to remove the
superfluous use of "java plugin" because there is no longer a non java
plugin as there was in 1.x.
Tika 1.x is end of life as of later this year. This change updates the
AttachmentProcessor to use tika 2. The goal was to keep the
functionality as close as possible, just with upgraded tika. The tests
have been slightly modified because of a small change in tika
functionality -- as of 2.4.0 it now adds an extra newline to the output
for every embedded attachment in a document. Also as part of this I have
broken apart the tika-parsers into individual dependencies. The reason
is that we are considering breaking this plugin apart, and want to know
exactly which parsers we pull in.
The default distribution is the only remaining build flavor, and has been for
quite a while now. This commit removes flavor from the internal Build
class. It keeps rest api compat for nodes info for now by hardcoding
`default`.
With https://github.com/elastic/elasticsearch/pull/81870, the Azure, GCS, and S3 repository types have separate, dedicated pages in the Elasticsearch guide. For consistency, this PR creates separate pages for the shared file system, read-only URL, and source-only repository types.
Related changes:
- Adds redirects to the plugins docs
- Fixes a few breaking changes that refer to the Azure, GCS, and S3 repositories as plugins.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Closes#81652.
Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
Until now, we have been extracted a few number of fields from the binary files sent to the ingest attachment plugin:
* `content`,
* `title`,
* `author`,
* `keywords`,
* `date`,
* `content_type`,
* `content_length`,
* `language`.
Tika has a list of more standard properties which can be extracted:
* `modified`,
* `format`,
* `identifier`,
* `contributor`,
* `coverage`,
* `modifier`,
* `creator_tool`,
* `publisher`,
* `relation`,
* `rights`,
* `source`,
* `type`,
* `description`,
* `print_date`,
* `metadata_date`,
* `latitude`,
* `longitude`,
* `altitude`,
* `rating`,
* `comments`
This commit exposes those new fields.
Related to #22339.
Co-authored-by: Keith Massey <keith.massey@elastic.co>
Makes several changes to consolidate snapshot and backup-related docs.
Highlights:
* Adds info about supported ESS snapshot repository types
* Adds docs for Kibana's Snapshot and Restore feature
* Combines tutorial pages related to taking and managing snapshots
* Consolidates explanations of the snapshot process
* Incorporates SLM into the snapshot tutorial
* Removes duplicate "back up a cluster" pages
Keeping the binary as a field within the document might consume a lot of resources. It is highly recommended to remove that field from the document, which can be done by adding a `remove` processor in th pipeline.
This commit allows an easier way of doing this by adding a new option `remove_binary`.
When set to `true`, it removes automatically the field at the end of the processor execution.
It defaults to `false` to keep the existing behavior and not introduce any breaking change.
Today it's not clear how to tell Elasticsearch to trust an S3-compatible
repository that presents a certificate issued by a private or
nonstandard CA. This commit expands the docs to say how.
Supersedes #65034
Relates #77081
Co-authored-by: Joost De Cock <joost@decock.org>
Co-authored-by: Joost De Cock <joost@decock.org>
* Add note on cleanup of S3 multipart uploads
Adds docs describing Elasticsearch's behaviour around leaking multipart uploads
and what to do about it.
Closes#44955
* instance -> example
The quota aware filesystem was added as a means of allowing
Elasticsearch to track the used space of the underlying filesystem in
virtualized environments. However, the need for it was due to a bug in a
much earlier version of Elasticsearch that always found the underlying
mount and checked it directely for usage. That bug has already been
fixed, so the there is no longer a need for this plugin. This commit
removes the plugin. We should consider separately whether there is still
a need for bootstrap plugins.
closes#70309
SimpleFS is deprecated and will be removed in Lucene 9. This commit
deprecates SimpleFS in 7.x and uses NIOFS for SimpleFS in Elasticsearch
7.15 or later as it offers superior or equivalent performance to
SimpleFS.
There are some changes in GCP. The service accounts are now their own button and not a child under the IAM/permissions anymore.
Co-authored-by: Philipp Kahr <philipp.kahr@elastic.co>
Azure Storage accounts offer several storage services including Blob Storage, Table Storage, File Storage, and Storage Queues. The intro page for this plugin should specify which type is used for elasticsearch snapshots. This info is necessary for pricing at very least.
Co-authored-by: joshschmitter <45405518+joshschmitter@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Improve indentation of code for discovery-gce
Improve the indentation by using a indentation level of two spaces to
improve readability and enable better copy&paste experience.
* Improve docs for GCP web-console and permissions
Match the description for the GCP web-console to the current state
and change the API-permission.
There is (no longer) a permission `compute.full_control`.
* Apply suggestions from code review
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Adds support for "Default Application Credentials" for GCS repositories, making it easier to set up a repository on GCP,
as all relevant information to connect to the repository is retrieved from the environment, not necessitating complicated
keystore setups.
Currently metadata fields like `_size` or `_doc_count` cannot be retrieved using
the fields API. With this change, we allow this if the field is explicitely
queried for using its name, but won't include metadata fields when e.g.
requesting all fields via "*".
With this change, not all metadata fields will be retrievable by using its name,
but support for "_size" and "_doc_count" (which is fetched from source) is
added. Support for other metadata field types will need to be decided case by
case and an appropriate ValueFetcher needs to be supplied.
Relates to #63569
Closes#66476. Add support for removing multiple plugins at the
same time to `elasticsearch-plugin`. Also change references from
"plugin name" to "plugin id", to align better with the installer
class.
We removed the global `repositories.s3.base_path` setting in 6.0 but it
is still mentioned in the docs for the S3 repository plugin. This commit
removes it from the docs.
Relates #24445
the bucket naming convention should be clear, underscores (which we use in all of our examples) are not allowed so we should really change all these examples.
