elasticsearch/docs/reference/connector/docs/connectors-redis.asciidoc

[#es-connectors-redis]
=== Redis connector reference
++++
<titleabbrev>Redis</titleabbrev>
++++

// Attributes (AKA variables) used in this file
:service-name: Redis
:service-name-stub: redis

The Redis connector is built with the Elastic connectors Python framework and is available as a self-managed <<es-build-connector, self-managed connector>>.
View the {connectors-python}/connectors/sources/{service-name-stub}.py[*source code* for this connector^] (branch _{connectors-branch}_, compatible with Elastic _{minor-version}_).

[discrete#es-connectors-redis-connector-availability-and-prerequisites]
==== Availability and prerequisites

This connector was introduced in Elastic *8.13.0*, available as a *self-managed* self-managed connector.

To use this connector, satisfy all <<es-build-connector, self-managed connector prerequisites>>.
Importantly, you must deploy the connectors service on your own infrastructure.
You have two deployment options:

* <<es-connectors-run-from-source, Run connectors service from source>>. Use this option if you're comfortable working with Python and want to iterate quickly locally.
* <<es-connectors-run-from-docker, Run connectors service in Docker>>. Use this option if you want to deploy the connectors to a server, or use a container orchestration platform.

[NOTE]
====
This connector is in *technical preview* and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Technical preview features are not subject to the support SLA of official GA features.
====

[discrete#es-connectors-redis-connector-usage]
==== Usage

To set up this connector in the UI, select the *Redis* tile when creating a new connector under *Search -> Connectors*.

For additional operations, see <<es-connectors-usage>>.

[discrete#es-connectors-redis-connector-docker]
==== Deploy with Docker

include::_connectors-docker-instructions.asciidoc[]

[discrete#es-connectors-redis-connector-configuration]
==== Configuration

`host` (required)::
The IP of your Redis server/cloud. Example:

* `127.0.0.1`
* `redis-12345.us-east-1.ec2.cloud.redislabs.com`

`port` (required)::
Port where the Redis server/cloud instance is hosted. Example:

* `6379`

`username` (optional)::
Username for your Redis server/cloud. Example:

* `default`

`password` (optional)::
Password for your Redis server/cloud instance. Example:

* `changeme`

`database`  (required)::
List of database index for your Redis server/cloud. * will fetch data from all databases. Example:

* `0,1,2`
* `*`
+
[NOTE]
====
This field is ignored when using advanced sync rules.
====

`ssl_enabled`::
Toggle to use SSL/TLS. Disabled by default.

`mutual_tls_enabled`::
Toggle to use secure mutual SSL/TLS. Ensure that your Redis deployment supports mutual SSL/TLS connections. Disabled by default. Depends on `ssl_enabled`.

`tls_certfile`::
Specifies the certificate from the Certificate Authority. The value of the certificate is used to validate the certificate presented by the Redis instance. Depends on `mutual_tls_enabled`.

`tls_keyfile`::
Specifies the client private key. The value of the key is used to validate the connection in the Redis instance.
Depends on `mutual_tls_enabled`.

[discrete#es-connectors-redis-connector-documents-and-syncs]
==== Documents and syncs

The connector syncs the following objects and entities:

* KEYS and VALUES of every database index


[NOTE]
====
* Permissions are not synced. *All documents* indexed to an Elastic deployment will be visible to *all users with access* to the relevant Elasticsearch index.
====

[discrete#es-connectors-redis-connector-sync-rules]
==== Sync rules
<<es-sync-rules-basic,Basic sync rules>> are identical for all connectors and are available by default.


[discrete#es-connectors-redis-connector-advanced-sync-rules]
==== Advanced Sync Rules

<<es-sync-rules-advanced,Advanced sync rules>> are defined through a source-specific DSL JSON snippet.

Use advanced sync rules to filter data at the Redis source, without needing to index all data into Elasticsearch.

They take the following parameters:

1. `database`:  Specify the Redis database index as an integer value.
2. `key_pattern`: 2. `key_pattern`: Pattern for finding keys in Redis.
3. `type`: Key type for the Redis.
+
Supported values: 

* `HASH`
* `LIST`
* `SET`
* `STREAM`
* `STRING`
* `ZSET`

[NOTE]
====
Provide at least one of the following: `key_pattern` or `type`, or both.
====

[discrete#es-connectors-redis-connector-advanced-sync-rules-examples]
===== Advanced sync rules examples

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-1]
====== Example 1

*Fetch database records where keys start with `alpha`*:

[source,js]
----
[
  {
    "database": 0,
    "key_pattern": "alpha*"
  }
]
----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-2]
====== Example 2

*Fetch database records with exact match by specifying the full key name:*

[source,js]
----
[
  {
    "database": 0,
    "key_pattern": "alpha"
  }
]
----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-3]
====== Example 3

*Fetch database records where keys start with `test1`, `test2` or `test3`:*

[source,js]
----
[
  {
    "database": 0,
    "key_pattern": "test[123]"
  }

----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-4]
====== Example 4

*Exclude database records where keys start with `test1`, `test2` or `test3`:*

[source,js]
----
[
  {
    "database": 0,
    "key_pattern": "test[^123]"
  }
]
----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-5]
====== Example 5

*Fetch all database records:*

[source,js]
----
[
  {
    "database": 0,
    "key_pattern": "*"
  }
]
----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-6]
====== Example 6

*Fetch all database records where type is `SET`:*

[source,js]
----
[
  {
    "database": 0,
    "key_pattern": "*",
    "type": "SET"
  }
]
----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-advanced-sync-rules-example-7]
====== Example 7

*Fetch database records where type is `SET`*:

[source,js]
----
[
  {
    "database": 0,
    "type": "SET"
  }
]
----
// NOTCONSOLE

[discrete#es-connectors-redis-connector-connector-client-operations]
==== Connector Client operations

[discrete#es-connectors-redis-connector-end-to-end-testing]
===== End-to-end Testing

The connector framework enables operators to run functional tests against a real data source, using Docker Compose.
You don't need a running Elasticsearch instance or Redis source to run this test.

Refer to <<es-build-connector-testing>> for more details.

To perform E2E testing for the Redis connector, run the following command:

```shell
$ make ftest NAME=redis
```
For faster tests, add the `DATA_SIZE=small` flag:

[source,shell]
----
make ftest NAME=redis DATA_SIZE=small
----

By default, `DATA_SIZE=MEDIUM`.

[discrete#es-connectors-redis-connector-known-issues]
==== Known issues

* The last modified time is unavailable when retrieving keys/values from the Redis database.
As a result, *all objects* are indexed each time an advanced sync rule query is executed.

Refer to <<es-connectors-known-issues>> for a list of known issues for all connectors.

[discrete#es-connectors-redis-connector-troubleshooting]
==== Troubleshooting

See <<es-connectors-troubleshooting>>.

[discrete#es-connectors-redis-connector-security]
==== Security

See <<es-connectors-security>>.