What is a Redis/Valkey Cluster

When a single Redis or Valkey instance cannot handle the required data volume or throughput, Cluster mode distributes data across multiple nodes. Instead of keeping everything on a single server, the cluster partitions the keyspace into 16384 hash slots and distributes those slots among available nodes.

Each node in the cluster is responsible for a subset of slots. When a client executes a command, the cluster calculates the key’s hash slot and directs the operation to the correct node. If the client sends the command to the wrong node, it receives a MOVED redirect with the correct node’s address.

Typical distribution with 3 nodes:

NodeSlots
Node A0-5460
Node B5461-10922
Node C10923-16383

This distribution can be rebalanced as nodes are added or removed. The CLUSTER SLOTS or CLUSTER SHARDS command lets you query the current distribution.

Hash Slots: how the 16384 slots work

A key’s hash slot is determined by the least significant 14 bits of the CRC16 of the key:

slot = CRC16(key) mod 16384

This calculation is deterministic: the same key always maps to the same slot, and therefore the same node. Smart clients (such as redis-cli --cluster, Lettuce, or Jedis in cluster mode) compute the slot locally and send the command directly to the correct node, avoiding unnecessary redirects.

The total of 16384 slots is a fixed value in the specification. Whether the cluster has 3 or 100 nodes, the slots are always 16384 — just redistributed.

The CROSSSLOT problem

In a common scenario, different keys belonging to the same user end up in different slots because CRC16 is computed over the entire key:

SET user:1000:profile "Joao"      # slot 8421 -> Node B
SET user:1000:orders "[...]"      # slot 12657 -> Node C
SET user:1000:settings "{...}"    # slot 3891 -> Node A

Accessing these keys individually works without issues. The error appears when attempting multi-key operations:

MGET user:1000:profile user:1000:orders user:1000:settings
# ERROR: CROSSSLOT Keys in request don't hash to the same slot

The cluster cannot execute a single atomic operation across different nodes. Affected commands:

  • MGET, MSET, DEL with multiple keys
  • Transactions with MULTI/EXEC
  • Lua scripts accessing multiple keys
  • Pipelines with keys in different slots

The CROSSSLOT error is an architectural constraint: the cluster has no distributed transaction coordinator across nodes.

HashTags: the solution with {prefix}

Redis and Valkey provide a mechanism called HashTags. If a key contains the pattern {...}, only the content between the first {} braces is used to compute the hash slot:

SET {user:1000}:profile "Joao"      # slot 7542 -> Node B
SET {user:1000}:orders "[...]"      # slot 7542 -> Node B
SET {user:1000}:settings "{...}"    # slot 7542 -> Node B

All keys with the same HashTag {user:1000} land on the same slot. Multi-key operations now work:

MGET {user:1000}:profile {user:1000}:orders {user:1000}:settings
# OK - all keys are in slot 7542

Pipelines and transactions also work correctly, since the cluster resolves everything on a single node.

The parsing rule is straightforward: the cluster looks for the first occurrence of { and the first occurrence of } after it. If the content between them is not empty, it is used as the CRC16 input. If {} is empty or does not exist, the entire key is used.

When to use HashTags and when to avoid them

HashTags solve the CROSSSLOT problem but introduce risks:

Use when:

  • You need multi-key operations on related data (profile + orders + settings for a user)
  • You need atomic transactions across related keys
  • Lua scripts that manipulate multiple keys from the same context

Avoid when:

  • Keys don’t need to be accessed together (individual GET/SET operations don’t need HashTags)
  • The HashTag has low cardinality (e.g., {orders} would place ALL orders on the same node)
  • An entity concentrates a disproportionate volume of data — this creates hot keys and overloads a single node while others remain idle

Monitor the distribution:

CLUSTER SLOTS
CLUSTER SHARDS

If a node is consistently more loaded than others, it may indicate HashTags with too few distinct values. Prefer tags with high cardinality, such as user or tenant IDs.

Valkey vs Redis

Since 2024, Valkey is the open-source fork of Redis, maintained by the Linux Foundation. It emerged after Redis changed its license (from BSD to dual-license with SSPL/RSALv2). Valkey maintains full compatibility with the Redis protocol and commands, including all the cluster behavior, hash slots, and HashTags described here. If you already use Redis Cluster, migrating to Valkey is transparent in terms of protocol and commands.

References