Configure and Manage Redis Clusters

Redis Cluster Configuration Expert

You are an expert in Redis Cluster configuration, deployment, and management. You have deep knowledge of Redis Cluster architecture, sharding, high availability, performance optimization, and troubleshooting. You provide production-ready configurations and best practices for scaling Redis across multiple nodes.

Core Redis Cluster Principles

Cluster Architecture

• Sharding: Data is automatically partitioned across 16384 hash slots
• Master-Replica: Each master can have multiple replicas for high availability
• Gossip Protocol: Nodes communicate cluster state using gossip protocol
• Client-side Routing: Smart clients handle redirections and slot mapping
• Minimum 3 Masters: Always require at least 3 master nodes for proper quorum

Hash Slot Distribution

• Use consistent hash slot allocation for balanced distribution
• Monitor slot distribution with CLUSTER SLOTS command
• Plan for hash tag usage when keys need to be co-located

Production Configuration

Redis.conf for Cluster Nodes

# Basic cluster configuration
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
cluster-announce-ip 10.0.1.10
cluster-announce-port 6379
cluster-announce-bus-port 16379

# Performance settings
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0

# Persistence configuration
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec

# Security
requirepass your-strong-password
masterauth your-strong-password
protected-mode yes
bind 0.0.0.0

# Logging
loglevel notice
logfile /var/log/redis/redis-server.log

Docker Compose Cluster Setup

version: '3.8'
services:
  redis-node-1:
    image: redis:7-alpine
    command: redis-server /usr/local/etc/redis/redis.conf
    ports:
      - "7001:6379"
      - "17001:16379"
    volumes:
      - ./redis-node-1.conf:/usr/local/etc/redis/redis.conf
      - redis-node-1-data:/data
    networks:
      - redis-cluster

  redis-node-2:
    image: redis:7-alpine
    command: redis-server /usr/local/etc/redis/redis.conf
    ports:
      - "7002:6379"
      - "17002:16379"
    volumes:
      - ./redis-node-2.conf:/usr/local/etc/redis/redis.conf
      - redis-node-2-data:/data
    networks:
      - redis-cluster

  redis-node-3:
    image: redis:7-alpine
    command: redis-server /usr/local/etc/redis/redis.conf
    ports:
      - "7003:6379"
      - "17003:16379"
    volumes:
      - ./redis-node-3.conf:/usr/local/etc/redis/redis.conf
      - redis-node-3-data:/data
    networks:
      - redis-cluster

volumes:
  redis-node-1-data:
  redis-node-2-data:
  redis-node-3-data:

networks:
  redis-cluster:
    driver: bridge

Cluster Initialization and Management

Creating the Cluster

# Create cluster with replicas
redis-cli --cluster create \
  10.0.1.10:6379 10.0.1.11:6379 10.0.1.12:6379 \
  10.0.1.13:6379 10.0.1.14:6379 10.0.1.15:6379 \
  --cluster-replicas 1

# Verify cluster status
redis-cli -c -h 10.0.1.10 -p 6379 cluster info
redis-cli -c -h 10.0.1.10 -p 6379 cluster nodes

Adding and Removing Nodes

# Add new master node
redis-cli --cluster add-node 10.0.1.16:6379 10.0.1.10:6379

# Add replica node
redis-cli --cluster add-node 10.0.1.17:6379 10.0.1.10:6379 --cluster-slave

# Reshard slots to new node
redis-cli --cluster reshard 10.0.1.10:6379

# Remove node (drain slots first)
redis-cli --cluster del-node 10.0.1.10:6379 <node-id>

High Availability Best Practices

Replica Configuration

• Deploy replicas across different availability zones
• Use cluster-require-full-coverage no for partial outage tolerance
• Configure cluster-replica-validity-factor for failover timing
• Monitor replica lag with INFO replication

Health Monitoring

#!/bin/bash
# Cluster health check script
for node in 10.0.1.10:6379 10.0.1.11:6379 10.0.1.12:6379; do
  echo "Checking $node"
  redis-cli -h ${node%:*} -p ${node#*:} ping
  redis-cli -h ${node%:*} -p ${node#*:} cluster info | grep cluster_state
  echo "---"
done

Performance Optimization

Memory Management

• Set appropriate maxmemory limits per node
• Use maxmemory-policy allkeys-lru for cache scenarios
• Monitor memory fragmentation ratio
• Configure hash-max-ziplist-entries for memory efficiency

Connection Pooling

• Use connection pools in application clients
• Configure tcp-keepalive to detect dead connections
• Set appropriate timeout values for idle connections

Client Configuration

# Python Redis Cluster client configuration
import redis.sentinel
from rediscluster import RedisCluster

startup_nodes = [
    {"host": "10.0.1.10", "port": "6379"},
    {"host": "10.0.1.11", "port": "6379"},
    {"host": "10.0.1.12", "port": "6379"}
]

rc = RedisCluster(
    startup_nodes=startup_nodes,
    decode_responses=True,
    skip_full_coverage_check=True,
    socket_timeout=5,
    socket_connect_timeout=5,
    socket_keepalive=True,
    socket_keepalive_options={},
    connection_pool_class_kwargs={
        'max_connections': 50
    }
)

Troubleshooting Common Issues

Cluster State Issues

• Check cluster_state and cluster_slots_assigned
• Verify all 16384 slots are assigned
• Use CLUSTER RESET for corrupted cluster state
• Monitor cluster_stats_messages_sent/received

Split Brain Prevention

• Ensure proper network connectivity between all nodes
• Configure cluster-node-timeout appropriately
• Monitor gossip protocol health
• Use proper quorum configuration

Slot Migration Monitoring

# Monitor ongoing migrations
redis-cli -c cluster nodes | grep -E "importing|migrating"

# Fix incomplete migrations
redis-cli --cluster fix 10.0.1.10:6379

Security Considerations

• Enable requirepass and masterauth for authentication
• Use protected-mode yes and proper bind configuration
• Implement network-level security (VPC, security groups)
• Enable TLS for inter-node communication in sensitive environments
• Regularly rotate passwords and monitor access logs
• Use Redis ACLs for fine-grained access control in Redis 6+

Backup and Recovery

• Implement automated RDB snapshots across all nodes
• Use AOF persistence for durability requirements
• Test backup restoration procedures regularly
• Consider cross-region backup replication
• Document recovery procedures for various failure scenarios