Distributed Cache (eg for twitter)
A cache is storage space for access to data kept in database. Application server instead of going to DB will check cache and in case of Cache Hit will return result
Requirements
Functional
1. Low latency: Cache instead of database (less than 5ms)
2. Automatic cache population & eviction
3. cache consistency: Updates should update cache across all cache
nodes.
Non Functional
1. High Availability: Cache cluster should survive node/zone
failures.
2. Horizontal Scalability: Add cache nodes without downtime using
consistent hashing.
3. High Throughput: Handle millions of QPS with sub-5ms response time.
Architecture
Meaning of Distributed in Cache <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< VERY IMPORTANT
Distributed does not mean Region1(redis) will sync data to
Region2(redis).
Redis is a cache—not the source of truth—so regions do not synchronize
cached data with one another; they rebuild cache entries from the
database or other authoritative backend when needed.
"Distributed" means how cache data is partitioned across multiple
Redis nodes (shards) within a 1 region ONLY.
Each region maintains its own independent Redis cluster(Cache), which
caches data for requests served in that region.
Block Diagram
1. No load balancer before Redis Cluster: See Below
2. Application talks to both: Redis for cache lookups. Database on
cache misses or writes.
flowchart TB
Client["Browser / Mobile App"]
GLB["Global Load Balancer"]
RLB["Regional Load Balancer
Cluster"]
RLB1["Regional Load Balancer
Cluster"]
subgraph API_GW["API GW Cluster"]
APIGW["API GW
(pod1)"]
APIGW1["API GW
(pod2)"]
end
subgraph Kubernets_Cluster["Kubernets Cluster"]
APPLB["kubernets LB/Envoy"]
APPLB1["kubernets LB/Envoy"]
end
subgraph APP["AppServer HPA"]
APP1["AppServer Pod 1
Redis_Cluster_Client(Library)"]
APP2["AppServer Pod 2"]
end
subgraph REDIS["Redis Cluster"]
subgraph SHARD1["Shard 1"]
M1["Master(pod1) TCP:6379"]
R1["Replica(pod2)"]
M1 -->|Replication| R1
end
subgraph SHARD2["Shard 2"]
M2["Master(pod3) TCP:6379"]
R2["Replica (pod4)"]
M2 -->|Replication| R2
end
end
DB["Cassandra / ScyllaDB"]
Client --> GLB
GLB --> RLB
GLB --> RLB1
RLB --> APIGW
RLB --> APIGW1
APIGW --> APPLB
APIGW --> APPLB1
APPLB --> APP1
APPLB --> APP2
APP1 -. SET(key, val)
GET(key)
ATOMICALLY run inside Redis POD .-> M1
M1 --> DB
M2 --> DB
User Reads his Timeline
User1 follows alice,bob want to read his timeline
// HTTP Request
GET /api/v1/timeline?limit=20 HTTP/1.1
Host: twitter.com
Authorization: Bearer eyJ...
Accept: application/json
User-Agent: Twitter-iOS
Cookie: session=abc123
// Timeline Response
[
{
"tweetId":991,
"text":"Hello World",
"author":"Alice"
},
{
"tweetId":990,
"text":"Good Morning",
"author":"Bob"
}
]
Sequence Diagram
sequenceDiagram participant User participant Browser participant GLB participant RLB participant Gateway participant Auth participant App participant Redis participant DB User->>Browser: Open Home Timeline Browser->>GLB: GET /api/v1/timeline?limit=1 Note over Browser,GLB: Cookie/JWT included GLB->>RLB: HTTPS Request RLB->>Gateway: Forward Gateway->>Auth: Validate Session Auth-->>Gateway: UserId=100 Gateway->>App: Fetch Timeline(UserId=100) App->>Redis: GET timeline:100 alt Cache Hit Redis-->>App: [991] App-->>Gateway: Tweet 991 else Cache Miss Redis-->>App: MISS App->>DB: Query Latest Tweet DB-->>App: Tweet 991 App->>Redis: SET timeline:100=[991] TTL=60s App-->>Gateway: Tweet 991 end Gateway-->>Browser: HTTP 200 + JSON Browser-->>User: Display "Hello World"
Redis Cluster
Redis Cluster Client Library
See Block Diagram, Application service used REDIS_CLUSTER_CLIENT
library(Lettuce Cluster Client(Java), hiredis cluster client(C++),
redis/go-redis/v9(Go)) to decide on which Redis master node request
should be sent
How redis cluster client learn the topology?
When the application starts, redis cluster client connects to any
redis node(redis-cluster.default.svc:6379) and toplology is returned
(Slots 0-1000 -> 10.0.0.21, Slots 1001-2000 -> 10.0.0.21)
/////////////// Application Server POD Code /////////////////
app() {
GET(timeline:100)
}
Redis Client
GET() {
slot_hash = CRC16("timeline:100") % 16384 //slot_hash = 990
}
Client already knows
Node Slots
Master 1 0-1000
Master 2 1001-2000
/////////////////////////////////////////////////////////////
Hash calculation on Redis
sequenceDiagram participant User box Application Pod participant App as AppSrv participant Client as Redis Cluster Client
Hiredis(C++)
go-redis(Go)
end participant M3 as Redis Master 3
10.0.0.3:6379 participant M1 as Redis Master 1
10.0.0.1:6379 participant M2 as Redis Master 2
10.0.0.2:6379 User->>App: GET /timeline (user=1) Note over App: Create string "timeline:1" App->>Client: GET("timeline:1") Note over Client: Compute CRC16("timeline:1")
CRC16 % 16384 = Hash(12553) Note over Client: Hash Table
Slot Cache
0-5000 → Master1(10.0.0.1:6379)
5001-10000 → Master2(10.0.0.2:6379)
10001-16383 → Master3(10.0.0.3:6379) Client->>M3: GET timeline:1 Note over M3: Hash table (Key: Value)
timeline:100 [991,990]
user:100 {...profile...}
tweet:991 {...tweet JSON...} M3-->>Client: [990,991] Client-->>App: Cached Tweet IDs App-->>User: Timeline Response
Hash Table on Redis Master(Not Redis Client library)
Redis Master Node Hash Table // Stored on Master node
Key Value
--------------------------------------------
timeline:100 [991,990]
timeline:101 [345,567]
user:100 {...profile...}
tweet:991 {...tweet JSON...}
followers:100 [45,88,90]
Region 1 (Redis Cluster)
+----------------------+ +----------------------+ +----------------------+
| Master :6379 | | Master :6379 | | Master :6379 |
| Shard 1 | | Shard 2 | | Shard 3 |
| Slots 0-5460 | | Slots 5461-10922 | | Slots 10923-16383 |
+----------+-----------+ +----------+-----------+ +----------+-----------+
| | |
Replication Replication Replication
| | |
+----------v-----------+ +----------v-----------+ +----------v-----------+
| Replica | | Replica | | Replica |
+----------------------+ +----------------------+ +----------------------+