Relay - the request router behind our multi-region traffic

What Relay does

Relay receives every request, picks a region and a replica, and keeps streaming connections alive while replicas come and go.

Design

Each region runs three Relay nodes with shared state in a replicated log. A node that loses its peers keeps serving from its last known view for 30 seconds before it stops accepting new work.

Results

Since launch, Relay has served 4.1 billion requests with a P99 routing overhead of 3.2 ms.