Duet: Cloud Scale Load Balancing with Hardware

25 Pages·2014·0.6 MB·English

by  Rohan Gandhi

Duet: Cloud Scale Load Balancing
with Hardware and Software
Rohan Gandhi Hongqiang Harry Liu Guohan Lu J i t u P a d h ye
Y. Charlie Hu Lihua Yuan Ming Zhang
1
Load Balancer is Critical
For Online Services
Terminology
Internet
VIP – Virtual IP
DIP – Direct IP
Mux – Multiplexer
VIP1, VIP2
Datacenter
Store VIP-DIP mapping
Load balancer (Mux)
Split VIP traffic to DIPs
Server Server Server Server
. . . . . .
DIP = 10.0.1.1 DIP = 10.0.2.2 DIP = 10.0.3.3 DIP = 10.0.4.4
bing.com OneDrive
VIP1 = 1.1.1.1 VIP2 = 1.1.1.2
Load balancer provides high availability and scalability
2
Existing LBs Have Limitations
Specialized Hardware LBs
Too costly Poor robustness
$100+ million for 15 Tbps 1+1 redundancy
Software LBs
Scale with demand High robustness
Scale up/down according to n+1 redundancy
VIP traffic
Software LBs have
cost and performance limitations
3
Software LB Design
DC Network
DIP2
Client SMux DIP1
VIP1 DIP1, DIP2
VIP1
. .
. .
Software LB Benefits:
• High robustness
• Scale with demand
4
Ananta: Cloud-scale load balancing [SIGCOMM’13]
Software LB has Limitations
100
n 10
n
o 80
o
i
t i
alf )c taz 60
n e i
i y sm 1 litU 40
c
n (
U
e
t P 20
a C
L
0.1 0
100k 300k 450k 100k 300k 450k
Traffic (pkts/sec) Traffic (pkts/sec)
High latency inflation: 200 usec Low capacity: 300k pkts/sec
5k SMuxes needed at 15Tbps traffic in 50k server DC
5
How can we build high performance,
low cost and robust load balancer?
Duet ideas
• Use commodity switches as hardware Muxes
• Use software Muxes as a backstop
6
Can Switch Act As a Mux
Switches offer:
• High capacity (500+ million pkts/sec)
• Low latency inflation (1 usec)
Mux functionalities Switch resources
• Split VIP traffic across DIPs • ECMP
• Forward VIP traffic to DIPs • Tunneling
7
Implementing HMux on Switch
VIP DIPs Forwarding Table ECMP Table Tunneling Table
10.0.0.1 Index DIPs
VIP
1.1.1.1/32
10.0.0.2
1.1.1.1/32 0 10.0.0.1

20.0.0.1 1.1.1.2/32 1 10.0.0.2
1.1.1.2/32
20.0.0.2 2 20.0.0.1

3 20.0.0.2

HMux
Dest: Src: Dest: Src: Dest: Src:
VIP Client DIP HMux VIP Client
8
Key Design Challenges
• Limited switch memory

• High failure robustness

• VIP assignment

• VIP migration
9
Challenge 1: Switches have Limited Memory
Workload: 100k+ VIPs and 1+ millions DIPs
Single HMux cannot store all VIPs and DIPs
Forwarding Table ECMP Table Tunneling Table
Index DIPs
VIP
1.1.1.1/32 0 10.0.0.1

1.1.1.2/32 1 10.0.0.2

2 20.0.0.1

3 20.0.0.2

Limits #VIPs Limits #DIPs
Table Forwarding ECMP Tunneling
Max. size 16k 4k 512
Max VIPs Max DIPs
10