Dynamic Load Balancing in Distributed Content-based Publish/Subscribe by Alex KY Cheung A ...

147 Pages·2006·0.82 MB·English

Dynamic Load Balancing in Distributed Content-based
Publish/Subscribe
by
Alex K. Y. Cheung
A thesis submitted in conformity with the requirements
for the degree of Master of Applied Science
Graduate Department of Electrical and Computer Engineering
University of Toronto
Copyright (cid:176)c 2006 by Alex K. Y. Cheung
ii
Abstract
Dynamic Load Balancing in Distributed Content-based
Publish/Subscribe
Alex K. Y. Cheung
Master of Applied Science
Graduate Department of Electrical and Computer Engineering
University of Toronto
2006
Distributed content-based publish/subscribe systems to date suffer from performance degrada-
tion and poor scalability under load conditions typical in real-world applications. The reason
for this shortcoming is due to the lack of a load balancing solution, which have rarely been
studied in the context of publish/subscribe. This thesis proposes a load balancing solution
specific for distributed content-based publish/subscribe systems that is distributed, dynamic,
adaptive, transparent, andaccommodatesheterogeneity. Thesolutionconsistsofthreekeycon-
tributions: a load balancing framework, a novel load estimation algorithm, and three offload
strategies. Experimentalresultsshowthattheproposedloadbalancingsolutionisefficientwith
less than 0.7% overhead, effective with at least 90% load estimation accuracy, and capable of
loadbalancingwith100% ofload initiatedat anedge nodeof theentiresystemusing real-world
data sets.
iii
Acknowledgements
Super thanks to Mom, Dad, sister (Annie), friends (Mr. Biggy, Mr.
Dum Dum, Mr. Chestnut) for their indirect support in my Master stud-
ies. Thanks to KUG friends (Chef, Knifey, Chopsticky), and people in
and outside the PADRES team (Eli, Guoli, Pengcheng, Serge, Vinod,
and others) for their friendly companionship and guidance. Thanks to
Cybermation for their funding in the PADRES project and their free
all-you-can-eat snacks and drinks. Last but not least, big thank you to
my supervisor, Professor Hans-Arno Jacobsen, for his unmatched super-
vision and giving me a lot of freedom.
Post defense: Thank you to my J.A.L. defense committee for giving an
almighty A+ for this thesis. Yay! :D
iv
Contents
List of Tables ix
List of Figures xi
List of Terminology xv
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Background and Related Work 7
2.1 Publish/Subscribe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Publish/Subscribe Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.2 Content-based Publish/Subscribe Language . . . . . . . . . . . . . . . . . 7
2.1.3 Content-based Publish/Subscribe Routing . . . . . . . . . . . . . . . . . . 10
2.1.4 Load in Publish/Subscribe . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Load Balancing in Different Software Layers . . . . . . . . . . . . . . . . . 12
2.2.2 Classification of Load Distribution Algorithms . . . . . . . . . . . . . . . 14
2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
v
3 Load Balancing Framework 19
3.1 Overview of Load Balancing Components . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Underlying Publish/Subscribe Architecture . . . . . . . . . . . . . . . . . . . . . 20
3.3 Load Detection Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3.1 Protocol for Exchanging Load Information. . . . . . . . . . . . . . . . . . 23
3.3.2 Triggering Load Balancing Based on Detection Results . . . . . . . . . . . 28
3.4 Load Balancing Mediation Protocols with Brokers and Subscribers . . . . . . . . 35
3.4.1 Mediating Local Load Balancing . . . . . . . . . . . . . . . . . . . . . . . 37
3.4.2 Mediating Subscriber Migration. . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.3 Mediating Global Load Balancing . . . . . . . . . . . . . . . . . . . . . . 39
4 Load Balancing Algorithm 41
4.1 Load Estimation Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.1.1 Estimating the Input and Output Load Requirements of Subscriptions . . 41
4.1.2 Matching Delay Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1.3 Input Utilization Ratio Estimation . . . . . . . . . . . . . . . . . . . . . . 44
4.1.4 Output Utilization Ratio Estimation . . . . . . . . . . . . . . . . . . . . . 45
4.1.5 Memory Utilization Ratio Estimation . . . . . . . . . . . . . . . . . . . . 45
4.2 Offload Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2.1 Input Offload Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2.2 Match Offload Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2.3 Output Offload Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5 Experiments 69
5.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.2 Macro Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.2.1 Local Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.2.2 Global Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.3 Micro Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.3.1 Local Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
vi
5.3.2 Global Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6 Conclusion 123
6.1 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Bibliography 127
vii
viii
List of Tables
4.1 Bit vector example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2 Initial load of brokers B1 and B2. . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.3 Load of B1 and B2 after input load balancing. . . . . . . . . . . . . . . . . . . . 62
4.4 Load of B1 and B2 after a favorable case of output load balancing. . . . . . . . . 62
4.5 Load of B1 and B2 after an unfavorable case of output load balancing. . . . . . 63
5.1 Subscription templates used in all experiments. . . . . . . . . . . . . . . . . . . . 71
5.2 Default values of load balancing parameters used in experiments. . . . . . . . . . 73
5.3 Broker specifications in local load balancing experiment. . . . . . . . . . . . . . . 74
5.4 Broker specifications in global load balancing experiment. . . . . . . . . . . . . . 91
5.5 Edge-broker specifications in global load balancing micro experiment. . . . . . . 115
ix
x