Resource Allocation Optimization in Large Scale Distributed Systems

215 Pages·2014·4.12 MB·English

AN ABSTRACT OF THE DISSERTATION OF
Thuan Duong-Ba for the degree of Doctor of Philosophy in Electrical and Computer
Engineering presented on May 16, 2014.
Title: Resource Allocation Optimization in Large Scale Distributed Systems
Abstract approved:
Thinh P. Nguyen Bella Bose
We studied the problem of resource allocation in large scale distributed applications such
as Online Social Networks (OSN) and Cloud Computing. In such settings, resource al­
location schemes need to efficient as well as adaptive to the time-varying environments.
The abstract resource allocation problem concerns with how to optimally use resources
for different tasks. In the context of this dissertation, the resources are servers and the
tasks are (a) the virtual machines in the cloud computing setting, and or users for on-
line social network applications. It is well-known that the general resource allocation
problem is NP-hard. Therefore, in this dissertation, we study a number of heuristic
algorithms designed for two primary objectives: 1) achieve reliability via load balanc­
ing among resource providers and 2) minimizing the energy consumption by reducing
unnecessary intercommunication loads among the servers.
Specifically, the dissertation has three main components. The first component deals
with optimal assignment of user data to servers to maximize load balance and mini­
mize power consumption. In this component, we propose a novel Distributed Perturbed
Greedy Search (DPGS) algorithm which combine both deterministic search and random
search to speed the convergence while avoiding local optimum. The empirical shows
that the DPGS has a fast convergence rate to the near optimal solution even when the
environment changes. The second component deals with the analysis on the conver­
gence rates of a general simulated annealing algorithm via the notion of adiabatic time.
We then apply the results to characterize the convergence rates for simulated anneal­
ing algorithm when applied to the optimal assignment in the component one. Finally,
the third component of the dissertation is concerned with optimal assignment of virtual
machines to servers in the context of cloud computing, in order to minimize the energy
subject to a given performance requirement. We show that the problem can be approx­
imated well as a convex problem, and propose convex relaxation technique to find the
optimal solution.
©c Copyright by Thuan Duong-Ba

May 16, 2014

All Rights Reserved
Resource Allocation Optimization in Large Scale Distributed Systems

by

Thuan Duong-Ba

A DISSERTATION

submitted to

Oregon State University

in partial fulfillment of

the requirements for the

degree of

Doctor of Philosophy

Presented May 16, 2014

Commencement June 2014
Doctor of Philosophy dissertation of Thuan Duong-Ba presented on May 16, 2014.
APPROVED:
Major Professor, representing Electrical and Computer Engineering

Director of the School of Electrical Engineering and Computer Science

Dean of the Graduate School
I understand that my dissertation will become part of the permanent collection of
Oregon State University libraries. My signature below authorizes release of my
dissertation to any reader upon request.
Thuan Duong-Ba, Author
ACKNOWLEDGEMENTS
First and foremost, I would like to express my acknowledgment of and deepest grat­
itude to my advisors, Professor Thinh Nguyen and Professor Bella Bose, for giving me
the opportunity to become a Ph.D. student and for their insight, great guidance, and en­
couragement during my studies at Oregon State University. Without their support this
dissertation would not have been possible.
During my research, I have also had opportunities of cooperating with many great
people at the School of Electrical Engineering and Computer Science. Professor Patrick
Chiang, thank you very much for the fruitful cooperation and guidance during my first
project. I would like to thank Professor Bechir Hamdaoui and Professor Yevgeniy
Kovchegov for their acceptance to be my committee members as well as valuable com­
ments on my dissertation.
I am also indebted to all my friends for the nice time we have spent together. Es­
pecially lab-mates who helped me so much from the first day I joined the group. They
helped me not only in getting familiar with a totally new life in the US, but also in
research.
Last but not least, I would like to express my sincere appreciation to my family and
relatives for continuously supporting and encouraging me during my studying in the US.
Finally, I want to thank my wife Huong Nguyen and my two children Tung and Nhung
for providing me a happy family with constant support and encouragement.
TABLE OF CONTENTS

Page

1 Introduction 1

1.1 Resource Allocation in Distributed Systems . . . . . . . . . . . . . . . . 1

1.1.1 OptimizationFormulation . . . . . . . . . . . . . . . . . . . . 3

1.1.2 Multi-objectiveOptimization . . . . . . . . . . . . . . . . . . . 4

1.1.3 An Example of Resource Allocation Problems . . . . . . . . . . 6

1.2 ContributionofThisDissertation . . . . . . . . . . . . . . . . . . . . . . 7

1.2.1 Resource Allocation for Load Balancing . . . . . . . . . . . . . 7

1.2.2 Resource Allocation for Energy Conservation . . . . . . . . . . 7

1.2.3 Convergence Rate of Simulated Annealing Algorithms . . . . . 8

1.3 OrganizationofThisDissertation . . . . . . . . . . . . . . . . . . . . . . 8

2 Background 9

2.1 OnlineSocialNetworkApplications . . . . . . . . . . . . . . . . . . . . 9

2.1.1 SocialNetworkAnalysis . . . . . . . . . . . . . . . . . . . . . 10

2.1.2 NetworkModels . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.3 NetworkGrowthModels . . . . . . . . . . . . . . . . . . . . . 13

2.2 GraphPartitioningProblem . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2.1 LocalSearchMethods . . . . . . . . . . . . . . . . . . . . . . 15

2.2.2 SpectralGraphPartitioning . . . . . . . . . . . . . . . . . . . . 18

2.2.3 Multi-levelGraphPartitioning . . . . . . . . . . . . . . . . . . 19

2.3 CloudComputing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3.1 TypesofServices . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3.2 TypesofCloudComputing . . . . . . . . . . . . . . . . . . . . 22

2.3.3 AttributesofCloudComputing . . . . . . . . . . . . . . . . . . 23

2.3.4 ResourceAllocation . . . . . . . . . . . . . . . . . . . . . . . . 24

2.4 Bin-PackingProblem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4.2 Hardness of The Bin-Packing Problem . . . . . . . . . . . . . . 26

2.4.3 GreedyAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . 27

2.5 MarkovChainMonteCarlo(MCMC)Method . . . . . . . . . . . . . . . 29

2.5.1 MarkovChain . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.5.2 ConvergenceRates . . . . . . . . . . . . . . . . . . . . . . . . 33

2.5.3 SimulatedAnnealing . . . . . . . . . . . . . . . . . . . . . . . 36

2.6 ConvexOptimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.6.1 ConvexSets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
TABLE OF CONTENTS (Continued)

Page
2.6.2 ConvexFunctions . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.6.3 ConvexOptimizationProblems . . . . . . . . . . . . . . . . . . 42

2.6.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.6.5 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3 Distributed Algorithms For Load Balancing In Large Scale Applications 58

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.3 ProblemFormulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.3.1 AlgorithmicOutline . . . . . . . . . . . . . . . . . . . . . . . . 72

3.3.2 TechnicalBuildingBlocks . . . . . . . . . . . . . . . . . . . . 73

3.4 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3.4.1 Centralized Simulated Annealing Algorithm . . . . . . . . . . . 76

3.4.2 Distributed Data Structures and Computations . . . . . . . . . . 77

3.4.3 Distributed Simulated Algorithm (DSA) . . . . . . . . . . . . . 81

3.4.4 Distributed Perturbed Greedy Search (DPGS) . . . . . . . . . . 82

3.4.5 ConvergenceAnalysisofDPGS . . . . . . . . . . . . . . . . . 86

3.5 SimulationResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.5.1 SmallSyntheticGraphs . . . . . . . . . . . . . . . . . . . . . . 97

3.5.2 LargeGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

4 Distributed Assignment And Replication For Online Social Networking Appli­
cations 107

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.3 ProblemFormulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.4 DistributedAlgorithm/Protocol . . . . . . . . . . . . . . . . . . . . . . . 115

4.5 SimulationResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
TABLE OF CONTENTS (Continued)

Page

5 Convergence Rate of Inhommogeneous Markov Chains 125

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

5.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

5.2.1 SimulatedAnnealingAlgorithm . . . . . . . . . . . . . . . . . 127

5.2.2 MarkovChainProperties . . . . . . . . . . . . . . . . . . . . . 129

5.3 Analysis Framework for Simulated Annealing . . . . . . . . . . . . . . . 131

5.4 Application to Client-Server Assignment . . . . . . . . . . . . . . . . . . 139

5.4.1 ProblemFormulation . . . . . . . . . . . . . . . . . . . . . . . 139

5.4.2 Simulated Annealing Based Algorithm . . . . . . . . . . . . . . 141

5.5 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

6 Joint Virtual Machine Placement and Migration Scheme for Datacenters 158

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

6.2 System Model, Assumptions, and Optimization Objectives . . . . . . . . 161

6.3 Multi-Objective Optimization to VM Management . . . . . . . . . . . . 164

6.3.1 Multi-Objective Optimization Function . . . . . . . . . . . . . 164

6.3.2 Graph Partitioning Based Rank Minimization . . . . . . . . . . 170

6.4 Relaxed Convex Optimization Based Heuristic Algorithm . . . . . . . . . 175

6.4.1 RelaxedConvexOptimization . . . . . . . . . . . . . . . . . . 175

6.4.2 Joint VM Placement and Migration (JPM) Algorithm . . . . . . 176

6.5 PerformanceEvaluationandDiscussion . . . . . . . . . . . . . . . . . . 179

6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

7 Conclusion and Future Work 186

7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

7.2 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Bibliography 189