Resource allocation in Cloud federation

162 Pages·2017·4.29 MB·English

by  Salma Rebai

THESE DE DOCTORAT CONJOINT TELECOM SUDPARIS et
L’UNIVERSITE PIERRE ET MARIE CURIE
Sp´ecialit´e: Informatique et T´el´ecommunications
Ecole doctorale: Informatique, T´el´ecommunications et Electronique de Paris
Present´ee par
Salma REBAI
Pour obtenir le grade de
DOCTEUR DE TELECOM SUDPARIS
Allocation et f´ed´eration des ressources
informatiques dans le Cloud
Soutenue le 13 Mars 2017
devant le jury compos´e de:
Prof. Samir TOHME´ Rapporteur Universit´e de Versailles
Prof. Jalel BEN OTHMAN Rapporteur Universit´e Paris 13
Prof. Marcelo DIAS DE AMORIM Examinateur UPMC – Paris 6
Prof. V´eronique VE`QUE Examinateur Universit´e Paris-Sud
Prof. Nadjib AIT SAADI Examinateur Universit´e Paris-Est
Dr. Jos´e NE`TO Examinateur Telecom SudParis
Prof. Djamal ZEGHLACHE Directeur de th`ese Telecom SudParis
Th`ese no 2017TELE0006
JOINT PHD THESIS BETWEEN TELECOM SUDPARIS AND
UNIVERSITY OF PIERRE ET MARIE CURIE
Speciality: Informatics and Telecommunications
Doctoral School: Informatique, T´el´ecommunications et Electronique de Paris
Presented by
Salma REBAI
To obtain the degree of
DOCTOR OF TELECOM SUDPARIS
Resource allocation in Cloud federation
Defended on 13 March 2017
Jury Members:
Prof. Samir TOHME´ Reporter University of Versailles
Prof. Jalel BEN OTHMAN Reporter University of Paris 13
Prof. Marcelo DIAS DE AMORIM Examiner UPMC – Paris 6
Prof. V´eronique VE`QUE Examiner University of Paris-Sud
Prof. Nadjib AIT SAADI Examiner University of Paris-Est
Dr. Jos´e NE`TO Examiner Telecom SudParis
Prof. Djamal ZEGHLACHE Thesis Director Telecom SudParis
Thesis no 2017TELE0006

To my parents Fouzia and Zouhir,
I am particularly indebted for your endless love, unconditional trust and continuous
support. Thanks for always believing in me and being at my side in everything I do!
To my dear husband Wael,
I am especially thankful for your understanding, encouragement, infinite support and
sincere love. Thanks for everything!
To my sisters Imen and Amal, and my brother Rami,
Thank you for always standing by my side during difficult times and for the fun
moments I have shared with you!
To all REBAI, ZOUAOUI and JRIBI family members,
Thanks for your love, kind support and encouragement!
Salma Rebai
ii
Abstract
Cloud computing is a steadily maturing large-scale model for providing on-demand IT
resources on a pay-as-you-go basis. This emerging paradigm has rapidly revolutionized
the IT industry and enabled new service delivery trends, including infrastructure exter-
nalization to large third-party providers. The Cloud multi-tenancy architecture raises
several management challenges for all stakeholders. Despite the increasing attention on
this topic, most efforts have been focused on user-centric solutions, and unfortunately
much less on the difficulties encountered by Cloud providers in improving their business.
Inthiscontext, CloudFederationhasbeenrecentlysuggestedasakeysolutiontothein-
creasingandvariableworkloads. Providershavingcomplementaryresourcerequirements
over time can collaborate and share their respective infrastructures, to dynamically ad-
justtheirhostingcapacitiesinresponsetousers’demands. However,joiningafederation
makes the resource allocation more complex, since providers have to also deal with co-
operation decisions and workload distribution within the federation. This is of crucial
importance for cloud providers from a profit standpoint and especially challenging in a
federation involving multiple providers and distributed resources and applications.
This thesis addresses profit optimization through federating and allocating resources
amongst multiple infrastructure providers. The work investigates the key challenges
and opportunities related to revenue maximization in Cloud federation, and defines
efficient strategies to govern providers’ cooperation decisions. The goal is to provide
algorithms to automate the selection of cost-effective distributed allocation plans that
simultaneously satisfy user demand and networking requirements. We seek generic and
robust models able to meet the new trends in Cloud services and handle both simple
and complex requests, ranging from standalone VMs to composite services requiring the
provisioning of distributed and connected resources.
In line with the thesis objectives, we first provide a survey of prior work on infras-
tructure resource provisioning in Cloud environments. The analysis mainly focuses on
profit-driven allocation models in Cloud federations and the associated gaps and chal-
lenges with emphasis on pricing and networking issues. Then, we present a novel exact
integer linear program (ILP), to assist IaaS providers in their cooperation decisions,
through optimal ”insourcing”, ”outsourcing” and local allocation operations. The dif-
ferent allocation decisions are treated jointly in a global optimization formulation that
splits resource request graphs across federation members while satisfying communica-
tion requirements between request subsets. In addition to the request topology, this
partitioning takes into account the dynamic prices and quotas proposed by federation
members as well as the costs of resources and their networking. The algorithm perfor-
mance evaluation and the identified benefits confirm the relevance of resource federation
in improving providers’ profits and shed light into the most favorable conditions to join
or build a federation. Finally, a new topology-aware allocation heuristic is proposed to
improve convergence times with large-scale problem instances. The proposed approach
uses a Gomory-Hu tree based clustering algorithm for request graphs partitioning, and
a Best-Fit matching strategy for subgraphs placement and allocation. Combining both
techniques captures the essence of the optimization problem and meets the objectives,
whilespeedingupconvergencetonear-optimalsolutionsbyseveralordersofmagnitude.
keywords: Cloud federation, profit optimization, distributed allocation, request split-
ting, linearintegerprogramming, Graphdecomposition, Gomory-Hutree, Best-Fitmatch-
ing.
Acknowledgements
It is a pleasure to thank and convey my most profound gratitude to all those people who
have contributed in one way or another to the achievement of this work.
I would like to express my deep gratitude and sincere thanks to my supervisor and
thesis director, Prof. Djamal ZEGHLACHE, for welcoming me in his research group at
Telecom SudParis and for his continuous support and guidance during my PhD study
years.
I am very grateful to my reading committee members, Prof. Samir TOHME´ and Prof.
Jalel BEN OTHMAN, for accepting to judge this work. Thank you for your precious
time, your interest, and your valuable feedback and suggestions to improve my disser-
tation work. My sincere thanks go also to the other members of my defense committee,
Prof. V´eronique VE`QUE,Prof. Nadjib AIT SAADI,Prof. Marcelo DIAS DE AMORIM
and Dr. Jos´e NE`TO, for their interest and valuable comments and for being part of my
thesis jury.
I extend heartfelt thanks to my friends and colleagues at Telecom SudParis for their
encouragements and support and for the fun moments we spent together. A special ac-
knowledgement is necessary for the administrative staff and especially the department’s
assistant for their continuous effort to facilitate administrative procedures. My warmest
thanks go also to my colleagues at ESME Sudria for the excellent and truly enjoyable
ambiance. I am very thankful for their encouragements and valuable advices whenever
I was in need.
Last but not least, my endless and deepest appreciations go to my family members:
my loving parents, my dearest husband, my caring brother and sisters, to whom I owe
so much. Thanks for making my life beautiful and for supporting me throughout my
thesis!
v

Contents
Abstract iii
Acknowledgements v
Contents vii
List of Figures x
List of Tables xii
Glossary of Acronyms xiii
1 Introduction 1
1.1 Scientific Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Research Problem and Objectives . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Motivations and Problem Statement . . . . . . . . . . . . . . . . . 5
1.2.2 Research Questions and Objectives . . . . . . . . . . . . . . . . . . 8
1.3 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Background and Foundations 12
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Cloud Computing Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.1 Cloud definition and key features . . . . . . . . . . . . . . . . . . . 13
2.2.2 Virtualization and Cloud Computing . . . . . . . . . . . . . . . . . 16
2.2.2.1 Server Virtualization . . . . . . . . . . . . . . . . . . . . 16
2.2.3 Cloud Services and Deployment Models . . . . . . . . . . . . . . . 18
2.3 Federated Inter-Cloud Environments . . . . . . . . . . . . . . . . . . . . . 21
2.3.1 Limitations of Single-Cloud Deployment Model . . . . . . . . . . . 21
2.3.2 Inter-Cloud: Definition, Benefits and Deployment Scenarios . . . . 22
2.3.2.1 Definition of the Inter-Cloud model . . . . . . . . . . . . 22
2.3.2.2 Benefits of Inter-Cloud Deployment Models . . . . . . . . 23
2.3.2.3 Architectural Classification of Inter-Cloud Scenarios . . . 24
2.3.3 Drivers and Barriers for Cloud Federation . . . . . . . . . . . . . . 28
vii
Contents viii
2.3.3.1 Drivers and Conditions for Federation Profitability . . . . 28
2.3.3.2 Economic Challenges and Enabling Standards . . . . . . 29
2.4 Resource Pricing in Cloud Computing . . . . . . . . . . . . . . . . . . . . 31
2.4.1 A General Taxonomy of IaaS Pricing Models . . . . . . . . . . . . 31
2.4.2 Common Pricing Types and Models . . . . . . . . . . . . . . . . . 32
2.4.2.1 Fixed Pricing . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.4.2.2 Dynamic Pricing . . . . . . . . . . . . . . . . . . . . . . . 34
2.4.2.3 Pricing Attributes and Resources Bundling . . . . . . . . 35
2.5 Thesis Scope and Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3 Cloud Resource Allocation: State of the Art 38
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2 Resource Provisioning and Allocation in the Cloud . . . . . . . . . . . . . 39
3.3 Resource Allocation in Single-Cloud Environments . . . . . . . . . . . . . 40
3.4 Resource Allocation in Multi-Cloud Environments . . . . . . . . . . . . . 41
3.4.1 Resource Allocation in Cloud Brokering Scenario . . . . . . . . . . 41
3.4.2 Resource Allocation in Hybrid Cloud . . . . . . . . . . . . . . . . . 42
3.4.3 Resource Allocation in Cloud Federation . . . . . . . . . . . . . . . 44
3.4.3.1 Cooperation and Profit-driven Resource Sharing . . . . . 45
3.4.3.2 Networking Requirements and Issues in Cloud Federation 47
3.4.3.3 Resource Pricing Issues in Cloud Federation . . . . . . . 49
3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4 Exact ILP-Based Algorithm for Federating and Allocating Resources 52
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.2 The System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2.1 Cloud Federation Model and Assumptions . . . . . . . . . . . . . . 54
4.2.2 Resources Requests Model . . . . . . . . . . . . . . . . . . . . . . . 55
4.2.3 Generic Pricing Model . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.3 Exact Federation Allocation Algorithm . . . . . . . . . . . . . . . . . . . . 58
4.3.1 Linear Integer Program Formulation . . . . . . . . . . . . . . . . . 60
4.4 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.4.1 Evaluation Environment . . . . . . . . . . . . . . . . . . . . . . . . 65
4.4.2 Comparative Baselines Approaches . . . . . . . . . . . . . . . . . . 66
4.4.3 Evaluation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4.3.1 Effectiveness of the Exact Federation Algorithm . . . . . 67
4.4.3.2 Favorable Federation Conditions . . . . . . . . . . . . . . 70
4.4.3.3 Scalability of the Exact Algorithm . . . . . . . . . . . . . 72
4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5 Graph Clustering based Algorithm for Resource Allocation in Cloud
Federation 77
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.2 Networking-Cost Aware Federating Resources Algorithm (NCAFedRA) . 79
5.2.1 Request Graph Partitioning . . . . . . . . . . . . . . . . . . . . . . 80
5.2.1.1 Gomory-Hu Tree Construction . . . . . . . . . . . . . . . 82