Using Github Profiles in Software Developer Recruitment and Hiring

130 Pages·2015·2.69 MB·English

Using Github Profiles in Software
Developer Recruitment and Hiring
Master’s Thesis
ADITYA REDDY NAGARAM

Using Github Profiles in Software
Developer Recruitment and Hiring
THESIS
submittedinpartialfulfillmentofthe
requirementsforthedegreeof
MASTEROFSCIENCE
in
COMPUTERSCIENCE
by
ADITYAREDDYNAGARAM
borninJagtial,India
SoftwareEngineeringResearchGroup
DepartmentofSoftwareTechnology
FacultyEEMCS,DelftUniversityofTechnology
Delft,theNetherlands
www.ewi.tudelft.nl
©2015 ADITYAREDDYNAGARAM.
Using Github Profiles in Software
Developer Recruitment and Hiring
Author: ADITYAREDDYNAGARAM
Studentid: 4243447
Email: [email protected]
Abstract
Social coding platforms can provide initial understanding about the skills exhib-
itedbythedevelopersontheseplatforms. Incontextswherecandidatessocialprofile
informationisusefulforrecruitingsoftwaredevelopers,theinformationregardingthe
developers on these platforms can be leveraged by the recruiters with some software
knowledge.However,recruitershavetoputmanyeffortsininferringaboutadeveloper
skillonsocialcodingplatforms.
Inthisthesis,weinvestigateonprovidingrelevantinformationregardingsoftware
developer capabilities on a social coding platform to the recruiters. We used GitHub
as our social coding platform for this purpose. We explored regarding, the attributes
to use for indicating the skills exhibited by a developer on GitHub. We also investi-
gated GitHub as a resource containing some potential software developer candidates
byrecommendingGitHubdeveloperprofile,solelybasedonskillsetrequirementsof
jobadvertisements.
Ourresultsindicatethatthegenerateddeveloperskillprofileshaveavalidsetof
attributes when combined, to indicate the regarding three skills exhibited by a soft-
ware developer on GitHub. However, the generated profile was slightly preferred by
the technical recruiters because of the profile’s complexity in understanding and in-
completeness. Intheinvestigationofrecommendingdeveloperprofilestosuitjobad-
vertisementrequirementsourrecommendationstrategycouldonlyachieveaprecision
of0.39onaverageandanNDPMrankingaccuracyvalueof0.43onaverage.
ThesisCommittee:
Chair: Prof. Dr. ArieVanDeursen,FacultyEEMCS,TUDelft
Universitysupervisor: Dr. ClaudiaHauff,FacultyEEMCS,TUDelft
CommitteeMember: Dr.GeorgiosGousios,Digitalsecuritygroup,RadboundUniversiteit
ii
Preface
Over the past two years, my journey as a master’s student of TU Delft have been a great
experience. Firstly, I would like to express my sincere thanks to Claudia Hauff, who have
guidedmeineveryphaseofworkandhelpedmeinthesuccessfulcompletionofmythesis
work. Iadmireherpatiencetocorrectmeinallmyshallowworksduringmythesiswork.
IwouldliketoextendmysincerethankstoGeorgiosGousios, whohaveguidedmein
theinitialpartofthesiswork. Furthermore, hisfeedbackinfinalisingthethesisreportwas
usefulandhavealsoacceptedtobepartofmythesiscommittee. Iwouldalsoliketothank
ArievanDeursenforhissupportwhilefinalisingthethesisreport.
Finally, I would like to thank my brother’s family in the Netherlands and my friends
whohavealwayssupportedmeinallthewayspossibleduringmystudiesinDelft. Iwould
particularly like to thank all my friends and other respondents who have extended their
supportduringevaluationofmythesis. Last,butnotleast,Iamgratefulforthesupportand
lovemyfamilyhavegivenmeallthroughmylifetoachievemydreams.
ADITYAREDDYNAGARAM
Delft,theNetherlands
August13,2015
iii

Contents
Preface iii
Contents v
ListofFigures ix
ListofTables xi
1 Introduction 1
1.1 ProblemDescription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 ResearchGoalandQuestions . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 ThesisWorkflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 ThesisStructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 RelatedWork 5
2.1 SocialcodingplatformGitHub . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 GitHubworkflow . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.2 PeerDeveloperProfiling . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.3 RecruiterDeveloperProfiling . . . . . . . . . . . . . . . . . . . . 7
2.1.4 Attributes used in literature to indicate popularity of projects on
GitHub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.5 ConclusionsoninvestigatedGitHubrelatedwork . . . . . . . . . . 9
2.2 RecommendationSystems . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Recommendationsystemtask . . . . . . . . . . . . . . . . . . . . 10
2.2.2 RecommendationAlgorithmTechniques . . . . . . . . . . . . . . 11
2.2.3 RecommendationSystemsForFacilitatingRecruitment. . . . . . . 13
2.2.4 Conclusionsoninvestigatedrecommendationsystemsrelatedwork 14
3 Approach 15
3.1 Softwaredeveloperprofile . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
v
CONTENTS
3.1.2 CollaborationskillsexhibitedbyGitHubuser . . . . . . . . . . . . 16
3.1.3 ProjectManagementAbility . . . . . . . . . . . . . . . . . . . . . 17
3.1.4 Programminglanguageskill . . . . . . . . . . . . . . . . . . . . . 19
3.2 Recommendingdeveloperprofilestoajobadvertisement . . . . . . . . . . 21
3.2.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2.2 Entity types differences between the keyword sets of a job adver-
tisementanddeveloperprofile . . . . . . . . . . . . . . . . . . . . 21
3.3 EvaluationApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.1 Evaluatingdeveloperprofile . . . . . . . . . . . . . . . . . . . . . 23
3.3.2 Evaluatingarecommendationtask . . . . . . . . . . . . . . . . . . 23
4 Implementation 25
4.1 Workflowoverviewfordeveloperprofileandrecommendations . . . . . . . 25
4.2 SoftwareDeveloperProfileGenerator . . . . . . . . . . . . . . . . . . . . 27
4.2.1 GHTorrentGitHubuser’sdata . . . . . . . . . . . . . . . . . . . . 27
4.2.2 Generatingcollaborationskill . . . . . . . . . . . . . . . . . . . . 29
4.2.3 Generatingprojectmanagementskill . . . . . . . . . . . . . . . . 30
4.2.4 Generatinglevelofprogramminglanguageskillprofile . . . . . . . 31
4.3 RecommendingProfiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.3.1 Schemausedforrecommendations . . . . . . . . . . . . . . . . . 33
4.3.2 Jobadvertisementdataindexing . . . . . . . . . . . . . . . . . . . 33
4.3.3 Representing software developer profile to accomplish recommen-
dations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.3.4 Recommendationstrategy . . . . . . . . . . . . . . . . . . . . . . 41
5 Evaluation 47
5.1 EvaluatingGeneratedDeveloperProfile . . . . . . . . . . . . . . . . . . . 47
5.1.1 Assessmentaspectsofdeveloperprofile . . . . . . . . . . . . . . . 47
5.1.2 Questionsforthedeveloperprofileassessment . . . . . . . . . . . 48
5.1.3 QuestionnaireandInterviewsetupforevaluation . . . . . . . . . . 52
5.2 EvaluatingRecommendationsOfProfilesForTheJobAdvertisements . . . 54
5.2.1 MetricsSelectionforEvaluatingtherecommendations . . . . . . . 55
5.2.2 Experimentsettingforevaluationofrecommendations . . . . . . . 56
6 Results 61
6.1 SoftwareDeveloperProfileAssessmentResults . . . . . . . . . . . . . . . 61
6.1.1 ASP1: Developerprofileanditsattributesvalidity . . . . . . . . . 62
6.1.2 DeveloperProfileAssessmentintheRecruitmentProcess . . . . . 82
6.1.3 ConclusionsontheProposedDeveloperProfile . . . . . . . . . . . 89
6.2 Accuracyoftherecommendationtask . . . . . . . . . . . . . . . . . . . . 90
6.2.1 ProfileRecommendationPrecision . . . . . . . . . . . . . . . . . . 90
6.2.2 RankingAccuracy . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.2.3 DrawBacksofourrecommendationStrategy . . . . . . . . . . . . 93
vi