Optimal Resource Allocation in Adaptive Survey - VU-DARE Home

172 Pages·2013·4.81 MB·English

Optimal Resource Allocation
in Adaptive Survey Designs
Calinescu Melania, 1983 -
Optimal Resource Allocation in Adaptive Survey Designs
ISBN 978-90-820349-1-2
⃝c I. M. Calinescu, Amsterdam 2013
Allrightsreserved. Nopartofthispublicationmaybereproducedunanyformorby
anyelectronicormechanicalmeans(includingphotocopying,recordingorinformation
storage and retrieval systems) without permission in writing from the author.
Cover design by Jakub Peˇc`anka and Melania Calinescu. The front cover visualizes
world internet connectivity using data from 2009 (sources: www.nationmaster.com
and CIA’s World Factbook). Each country is depicted as a circle with radius given
by a logarithmic transformation of the country’s population. In each pie chart the
green colour indicates the percentage of the country’s population that has access to
internet.
Printed by GVO Drukkers & Vormgevers B.V. | Ponsen & Looijen, Ede
VRIJE UNIVERSITEIT
Optimal Resource Allocation
in Adaptive Survey Designs
ACADEMISCH PROEFSCHRIFT
ter verkrijging van de graad Doctor aan
de Vrije Universiteit Amsterdam,
op gezag van de rector magnificus
prof.dr. F.A. van der Duyn Schouten,
in het openbaar te verdedigen
ten overstaan van de promotiecommissie
van de Faculteit der Exacte Wetenschappen
op woensdag 13 november 2013 om 9.45 uur
in de aula van de universiteit,
De Boelelaan 1105
door
Ionela Melania Calinescu
geboren te Cimpulung, Roemeni¨e
promotor: prof.dr. G.M. Koole
copromotoren: dr. S. Bhulai
dr.ir. J.G. Schouten
Acknowledgements
Obtaining a PhD degree is a long journey full of tough and sweet moments that
requires curiosity and enthusiasm for discovery, but frustration and desperation are
rarelyfaraway. Onthisjourney,manypeoplehavehelpedmegetthroughthetougher
moments, but also shared with me the satisfaction of a job well done. The time has
come for me to thank them for all the support I received.
Mydeepestgratitudegoestomysupervisors,SandjaiBhulaiandBarrySchouten.
Sandjai, if it was not for your encouragement to take on this project I would not be
in this position today. There are many things I value about working with you. I
admireyourconstantenthusiasmandtalentfordevisingelegantsolutionstocomplex
problems. ItwasessentialtomysuccessthatIcouldalwayscountonyoutoanswermy
questions, whether they regarded research, administration or personal development.
I very much appreciated our weekly meetings, which made me work hard to raise
interesting questions and even harder to find suitable answers. I hope there will be
many occasions in the future to continue our discussions.
Barry,Ihavealwaysappreciatedandfeltinspiredbyyourpatienceanddedication
toyourwork. Yourcarefordetailtaughtmehowtobemoreprecisewhileyourdiverse
perspectivesontheprojecthelpedourresearchgrowataveryfastpace. Ihavelearned
a lot about survey methodology during our regular meetings “op de gang” as well as
the “do’s” and “don’ts” in survey practice during the surprisingly adventurous trips
to Heerlen. Your constant interest in promoting our results enabled me to become a
contributor to a Wiley Series book and meet some of the most influential researchers
in the field.
To my promotor, Ger Koole, thank you for giving me the opportunity to join the
OBP group and for the inspiring brainstorming sessions in the Alps.
I also want to thank the reading committee members, James Wagner, Annemieke
Luiten,MathiscadeGunst,BertZwartandRommertDekkerfortheircarefulreading
of my thesis, interesting feedback and for dealing with the flood of emails about my
vi Acknowledgements
defence date.
Additional thanks go to my research group at VU for the great time we had to-
gether. ThankyouAlexforkeepingacriticaleyeonmyDutchemailsandassembling
a fluent samenvatting from the chaotic pieces I sent you. To the one-day-a-week re-
searchers,formerandcurrent,thankyouformakingThursdaytheliveliestdayofthe
week. Masha, Demeter, thank you for fun pancake evenings. Alwin, thank you for
the pleasant three years in R-550 and lovely time together with Sylvia.
To the statistics group at VU, thank you for the joyful lunches and coffee breaks,
alas far too rarely enriched with cakes. I believe I owe at least one piece of cake to
many of you, and I promise to pay my dues at my defence party. Beata, thank you
for lovely mid-afternoon chats and for helping me find my passion by taking me to
the best dance class ever.
To my colleagues at Centraal Bureau voor de Statistiek, thank you for making
my time as a PhD student sometimes seem like a serious office job and for trying
to teach me a little Dutch as well. Nino, you brightened up my cloudiest days, I
am truly grateful for having had you there. Henk, Fatima, thank you both for the
many humorous moments in the office. Jan, Sander, thank you for solving my CBS-
beginner questions. Marriette, Martijn, thank you for clarifying some of the many
data collection mysteries.
ToTargol,youhaveknownmygoodsandbadsforthelongest,thankyouforbeing
my best friend all this time. To my Romanian friends, thank you for making home
seem just a step away, especially on December 1st. To my dear friends from home,
thank you for having me over and finding time on short notice to dine and chat. To
my Dutch friends, Laurens, Jeanine, Max, Ivar, thank you for great parties, patient
Dutch lessons and tasty dinners. Thank you Jan, Jarda, Tom´aˇs, Nikke, Fulvio, Raja
for fun trips and dinners together.
ToJakub,withwhomIhavesharedmypastsixyears,thankyouforyourconstant
and creative help, for stirring my curiosity to learn many new things and for a spicy
combination of happy moments and tough life lessons. Last but surely not least, to
my family, Sister, Mom and Dad, thank you for your unconditional love and support
and for the goodies packages that so often brought a little bit of home all the way to
Amsterdam!
Melania Calinescu
September 2013
Contents
1 Introduction 1
1.1 Surveys in practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Contributions of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Overview of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Adaptive survey designs and Markov decision theory: an intro-
duction 11
2.1 Resource allocation problems . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Markov decision theory . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 A brief introduction to sample surveys . . . . . . . . . . . . . . . . . . 20
2.4 Adaptive survey designs . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 The survey resource allocation problem 31
3.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Adaptive survey design policies . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Budget and capacity constraints . . . . . . . . . . . . . . . . . . . . . 39
3.4 Numerical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 The survey resource allocation problem for multiple quality indi-
cators 49
4.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2 The two-step algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.3 Numerical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5 The survey resource allocation problem and measurement errors 65
5.1 Measurement errors in surveys: an introduction . . . . . . . . . . . . . 66
viii Contents
5.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.3 Problem solving technique . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.4 Case study: the Dutch Labor Force Survey . . . . . . . . . . . . . . . 73
5.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.6 Appendix: additional optimization results . . . . . . . . . . . . . . . . 86
6 Adaptive survey designs to minimize survey mode effects 91
6.1 Survey mode effects: an introduction . . . . . . . . . . . . . . . . . . . 93
6.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.3 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.4 Case study: the Dutch Labor Force Survey . . . . . . . . . . . . . . . 100
6.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7 Dynamic learning in adaptive survey designs 119
7.1 Literature review multi-armed bandit problems . . . . . . . . . . . . . 120
7.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
7.3 Solving the budgeted MAB via dynamic programming . . . . . . . . . 127
7.4 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
8 Future research directions 141
Bibliography 144
Summary 157
Samenvatting 161
1
Introduction
How did the recent global financial crisis change the world’s economic landscape?
What are the repercussions of current economic and political policies on the future
societaldevelopment,asquantifiedthroughindicatorssuchastheunemploymentrate,
average household income, consumer confidence index? To answer such questions,
policy makers have to collect information from the population and summarize it in
a meaningful way. This is where survey organizations and statistical bureaus play
a crucial role. Collecting information from the entire population requires significant
amountsoftimeandmoney. Alternatively,asamplesurveymaybeconducted,where
onlyasamplefromthespecifiedpopulationisrequestedtoprovideinformation. Using
the results from the survey sample, knowledge can be obtained about the population
of interest.
1.1 Surveys in practice
Surveysareusedallaroundtheworldtomeasuresocio-economicstatusandwell-being
ofpeople,totesttheories,andmakepolicydecisions. However,thedifferentstatistics
computed from the survey data are of interest only if they accurately describe the
correspondingpopulationattribute. Multiplefactorsplayarolethroughoutthecourse
of a survey from its planning to the final systematization of the results. Some factors
may disrupt the framework of statistical inference theory and sampling theory that
grant methods to describe accurately (enough) population’s characteristics given the
surveysampleresults. Suchfactorsarepeople’slackofunderstanding(orinterest)as
towhytheyhavebeenselectedtoparticipateinasurvey,theirattitudetowardsareas
such as privacy and confidentiality of personal information, and the influence exerted
by the attributes of the survey design on their decision to participate in the survey.
Addressing these factors and related social science questions is an integral part of
2 Surveys in practice
3
0. Nonresponse rate=0.05
Nonresponse rate=0.3
Nonresponse rate=0.5
2
0.
as 0.1
bi
nt
e
d
on 0
p
s
e
nr
No 0.1
−
2
0.
−
3
0.
−
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Nonrespondent mean
Figure 1.1: Level of nonresponse bias for various nonresponse rates and nonre-
sponse means; respondent mean fixed at 0.50.
surveyresearch. Tremendousefforthasbeeninvestedintounderstandinghowhuman
behavior and thought may impact the precision and accuracy of survey statistics and
how the effects may be reduced or adjusted (see overviews in Groves et al. 2002,
Lepkowski et al. 2007 and Bethlehem et al. 2011).
In a perfect world, all sampled population units would be willing to participate
in the survey and provide all the requested data. In practical situations, however,
information from some sample units is missing due to factors such as those listed
above. This is called nonresponse and it is one of the most studied errors in the
survey literature. Classic inferential properties of sample estimates require these
statistics be computed from the entire sample. One example statistic is the sample
mean as an estimator of the population mean. In the presence of nonresponse the
sample mean is reduced to the respondent mean, i.e., the sample mean is obtained
based only on information coming from the pool of respondents. The deviation of
the respondent mean from the full sample mean is called nonresponse bias and it is a
function of the nonresponse rate (i.e., the proportion of nonrespondents in the entire
sample)andthedifferencebetweentherespondentandnonrespondentmeans. Figure
1.1 (from Groves and Couper 1998) illustrates the consequences of nonresponse
rates on the precision of the survey estimates. Given the respondent mean of 0.50,