Diversity in Survey Questions on the Same Topic: Techniques for Improving Comparability PDF
Preview Diversity in Survey Questions on the Same Topic: Techniques for Improving Comparability
Social Indicators Research Series 68 Tineke de Jonge Ruut Veenhoven Wim Kalmijn Diversity in Survey Questions on the Same Topic Techniques for Improving Comparability Social Indicators Research Series Volume 68 SeriesEditor AlexC.Michalos,FacultyofArtsOffice,BrandonUniversity,Brandon,Manitoba, Canada Editors EdDiener,UniversityofIllinois,Champaign,USA WolfgangGlatzer,J.W.GoetheUniversity,FrankfurtamMain,Germany TorbjornMoum,UniversityofOslo,Norway MirjamA.G.Sprangers,UniversityofAmsterdam,TheNetherlands JoachimVogel,CentralBureauofStatistics,Stockholm,Sweden RuutVeenhoven,ErasmusUniversity,Rotterdam,TheNetherlands This series aims to provide a public forum for single treatises and collections of papersonsocialindicatorsresearchthataretoolongtobepublishedinourjournal Social IndicatorsResearch.Like the journal, the book series deals with statistical assessmentsofthequalityoflifefromabroadperspective.Itwelcomestheresearch onawidevarietyofsubstantiveareas,includinghealth,crime,housing,education, family life, leisure activities, transportation, mobility, economics, work, religion andenvironmentalissues.Theseareasofresearchwillfocusontheimpactofkey issuessuchashealthontheoverallqualityoflifeandviceversa.Aninternational reviewboard,consistingofRuutVeenhoven,JoachimVogel,EdDiener,Torbjorn Moum,MirjamA.G.SprangersandWolfgangGlatzer,willensurethehighquality oftheseriesasawhole. More information about this series at http://www.springer.com/series/6548 Tineke de Jonge • Ruut Veenhoven Wim Kalmijn Diversity in Survey Questions on the Same Topic Techniques for Improving Comparability TinekedeJonge RuutVeenhoven ErasmusHappinessEconomics ErasmusHappinessEconomics ResearchOrganisation ResearchOrganisation ErasmusUniversityRotterdam ErasmusUniversityRotterdam Rotterdam,TheNetherlands Rotterdam,TheNetherlands WimKalmijn(deceased) ErasmusHappinessEconomics ResearchOrganisation ErasmusUniversityRotterdam Rotterdam,TheNetherlands ISSN1387-6570 ISSN2215-0099 (electronic) SocialIndicatorsResearchSeries ISBN978-3-319-53260-8 ISBN978-3-319-53261-5 (eBook) DOI10.1007/978-3-319-53261-5 LibraryofCongressControlNumber:2017933832 ©SpringerInternationalPublishingAG2017 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained hereinor for anyerrors oromissionsthat may havebeenmade. Thepublisher remainsneutralwith regardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. Printedonacid-freepaper ThisSpringerimprintispublishedbySpringerNature TheregisteredcompanyisSpringerInternationalPublishingAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Preface Today, we are investing more in scientific research than ever before in human history, and a search in Google Scholar suggests that in 2015 some eight million scientificpublications were produced. Allthis research is assumed toresultinthe ‘accumulationofknowledge’,thoughttotakeplacethroughanexchangeofinfor- mationinthis‘academicforum’.Realityismorecomplicatedthanthis.Thereisnot one academic forum, but rather many different market places, and exchange of knowledgeinthesebazaarsisconsiderablyhinderedbythedifferentlanguagesand currenciesused.Asaresult,muchofthegatheredinformationgetslostinthedust of libraries. Obstacles to accumulation of knowledge differ across academic set- tings. In this book, we address one particular problem in one particular research field. This book is about a problem in survey research: in this case, large-scale questionnaire studies among samples of the general population in countries. We focuson‘socialsurveys’thatareaboutthequalityoflifeinnations.Mostmodern countries conduct such surveys periodically, for example, the ‘Understanding Society’ study in the UK, the Level of Living survey in Sweden and the Public Opinion Survey on the Life of the Nation in Japan. Then, there are international survey programs, such as the Eurobarometer, the European Social Survey, the GallupWorldPollandtheWorldValuesSurvey.Worldwide,somemillionpeople takepartinsuchsurveyseveryyear. Social surveys address similar themes, such as social position and opinions. Manyofthetopicsareidentical;allthesurveyshavequestionsaboutage,sexand education,andmostalsoaskaboutincomeandethnicity.Yet,thequestionsasked onthesemattersareoftenformulateddifferently,forexample,questionsaboutthe education level of the respondent sometimes ask the respondent to give a number for years of education, while in other surveys, a list of education types is given, from which the respondent mustselect the final type achieved. In this case, many differentclassificationsofschoollevelcanbeusedwhichvarywithinandbetween countries.Inthismixtureofwaysofaskingthesamequestionofeducationlevels v vi Preface acrossnationsandtime,typicallylessthanhalfoftheavailablesurveydatacanbe usedforsuchpurpose. Oneofthecommontopicsinsocialsurveysis‘happiness’,thatis,thesatisfac- tionwithone’slifeasawhole.Thismatteristypicallymeasuredusingsingledirect questions, such as: ‘Taking all together, how happy would you say you are these days?Wouldyousayyouareveryhappy,prettyhappyornottoohappy?’Todate (2016), such questions have figured in some 10,000 survey studies and have been answered by some 200,000 respondents. The observed distributions of these responses are gathered in the collection ‘Happiness in Nations’ in the World Database of Happiness (WDH) (Veenhoven 2016). Viewing this collection of data, we can see that differently formulated questions have been used and that there are also many differences in the response options presented to respondents. Again, this means that only part of the data is available for comparing happiness across nations and time. These problems are not confined to comparing levels of educationandhappinessacrosspopulationsandnations.Theyarefoundformany othertopicsinsurveyresearch,suchashealthcareorcustomersatisfaction, In this book, we present methods for dealing with this diversity in survey questions on the same subject; we review existing methods used to homogenize data and propose new ones. The book is a spin-off from the World Database of Happiness,themainaimofwhichistocollateandmakeavailableresearchfindings onthesubjectiveenjoymentoflifeandtopreparethesedataforresearchsynthesis. The first methods we discuss were proposed in the book Happiness in Nations (Veenhoven1993,Chap.7‘HowtheDataAreHomogenized’)whichwereusedat the inception of the World Database of Happiness. Some 10 years later, a new method was introduced: the International Happiness Scale Interval Study (HSIS) (Veenhoven2008).TakingtheHSISasabasis,WimKalmijn(2010)developedthe ContinuumApproach.Then,buildingonKalmijn’swork,TinekeDeJonge(2015) developedtheReferenceDistributionMethod. Inthisbook,wedescribetheevolutionoftheseinnovationsandprovideaview ofwherewestandnow.Wegoontosuggestwaystoevolvethislineofresearch. The book is based on the doctoral dissertation of Tineke DeJonge (2015); most chapters are based on articles authored in most cases by the three of us and Lidia ArendsoftheErasmusUniversityRotterdam. WimKalmijnpassedawayinNovember2015aged81;hecontributedmuchto the preparation of this book. We thank Willem Saris of the University Pompeu Fabra in Barcelona for his comments on the draft version and Miranda Aldham- Breary,seniorvolunteeroftheWDH,forimprovingtheEnglish. Rotterdam,TheNetherlands RuutVeenhoven Preface vii References De Jonge, T. (2015). Different survey questions on the same topic. How to make responses comparable? Dissertation, Erasmus University Rotterdam, The Netherlands. Enschede: IpskampDrukkers.Availableathttp://repub.eur.nl/pub/78339 Kalmijn,W.M.(2010).Quantificationofhappinessinequality.Dissertation,ErasmusUniversity Rotterdam, The Netherlands. Enschede: Ipskamp Drukkers. Available at http://repub.eur.nl/ pub/21777 Veenhoven, R. (1993). Happiness in nations, subjective appreciation of life in 56 nations, 1946–1992(Studiesinsocial-culturaltransformation,2).Risbo,ErasmusUniversityRotter- dam,Netherlands.Availableat:https://personal.eur.nl/veenhoven/Pub1990s/93b-con.html Veenhoven,R.(2008).TheInternationalScaleIntervalStudy.InV.Møller&D.Huschka(Eds.), Quality of Life in the new millennium: ‘Advances in quality-of-life studies, theory and research’,Part2:Refiningconceptsandmeasurementtoassesscross-culturalquality-of-life (SocialIndicatorResearchSeries,Vol.35,pp.45–58).Dordrecht:SpringerPress. Veenhoven,R.(2016).Happinessinnations.WorldDatabaseofHappiness,ErasmusUniversity Rotterdam.Availableat:http://worlddatabaseofhappiness.eur.nl/hap_nat/nat_fp.php?mode¼1 Contents PartI ComparingResponsestoDifferentSurveyQuestionsonthe SameTopic:ProblemsandConventionalSolutions 1 DiversityinSurveyItemsandtheComparabilityProblem. . . . . . . 3 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 AnImpressionoftheDiversityinSurveyItems. . . . . . . . . . . . . . . . . 4 TheIncomparabilityofTimeSeriesfromDifferentSurveys andDifferentTimePeriods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 ConventionalScaleTransformationMethodsforItemsfrom DifferentSurveys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 LinearStretch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 SemanticJudgmentofFixedWordValue. . . . . . .. . . . . . . . . . . .. 12 SomeWordsonMethodstoCorrectforResponseHeterogeneity attheIndividualLevel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 TheNeedforFurtherInnovation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 PartII Innovation1:TheHappinessScaleIntervalStudy 2 TheHappinessScaleIntervalStudy. . . . . . . . . . . . . . . . . . . . . . . . 19 IntroductiontotheHappinessScaleIntervalStudy. . . . . . . . . . . . . . . 19 TheScaleIntervalRecorder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 DifferencewithConventionalMethodsforScaleTransformation. . 21 ThreeScaleTransformationMethodsAppliedto EmpiricalData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 TheThreeTransformationMethodsandthe ComparabilityProblem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 ix x Contents 3 UseofHappinessScaleIntervalStudiesinThisBook. . . . . . . . . . . 27 ResearchQuestionsAddressedinThisBookUsing HSIS-Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 HSIS-StudiesUsedandtheSelectionofSurveyItems. . . . . . . . . . . . 28 TheUSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 TheNetherlands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Spain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 RepresentativenessofJudges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4 EquivalenceofRatingScalesUsingDifferentKeywords. . . . . . . . . 31 SurveyswithEquivalentItemsfor‘Happiness’and ‘SatisfactionwithLife’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 TheProblem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 TheScaleIntervalRecorderasanInstrumenttoCompare theIntervalsRepresentedbyEquivalentResponseOptions. . . . . . . . . 33 DifferencesinValueRangesRepresentedbyResponseOptions LabeledinDutch,SpanishandEnglish. . . . . . . . . . . . . . . . . . . . . . . 34 ResponseOptionsLabeledinDutch. . . . . . . . . . . . . . . . . . . . . . . . 34 ResponseOptionsLabeledinSpanish. . . . . . . . . . . . . . . . . . . . . . 38 ResponseOptionsLabeledinEnglish. . . . . . . . . . . . . . . . . . . . . . 41 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 MethodologicalConsideration. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 ImplicationoftheMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 AdviceforFurtherResearch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Conclusion.. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . . .. . . 45 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5 ‘VeryHappy’IsNotAlwaysEquallyHappy. . . . . . . . . . . . . . . .. . 47 EvaluationoftheLabelsofResponseOptionsinthe ContextoftheScale. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 TheSemanticJudgmentofFixedWordValue MethodRevisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 TheContextoftheScaleandComparability. . . . . . . . . . . . . . . . . . 48 InterpretationofResponseOptionsoftheSameItemin DifferentLanguages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 TranslationandComparability. . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 DifferenceinInterpretationBetweenDifferentLanguages. . . . . . . 51 DifferentQuestionsAbouttheSameTopic,ButwithIdentical ResponseOptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 TheLeadingQuestionandComparability. . . .. . . . .. . . . .. . . . .. 54 DifferenceinInterpretationGivenaDifferenceinLeading Question. . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . . . . . . . . . .. . 55
The list of books you might like
