Generating and Interpreting Referring Expressions in Context Dustin Arthur Smith

111 Pages·2013·2.06 MB·English

Generating and Interpreting Referring Expressions in
Context
by
Dustin Arthur Smith
B.S., Wake Forest University (2005)
S.M., Massachusetts Institute of Technology (2007)
Submitted to the Program of Media Arts and Sciences, School of Architecture and
Planning
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
September 2013
© MIT, 2013.
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Program of Media Arts and Sciences, School of Architecture and Planning
September 1, 2013
Certified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Henry A. Lieberman
Principal Research Scientist, MIT Media Lab
Thesis Supervisor
Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Pattie Maes
Associate Academic Head, Program of Media Arts and Sciences
Generating and Interpreting Referring Expressions in Context
by
Dustin Arthur Smith
SubmittedtotheProgramofMediaArtsandSciences,SchoolofArchitectureandPlanning
onSeptember1,2013,inpartialfulfillmentofthe
requirementsforthedegreeof
DoctorofPhilosophy
Abstract
Referring expressions with vague and ambiguous modifiers, such as “a quick visit” and “the
bigmeeting,”aredifficultforcomputers tointerpretbecausetheirmeaningsare definedinpart
by context. For the hearer to arrive at the speaker’s intended meaning, he must consider the
alternativedecisionsthatthespeakerwasfacedwithincontext. Toaddressthesechallenges,I
proposeanewapproachtobothgeneratingandinterpretingreferringexpressionsbasedonbelief-
stateplanning andplan recognition. Planning inbelief spaceoffers awayto capturereferential
uncertaintyandtheincrementalnatureofgeneratingandinterpretation,becauseeachbeliefstate
representsacompleteinterpretation. Thecontributionsofmythesisareasfollows:
(1) A computational model of reference generation and interpretation that is fast, incremental,
andnon-deterministic. ThismodelincludesalexicalsemanticsforafragmentofEnglishnoun
phrases,whichspecifiestheencodedmeaningsofdeterminers(quantifiersandarticles),gradable
andambiguousmodifiers. Itperformsinrealtime,evenwhenthehypothesisspacegrowsvery
large. Because it’s incremental, it avoids considering possibilities that will later turn out to be
irrelevant.
(2) The integration of generation and interpretation into a single process. Interpretation is
guidedbycomparisontoalternativesproducedbythegenerationmodule. Whenfacedwithan
underspecifieddescription,thesystemuseswhatitcouldhavesaid andcomparesthattowhatthe
speakerdidsay. Reasoningaboutalternativedecisionsfacilitatesinferencesofthissort: “Sheate
someofthetuna"meansnotallofit,otherwiseyouwouldhavesaid,“Sheatethetuna."
Thisapproachhasbeenimplementedandevaluatedusingacomputationalmodel,AIGRE.Ialso
created a testbed for comparing human judgments of referring expressions to those produced
by our algorithm (or others). In an online experiment with Mechanical Turk, we attained 94%
coverageofhumanresponsesinasimplegeometricaldomain,aswellaslower,butstillencouraging,
coverageinamorecomplex,real-worlddomain.
Themodel,AIGRE,demonstratesthatmanagingthevaguenessandambiguityinnaturallanguage,
whilestillnoteasy,isneverthelesspossible. Thedaywherewewillroutinelytalktoourcomputers
inunconstrainednaturallanguageisnotfaroff.
ThesisSupervisor: HenryA.Lieberman
Title: PrincipalResearchScientist,MITMediaLab
Generating and Interpreting Referring Expressions in
Context
by
Dustin Arthur Smith
Submitted to the Program of Media Arts and Sciences, School of Architecture and
Planning
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
September 2013
© MIT, 2013.
Reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Marvin Minsky
Professor of Media Arts & Sciences
MIT Media Lab
Generating and Interpreting Referring Expressions in
Context
by
Dustin Arthur Smith
Submitted to the Program of Media Arts and Sciences, School of Architecture and
Planning
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
September 2013
© MIT, 2013.
Reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Agustín Rayo
Associate Professor
MIT Department of Linguistics & Philosophy
ThisthesisisdedicatedtoPushSingh
1972-2006

Acknowledgments
I owe my advisor, Henry Lieberman, enormous gratitude for sharing his invaluable advice to-
ward research and all aspects of my intellectual life. I thank my readers: Marvin Minsky for a
decadeofsupport,inspiringideas,andcourageouslybigthinking. AgustínRayo,forhisenergy,
encouragementandprecisewaysforthinkingaboutvagueness.
None of my pursuits would have been possible without my parents unwavering support and
love—Art, Eric & Sharon. I’m thankful for a large and loving family, especially my siblings:
Alexis, Allison, Erin & Matt, and Roy; and my grandparents: Joe & Shelia, and Lily & Arthur.
Unfortunatelymy grandfather, ArthurNorbertSmith,is nolongeraround; hewasan engineer
withagreatsenseofhumorandabilitytogettothecoreofmattersquickly. Iamgratefulforthe
supportfromtheAhren,Brown,Coleman,CabralandCuneofamilies. Marcio,forhislove,humor
and companionship. Marcio, Erin, Art and Sharon deserve special recognition for copyediting
undershortdeadlines.
Iamgratefulforallmyfriends; they’vekeptmeinsanethroughouttheyears, especially: Simon,
Bo,Scotty,theNicos,Nick,Spencer,Ernie,Kristin,Kyle,Karthik,Dan,RyanandChip.
IwouldliketothankNicholasNegroponteforcreatingtheMediaLab,andallofthesponsorswho
supportit. Thelabhasgivenmetremendousfreedomandsupportovertheyearsandexposedme
topeoplewithamazingpassionandenthusiasm.
Myresearch hasbenefited fromthe supportand discussionofnumerous friends, colleaguesand
administratorsatMITandelsewhere,including: ClioAndris,KenArnold,NathanArtz,Barbara
Barry,Walter Bender,Kristina Bonikowski,AmnaCarreiro, Brendan Casey, YinFuChen, Jaewoo
Chung,DavidDalrymple,BenDeen,KarthikDinakar,NickDufour,IanEslick,CharlesFracchia,
ChristopherFry, ErrinFulp,Felice Gardner,CatherineHavasi, KasiaHayden,Mako Hill,Roarke
Horstmeyer,GlebKuznetsov,SimonLaflamme,CameronLevy,HugoLiu,MarcusLowe,Anomal
Madan,PattieMaes,SeanMarkan,BillMartin,electronicMax,PranavMistry,ManasMittal,Bo
Morgan,ElanPavlov,NicolasPinto,TomRoberts,LukeSchiefelbein,AmeliaServi,PushSingh,
RobSpeer,GerrySussman,AleaTeeters,ScottyVercoe,Ben&RebeccaWaberandAaronZinman.
Thankyou!
7
8
Contents
TitlePage 1
Abstract 2
Acknowledgments 7
Contents 9
1 LinguisticReferenceinContext 13
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2 Whatarereferringexpressions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.3 Whatdoreferringexpressionsmean? . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.3.1 Referringexpressionshaveintensionalandextensionalmeanings . . . . . 16
1.4 Architecturalconstraintsonthereferenceprocesses . . . . . . . . . . . . . . . . . 18
1.4.1 FastandIncremental . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.2 Non-deterministic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.5 Constrainingcontext . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.5.1 Identifyingrelevantcontextualfactors . . . . . . . . . . . . . . . . . . . . 22
1.5.2 Collectingdatainaconstrainedcommunicationsetting . . . . . . . . . . 23
1.5.3 ContextSet=ReferentialDomain+combinatoricpossibilities . . . . . . . 25
1.6 Characterizingthetworeferencetasks . . . . . . . . . . . . . . . . . . . . . . . . 26
1.6.1 Formalizingthereferringexpressiongeneration(REG)task . . . . . . . . 26
1.6.2 Formalizingthereferringexpressioninterpretation(REI)task . . . . . . . 27
1.6.3 Definingcommunicationsuccessandfailure . . . . . . . . . . . . . . . . . 28
1.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
9
2 AIGRE,abelief-stateplanningapproachtoreference 29
2.1 GeneratingReferringExpressionsasPlanning . . . . . . . . . . . . . . . . . . . . 29
2.2 InterpretationasPlanRecognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3 RepresentingLexicalItemsasActions . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.4 RepresentingIntensionsasBeliefStates . . . . . . . . . . . . . . . . . . . . . . . . 33
2.4.1 Beliefstates,arepresentationforuncertainty . . . . . . . . . . . . . . . . 34
2.4.2 Quantifyingtheextensionalcomplexityofmeaning . . . . . . . . . . . . . 35
2.4.3 Beliefstateimplementationdetails . . . . . . . . . . . . . . . . . . . . . . 36
2.4.4 Actionimplementationdetails . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3 BuildingtheLexicon 39
3.1 Referringtosets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.1 PluralsandCardinals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Referringtosetsofsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Representingunspecificmeanings . . . . . . . . . . . . . . . . . . . . . . 41
3.2.2 AnunconventionaltreatmentofEnglishquantifiers . . . . . . . . . . . . 41
3.2.3 Representingfreechoiceininterpretations . . . . . . . . . . . . . . . . . . 42
3.2.4 Asimplemeaningforthedefinitearticle . . . . . . . . . . . . . . . . . . . 43
3.2.5 RepresentingNegation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3 Afunctionforderivingtheextensionfromtheintension . . . . . . . . . . . . . . 45
3.4 Referringtomutuallyincompatiblesets . . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.1 LexicalAmbiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.2 VaguenessandGradability . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.5 Ellipsisandtheproblemofmissingwords . . . . . . . . . . . . . . . . . . . . . . 56
3.5.1 Representingsyntacticstateintheintension . . . . . . . . . . . . . . . . . 57
3.5.2 Assumingdefaultactionswhenneeded . . . . . . . . . . . . . . . . . . . . 57
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4 ControllingSearch 59
4.1 AIGRE’ssearchframework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2 Definingthesearchcomponentsforbothreferencetasks . . . . . . . . . . . . . . 62
4.2.1 Goal-testfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
10