The Hanson-Yudkowsky AI-Foom Debate PDF
Preview The Hanson-Yudkowsky AI-Foom Debate
The Hanson-Yudkowsky AI-Foom Debate Robin Hanson and Eliezer Yudkowsky The Hanson-Yudkowsky AI-Foom Debate The Hanson-Yudkowsky AI-Foom Debate RobinHansonandEliezerYudkowsky MIRI RobinHansonisanassociateprofessorofeconomicsat GeorgeMasonUniversityandaresearchassociateatthe FutureofHumanityInstituteofOxfordUniversity. EliezerYudkowskyisaResearchFellowattheMachine IntelligenceResearchInstituteandistheforemost researcheronFriendlyAIandrecursiveself-improvement. Publishedin2013bythe MachineIntelligenceResearchInstitute, Berkeley94704 UnitedStatesofAmerica intelligence.org ReleasedundertheCreativeCommons Attribution-NonCommercial-ShareAlike3.0Unported license. CCBY-NC-SA3.0 isbn-10:1939311039 isbn-13:978-1-939311-03-0 (pdf) TheMachineIntelligenceResearchInstitutegratefully acknowledgeseachoftheauthorsfortheirideasand contributionstowardthisimportanttopic.Specialthanks toCarlShulmanandJamesMillerfortheirguestpostsin thedebate. Allchaptersandcommentsarewrittenbyandcopyright theirrespectiveauthors.BookcovercreatedbyWeni PratiwiandAlexVermeer. iv Contents Contents v Foreword viii I Prologue 1 FundUberTool?—RobinHanson 3 2 EngelbartasUberTool?—RobinHanson 6 3 FriendlyTeams—RobinHanson 9 4 FriendlinessFactors—RobinHanson 12 5 TheWeakInsideView—EliezerYudkowsky 18 6 SettingtheStage—RobinHanson 25 7 TheFirstWorldTakeover—EliezerYudkowsky 29 8 Abstraction,NotAnalogy—RobinHanson 38 9 WhenceYourAbstractions?—EliezerYudkowsky 43 II MainSequence 10 AIGoFoom—RobinHanson 51 11 OptimizationandtheIntelligenceExplosion—Eliezer Yudkowsky 55 12 Eliezer’sMeta-levelDeterminism—RobinHanson 65 v Contents 13 ObservingOptimization—EliezerYudkowsky 72 14 Life’sStoryContinues—EliezerYudkowsky 81 15 EmulationsGoFoom—RobinHanson 88 16 BrainEmulationandHardTakeoff—CarlShulman 107 17 BillionDollarBots—JamesMiller 117 18 SurprisedbyBrains—EliezerYudkowsky 120 19 “Evicting”BrainEmulations—CarlShulman 130 20 Cascades,Cycles,Insight...—EliezerYudkowsky 135 21 WhenLifeIsCheap,DeathIsCheap—RobinHanson 146 22 ...Recursion,Magic—EliezerYudkowsky 152 23 Abstract/DistantFutureBias—RobinHanson 161 24 Engelbart:InsufficientlyRecursive—EliezerYudkowsky 166 25 TotalNanoDomination—EliezerYudkowsky 175 26 DreamsofAutarky—RobinHanson 189 27 TotalTechWars—RobinHanson 196 28 SingletonsRuleOK—EliezerYudkowsky 204 29 StuckInThroat—RobinHanson 216 30 DisappointmentintheFuture—EliezerYudkowsky 219 31 IHeartCyc—RobinHanson 225 32 IstheCity-ularityNear?—RobinHanson 231 33 RecursiveSelf-Improvement—EliezerYudkowsky 235 34 WhitherManufacturing?—RobinHanson 253 35 HardTakeoff—EliezerYudkowsky 258 36 TestNear,ApplyFar—RobinHanson 270 37 PermittedPossibilitiesandLocality—EliezerYudkowsky273 38 UnderconstrainedAbstractions—EliezerYudkowsky 287 39 BewareHockey-StickPlans—RobinHanson 298 40 EvolvedDesires—RobinHanson 303 41 SustainedStrongRecursion—EliezerYudkowsky 314 42 FriendlyProjectsvs.Products—RobinHanson 329 43 IsThatYourTrueRejection?—EliezerYudkowsky 333 vi RobinHansonandEliezerYudkowsky 44 SharedAIWins—RobinHanson 339 45 ArtificialMysteriousIntelligence—EliezerYudkowsky 344 46 WrappingUp—RobinHanson 352 47 TrueSourcesofDisagreement—EliezerYudkowsky 359 48 TheBadGuyBias—RobinHanson 371 49 Disjunctions,Antipredictions,Etc.—EliezerYudkowsky 374 50 AreAIsHomoEconomicus?—RobinHanson 381 51 TwoVisionsOfHeritage—RobinHanson 385 52 TheMechanicsofDisagreement—EliezerYudkowsky 390 III Conclusion 53 WhatCoreArgument?—RobinHanson 397 54 WhatIThink,IfNotWhy—EliezerYudkowsky 402 55 NotTakingOvertheWorld—EliezerYudkowsky 409 IV Postscript 56 WeAgree:GetFroze—RobinHanson 419 57 YouOnlyLiveTwice—EliezerYudkowsky 423 58 Hanson-Yudkowsky Jane Street Debate 2011—Robin HansonandEliezerYudkowsky 431 59 DebatingYudkowsky—RobinHanson 493 60 FoomDebate,Again—RobinHanson 499 61 AI-FoomDebateSummary—KajSotala 505 62 IntelligenceExplosionMicroeconomics—EliezerYud- kowsky 562 Bibliography 715 vii Foreword Inlate2008,economistRobinHansonandAItheoristEliezerYud- kowskyconductedanonlinedebateaboutthefutureofartificialintel- ligence,andinparticularaboutwhethergenerallyintelligentAIswill beabletoimprovetheirowncapabilitiesveryquickly(a.k.a.“foom”). James Miller and Carl Shulman also contributed guest posts to the debate. Theoriginaldebatetookplaceinalongseriesofblogposts,which are collected here. This book also includes a transcript of a 2011 in-person debate between Hanson and Yudkowsky on this subject, a summary of the debate written by Kaj Sotala, and a 2013 techni- calreportonAItakeoffdynamics(“intelligenceexplosionmicroeco- nomics”)writtenbyYudkowsky. Commentsfromtheauthorsareincludedattheendofeachchap- ter,alongwithalinktotheoriginalpost. Thecuriousreaderisen- couraged to use these links to view the original posts and all com- viii RobinHansonandEliezerYudkowsky ments. This book contains minor updates, corrections, and addi- tionalcitations. ix
Description:The list of books you might like
