The Technological Singularity: Managing the Journey

263 Pages·2017·1.99 MB·English

by  Victor Callaghan, James Miller, Roman Yampolskiy, Stuart Armstrong

THE FRONTIERS COLLECTION
Series editors
Avshalom C. Elitzur
Iyar The Israel Institute for Advanced Research, Rehovot, Israel
e-mail: [email protected]
Zeeya Merali
Foundational Questions Institute, Decatur, GA 30031, USA
e-mail: [email protected]
T. Padmanabhan
Inter University Centre for Astronomy and Astrophysics (IUCAA), Pune, India
e-mail: [email protected]
Maximilian Schlosshauer
Department of Physics, University of Portland, Portland, OR 97203, USA
e-mail: [email protected]
Mark P. Silverman
Department of Physics, Trinity College, Hartford, CT 06106, USA
e-mail: [email protected]
Jack A. Tuszynski
Department of Physics, University of Alberta, Edmonton, AB T6G 1Z2, Canada
e-mail: [email protected]
Rüdiger Vaas
Bild der wissenschaft, Redaktion Astronomie, Physik, 70771 Leinfelden‐
Echterdingen, Germany
e-mail: [email protected]
THE FRONTIERS COLLECTION
Series Editors
A.C. Elitzur Z. Merali T. Padmanabhan M. Schlosshauer
M.P. Silverman J.A. Tuszynski R. Vaas
The books in this collection are devoted to challenging and open problems at the
forefront of modern science, including related philosophical debates.In contrast to
typical research monographs, however, they strive to present their topics in a
manner accessible also to scientifically literate non-specialists wishing to gain
insight into thedeeper implicationsandfascinating questions involved.Taken asa
whole,theseriesreflectstheneedforafundamentalandinterdisciplinaryapproach
to modern science. Furthermore, it is intended to encourage active scientists in all
areas to ponder over important and perhaps controversial issues beyond their own
speciality. Extending from quantum physics and relativity to entropy, conscious-
ness and complex systems—the Frontiers Collection will inspire readers to push
back the frontiers of their own knowledge.
More information about this series at http://www.springer.com/series/5342
Forafulllistofpublishedtitles,pleaseseebackofbookorspringer.com/series/5342
⋅
Victor Callaghan James Miller
⋅
Roman Yampolskiy Stuart Armstrong
Editors
THE TECHNOLOGICAL
SINGULARITY
Managing the Journey
123
Editors
Victor Callaghan RomanYampolskiy
Schoolof Computer andElectrical Department ofComputer Engineering
Engineering andComputer Science
University of Essex University of Louisville
Essex Louisville,KY
UK USA
James Miller Stuart Armstrong
EconomicsFaculty Faculty of Philosophy
Smith College University of Oxford
Northampton, MA Oxford
USA UK
ISSN 1612-3018 ISSN 2197-6619 (electronic)
THEFRONTIERS COLLECTION
ISBN978-3-662-54031-2 ISBN978-3-662-54033-6 (eBook)
DOI 10.1007/978-3-662-54033-6
LibraryofCongressControlNumber:2016959969
©Springer-VerlagGmbHGermany2017
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
foranyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringer-VerlagGmbHGermany
Theregisteredcompanyaddressis:HeidelbergerPlatz3,14197Berlin,Germany
Foreword
The technological singularity is based on the ongoing improvements in and power
ofartificialintelligence (AI)which,atsome point,willenableintelligent machines
to design successive generations of increasingly more powerful machines, even-
tuallycreatingaformofintelligencethatsurpassesthatofhumans(Kurzweil2006).
The technological singularity (or ‘singularity’ as I shall refer to it from hereon) is
theactualpointpastwhicheventsarebeyondthecontrolofhumans,resultingeither
in humans upgrading (with implants) to become cyborgs or with intelligent
machinestakingcompletecontrol.Aswellasbeingnotparticularlygoodnewsfor
ordinary humans, it is clearly an extremely critical point in time.
Already AI exhibits several advantages over the human form of intelligence.
Theseareperhapsmosteasilywitnessedinsuchtermsasmathematicalprocessing,
memory, multidimensional operation, multi-sensory capabilities, body extension
and,perhapsaboveallelse,heightenedformsofcommunication.Astheseadvances
accelerate, producing artificial general intelligence (AGI) whereby its future capa-
bilitiesmaywellbeimpossibleforhumanstocomprehend.Thisbookseekstogive
us some understanding of what lies ahead. In doing so, I believe the coverage is
balanced, realistic and not at all sensationalist. Whilst the possibility of a major
machinetakeoverissensiblyrecognisedbyallauthors,manyofthearticleslookat
ways of combating the situation or at least of how humans might live with intel-
ligent machines rather than being ruled by them.
This particular work is therefore of considerable importance, containing, as it
does, a focused collection of articles by some of the world’s leading thinkers and
expertsonthetopic.Inthisforeword,Idonotwishtomerelygiveatwo-lineprecis
of each entry but rather I have selected some of the works that caught my eye in
particular. What follows is therefore my completely biased assessment.
Importantly,thetwochaptersbyYampolskiyandSotala,whichformPartI,take
aseriouslookatwhatexactlyAGIisallaboutandwhattherealrisksare.Theygo
ontoconsiderindepthawidevarietyofproandconargumentswithregardtothe
reality of these risks. These include giving a voice to those who feel that any
dangersare(astheycallthem)over-hypedandweneedtodonothingbutwecanall
v
vi Foreword
sleep soundly. In this part, some of the different possible outcomes of the singu-
larityaredescribed indetail,rangingfromanintelligentmachinetakeoverthrough
human-into-cyborgs upgrades but also encapsulating the possibility of a direct
merger involving biological brains in a technological body.
Part II examines various aspects of a potential singularity in greater depth.
Soares’article,forexample,looksatanapproachtoaligntheinterestsanddirection
ofsuper-intelligententitieswiththoseofhumans,orratherhowhumanscanrealign
their own thinking and mode of operation to potentially encompass the rise of
intelligent machines. On an even more conservative note, Barrett and Baum con-
sider ways in which, as they read it, the low probability of an artificial
super-intelligence canbemanaged.Inthisway, therisk isacknowledged but,they
argue, safety features can be built in and human progress can be steered
appropriately.
ZhengandAkhmadconsider,fromaperspectiveofchange agencytheory,how
the course of any singularity will be driven by socio-economic factors and natural
human development. In particular, they map out the path of change. For us to
realise that intelligentmachinescannotonly learn andadapt theirways andgoals,
butalsopassthisontothenextgenerationisanimportantfactor.Thisispickedup
on nicely by Majot and Yampolskiy in their article on recursive, self-improving
machines. They believe that due to natural limits, things may well not turn out as
badassomesuggest.Indeed,theirmessageisthatthesuggestedever-wideninggap
between machine and human intelligence will not be as problematic as was first
thought and that any dangerous situation could be halted.
Part III is entitled reflections and contains, amongst other things, a reprint of
Vinge’s original article on the singularity. This is a sterling contribution that is in
itselfwellworthreading.Inmanyways,itputstherestofthisbookinperspective
andmakesonerealisehowrelativelyrecentisthewholefieldofstudyonwhichthis
bookisbased.Lookingbackmerelyadecade,termssuchas‘humanenhancement’
were very much aimed at individual enhancements, often bringing that individual
backtowardsahumannormforsomeonewhohadlostanabilityduetoanaccident
orillness.Now,because oftheavailable technology andthescientificexperiments
whichhavesincetakenplace,wecanreadilyexploreanddiscussthepossibilitiesof
enhancement beyond a human norm.
In terms of artificial intelligence, just as Alan Turing predicted over 60 years
ago,‘theuseofwordsandgeneraleducatedopinionwillhavealteredsomuchthat
onewillbeabletospeakofmachinesthinkingwithoutexpectingtobecontradicted’
(Turing 1950). Turing also saw machine intelligence to be somewhat different to
humanintelligence,whichwecannowgenerallyseeittobeforallsortsofreasons.
One consequence of this difference is that it can potentially outperform the human
version, rather like an aeroplane, in many ways, outperforms a bird in terms of
flying. The actual instantiations of artificial intelligence and a more powerful arti-
ficial general intelligence therefore form an area of extremely exciting study in
which new discoveries are being made regularly. However, along with many
positive uses comes the threat of the singularity with all its consequences for
present day humanity.
Foreword vii
It would probably be sensible for readers to take on board the messages con-
tainedinthisbooksoonerratherthanlater.Indifferentways,humansareendowing
certain machines with increasing intellectual abilities and deferring to them in an
ever-increasingfashion.Unfortunately,humanswillmostlikelynotbeawarewhen
the singularity has been reached until it is too late. Some pundits have suggested
thatindicatorssuchasthepassingoftheTuringtestmaywellgive usaclearsign.
However, this has been shown to be false (Warwick and Shah 2016). Most likely,
thesingularitywillhappenbeforewerealiseitandbeforewecandoanythingabout
it.Ratherthangettingclosertotheedgeofacliff,aswetakeeachstepforwardwe
will feel that all is OK until we take one step too far and there will be no turning
back—the singularity will be upon us.
Prof. Kevin Warwick
Deputy Vice Chancellor (Research)
Coventry University
Coventry, UK
References
Kurzweil,R.,(2006)“TheSingularityisNear”,Duckworth.
Turing, A. (1950) “Computing machinery and intelligence”. Mind, LIX, pp. 433–460.
doi:10.1093/mind/LIX.236.433.
Warwick,K.andShah,H(2016)“PassingtheTuringTestdoesnotmeantheendofHumanity”,
CognitiveComputation,DOI:10.1007/s12559-015-9372-6,February,2016.
Acknowledgements
We are indebted to many people for the successful completion of this book. We
should start by acknowledging the pivotal role played by Amnon H. Eden,
JamesHMoor,JohnnyH.SorakerandEricSteinhart,theeditorialteamofthe
earlier volume of the Springer Frontiers Collection “Singularity Hypotheses: A
Scientific and Philosophical Assessment”, who’s excellent work opened up the
opportunity for this follow-on publication. In particular, we are especially grateful
to Dr Amnon Eden who spotted the opportunity for this book and who wrote the
originalproposaltoSpringerbeforepassingtheproject,mostgraciously,overtothe
current editorial team. We are also pleased to express our deep gratitude to the
excellent support provided by the Springer team, especially Dr Angela Lahee
Executive Editor, Physics, Springer Heidelberg, Germany for the highly proficient
supportand genuine enthusiasm she hasshown tous throughout the productionof
this book. Without Angela’s support this book would not have been possible.
Likewise, we are grateful to Springer's Indian team, most notably, Shobana
Ramamurthy, for overseeing the final stages of production. We should also
acknowledge the Creative Science Foundation (www.creative-science.org) and
theirheadofmediacommunications,JenniferO’Connor,forhosting the‘callfor
chapters’ (and supporting information) on their website and promoting it
throughout theirnetwork.Finally,we wishto expressour deeply felt thanks tothe
authors who have generously allowed us to reproduce their articles in support of
this book. In particular we wish to thank the authors of the blogs reproduced in
Chapter 14 (Singularity Blog Insights) namely, Eliezer Yudkowsky (Three Major
SingularitySchools),StuartArmstrong(AITimelinePredictions:AreWeGetting
Better), Scott Siskind (No Time Like the Present for AI Safety Work) and Scott
Aaronson (The Singularity is Far). Last, but not least, we wish to thank Vernor
Vinge for his permission to reproduce his visionary 1993 article “The Coming
TechnologicalSingularity:HowtoSurviveinthePost-humanEra”intheappendix
ix
x Acknowledgements
of this book. By way of a concluding reflection, we should like to add that we are
grateful to you, the reader, for taking the time to read this book and, as a conse-
quence, to be part of a collective consciousness that we hope will influence any
upcoming singularity towards an outcome that is beneficial to our world.
Victor Callaghan
James Miller
Roman Yampolskiy
Stuart Armstrong