thesis - Alex McLean

172 Pages·2012·10.68 MB·English

Artist-Programmers
and
Programming Languages for the Arts
Christopher Alex McLean
esis submied to Goldsmiths,
University of London,
for the degree of Doctor of Philosophy.
October 2011
DedicatedtoJess,Harvey,Mum,Dad,AndreaandStuartandtherestofmyfamily,to
whomIoweeverything,andwiththememoryofthosewhosupportedthisworkbut
passedawaybeforeseeingitsend,namelymyFather-in-LawDr. DavidElmore,and
mycatOlga.
2
Abstract
Weconsidertheartist-programmer,whocreatesworkthroughitsdescriptionassourcecode.
e artist-programmer grandstands computer language, giving unique vantage over human-
computerinteractioninacreativecontext. Wefocusonthehumaninthisrelationship,noting
thathumansuseanamalgamoflanguageandgesturetoexpressthemselves. Accordinglywe
exposethedeeprelationshipbetweencomputerlanguagesandcontinuousexpression, exam-
ining how these realms may support one another, and how the artist-programmer may fully
engagewithboth.
Our argument takes us up through layers of representation, starting with symbols, then
words,languageandnotation,toconsidertherolethattheserepresentationsmayplayinhu-
mancreativity. Weformacross-disciplinaryperspectivefrompsychology,computerscience,
linguistics, human-computer interaction, computational creativity, music technology and the
arts.
We develop and demonstrate the potential of this view to inform arts practice, through
the practical introduction of soware prototypes, artworks, programming languages and im-
provisedperformances. Inparticular,weintroduceworkswhichdemonstratetheroleofper-
ception in symbolic semantics, embed the representation of time in programming language,
include visuospatial arrangement in syntax, and embed the activity of programming in the
improvisationandexperienceofart.
3
Anowledgements
ImustfirstthankGeraintWigginsforhisaentivesupervision,mentorshipandinsights,help-
ingmealongafascinatingjourney. Ihavegreatlybenefitedfromhisstrivingfocusonhonest,
rigourousresearch,andIdonotdoubtthathissupportthroughMScandPhDresearchrequired
aleapoffaithonhispart,showingbothanopenmindanddeepgenerosity. Mythanksextend
toalltheothermembersoftheIntelligentSoundandMusicSystemsgroupfordailylunchtime
discussions on all things, I thank you all and must name my co-supervisor Mark d’Inverno,
mycollaboratorJamieForth,andalsoDanielJonesforhisgenerousanddetailedcriticism. My
thanks extend further to envelop the department as a whole, including Tim Blackwell, Mick
Grierson, Janis Jefferies, Frederic Leymarie and Robert Zimmer for their encouragement and
support. Ithasbeenagreatprivilegetoworkamongstsuchcreativeminds.
epathleadingtothisthesisreallybeganintheyear2000,whenAdrianWardencouraged
me to experiment with algorithmic composition. We formed the generative music (and later,
livecoding)bandSlub,joinedbyDaveGriffiths. Manyoftheideasexploredherehaverootsin
thiscollaboration,andIcannotbeartoimaginelifewithoutSlub. Mythankstoboth.
MyforaysintoElectronicandSowareArtalsopavedthewaytothepresentwork,made
possible with the support, encouragement and collaboration of Saul Albert, Amy Alexander,
Geoff Cox, Olga Goriunova, Douglas Repeo, EunJoo Shin and Alexei Shulgin. My journey
into research also received great support from my former colleagues at state51, particularly
Paul Sanders, who has always been generous with rich ideas, encouragement and criticism.
ankyouall.
Hemaynotremember,butmyfirstthoughtsaboutaresearchdegreewereplantedbyRob
Saunders,theAIandCreativityresearcher. Wehavehardlycrossedpathssince,butwithouthis
encouragementImaynothavemadetheleap. Followingthis,SimonEmmersonwasextremely
supportive, including helping secure funding for my MSc, making the whole thing possible.
anksboth.
Speakingofwhich,ImustalsosincerelythankthePRSFoundationforfundingthefeesfor
my MSc in Arts Computing, and the EPSRC for the full funding of my PhD studentship. It is
4
wonderfulthatthesegreatinstitutionsfundunusualareasofresearch.
e final stage of writing up the present thesis was completed within the OAK group at
theUniversityofSheffield,specialthanksgotoSimonTuckerandFabioCiravegnaforhelping
makethetransitionfromresearchstudenttoresearchersoenjoyable.
I have met with many influential minds over the last four years, more than I could list
here. ButIreallymustthankSamAaron,SarahAngliss,DavidBausola,RenickBell,Vaughan
Bell,RossBencina,AlanBlackwell,GrahamBooth,OllieBown,AndrewBrown,JohnBurton,
ArturoCastro,GrahamColeman,ChristofDamian,EleanorDare,BenFields,PhilipGalanter,
Joel Gethin-Lewis, Bruno Gingras, Sam Freeman, Mahew Fuller, Alex Garacotche, Enrico
Glerean, Jake Harries, Cormac Heron, Sco Hewi, Tom Holley, Sampath Jagannathan, Sergi
Jorda, Ryan Jordan, David Lewis, or Magnusson, Charles Mahews, Jon McCormack, An-
drew McFarland, Daniel Mu(cid:127)llensiefen, Rob Myers, Eric Namour, Ashraf Nehru, Julian Oliver,
Fredrik Olofsson, Irini Papadimitriou, Rob Partington, Aneeta Patel, Marcus Pearce, Hester
Reeve,ChristopheRhodes,ChrisRoast,JulianRohrhuber,OlivierRuellet,AshSagar,Andrew
Sorensen,DanStowell,AndyStuder,PatrickTresset,KassenOud,MichelePasin,JuanRomero,
GeWang,MitchellWhitelaw,ScoWilson,MahewYee-King,andeveryoneinDorkbot,eu-
gene,openlab,POTACandTOPLAP.
I must also thank those who having bought me drinks, proofread my work, shared their
ideasand/orofferedhelpfuladvice,havefailedtofindtheirnameabove. Myspecialthanksto
you,andIapologiseforforgeingaboutyouatthiscrucialmoment,especially myfriendand
collaborator Nick Collins, I have reached this point only by relying upon Nick as a deep well
ofinspirationandenthusiasmoverthepast10years,andhereallydeservesbeer.
Finally I would like to thank Jess Elmore for her understanding and support, and our son
HarveyMcLean. HarveyisonemontholderthanmyPhDstudentship,andhisspeedydevelop-
mentintoawalking,talking,lovingandcomplaininghumanbeinghasbeenextremelyuseful
inkeepingmealerttothepassingoftime.
5
Contents
1 Introduction 13
1.1 Artist-Programmers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.1.1 Computerart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.1.2 GenerativevsSowareart . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.1.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2 ProgrammingLanguagesfortheArts . . . . . . . . . . . . . . . . . . . . . . . 17
1.3 Aims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5 Originalcontributionstoknowledge . . . . . . . . . . . . . . . . . . . . . . . . 19
2 Symbols 20
2.1 Situatingsymbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 Symbolsincognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2.1 Mentalimagery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.2 MentalImageryandProgramming . . . . . . . . . . . . . . . . . . . . 26
2.2.3 DualCodinginSourceCode . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.4 LanguageandSituatedSimulation . . . . . . . . . . . . . . . . . . . . 29
2.2.5 ConceptualSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2.6 Metaphor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.3 AnthropomorphismandMetaphorinProgramming . . . . . . . . . . . . . . . 33
2.4 Synaesthesia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.5 Artisticsynaesthesia? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.6 AcidSketching–SemanticImagery . . . . . . . . . . . . . . . . . . . . . . . . 38
2.7 Phonemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.8 Microphone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.9 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6
C
3 Words 45
3.1 PerceivingSpeechasMovement . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2 VocableWordsinMusicTradition . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3 Timbre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.3.1 Multi-DimensionalScaling . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.3.2 GroundingTimbreinMovement . . . . . . . . . . . . . . . . . . . . . 53
3.3.3 MusicofTimbre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.3.4 DefiningTimbre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.5 Timbralanalogies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.3.6 Soundsourcemodelling . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3.7 UniversalityofTimbre . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.4 Articulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4.1 RhythminSpeech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.4.2 Soundpoetry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.4.3 WordsinMusicTechnology . . . . . . . . . . . . . . . . . . . . . . . . 61
3.5 Vocablesynthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.5.1 Babble-vocableKarplus-Strongsynthesis . . . . . . . . . . . . . . . . 63
3.5.2 Mesh-vocablewaveguidesynthesis . . . . . . . . . . . . . . . . . . . 64
3.5.3 Vocablemanipulationandanalysis . . . . . . . . . . . . . . . . . . . . 66
3.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4 Language 68
4.1 NaturalandComputerLanguage . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.2 Music,LanguageandCognitiveSemantics . . . . . . . . . . . . . . . . . . . . 72
4.3 DeclarativevsImperative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.4 DomainSpecificLanguageforPaern . . . . . . . . . . . . . . . . . . . . . . . 77
4.5 Tidal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5.2 Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.5.3 Paerngenerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.5.4 Parsingstringsintopolymetricpaerns . . . . . . . . . . . . . . . . . 84
4.5.5 Paerncombinators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.6 OpenSoundControlpaerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.6.1 Scheduling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.6.2 Sendingmessages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.6.3 Useinimprovisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7
C
4.6.4 Futuredirections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5 Notation 95
5.1 CognitiveDimensionsofNotation . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.2 NotationinTime. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.3 Visualnotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.3.1 PatcherLanguages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.4 NotationandMentalImagery . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.5 GeometryinSyntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.6 VisualProgrammingwithTexture . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.6.1 Geometricrelationships . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.6.2 UserInterface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.6.3 TextureinPractice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.6.4 CognitiveDimensionsofTexture . . . . . . . . . . . . . . . . . . . . . 112
5.6.5 Futuredirections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6 Creativity 115
6.1 Programmerculture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.2 Conceptsincreativebehaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.3 CreativeProcesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.3.1 CreativeProcessofBricolage . . . . . . . . . . . . . . . . . . . . . . . 122
6.4 SymbolsandSpace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.5 Componentsofcreativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6.6 ProgramminginTime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.7 Embodiedprogrammers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.8 LiveCodinginPractice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.8.1 “Obscurantismisdangerous. Showusyourscreens.” . . . . . . . . . . 131
6.8.2 CognitiveDimensionsofLiveCoding. . . . . . . . . . . . . . . . . . . 132
6.9 Livecodersoncomputationalcreativity . . . . . . . . . . . . . . . . . . . . . . 133
6.9.1 esubjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.9.2 Creatinglanguage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.9.3 Codeandstyle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
6.9.4 Livecodingasanovelapproach . . . . . . . . . . . . . . . . . . . . . . 135
6.9.5 Computationalcreativity . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8
C
6.9.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.10 Slub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.10.1 ReflectionsfollowingaSlubperformance. . . . . . . . . . . . . . . . . 140
6.11 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
7 Futuredirections 144
7.1 eFreedomofInterpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
7.2 Sowareengineeringstandards . . . . . . . . . . . . . . . . . . . . . . . . . . 146
7.3 Cyclicrevisioncontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
A Livecodingsurvey 166
A.1 Definitionsoflivecoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
A.2 Comparinglivecoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
9
Resear outputs
e present thesis has been developed and disseminated through the following activities and
publications.
Practice
AcidSketing(x2.6). Exploringdrawnanaloguesymbolsindigitalart. InstalledintheDork-
bottentattheBigChillFestival,August2010.
Microphone (x2.8). Exploring vocal analogue symbols in digital art. Created with EunJoo
Shin, installed at the “Unleashed Devices” group show at Watermans Gallery, September/Oc-
tober2010.
Babble (x3.5.1). System for improvising musical timbre with wrien words. A permanent
on-lineworkcommissionedbyArnolfini,BristolinNovember2008.
Tidal(x4.4)andTexture(x5.6). Computerlanguagedesignedfortheimprovisationofmusical
paern,withvisuospatialnotation. Releasedwithafree/opensourcelicense,andusedbythe
presentauthorinpublicmusicperformances.
LiveCodingPerformances(x6.8). efollowingisalistoflivecodingperformancesfeatur-
ing the present author during the period of research. ose marked with an asterisk were as
partofacollaborationwithAdrianWardand/orDaveGriffithsasSlub,whichisdiscussedin
x6.10.
(cid:15) Area10,Peckham,London,12thApril2008*
(cid:15) IvyHouse,Nunhead,London,4thApril2008*
(cid:15) ursdayClub,Goldsmiths,London,5thJune2008*
(cid:15) PublicLife,London,18thJuly2008*
(cid:15) SecretGardenParty,Cambridgeshire,26thJuly2008*
10