Programming Mixed Music in ReactiveML

12 Pages·2016·1.04 MB·English

Programming Mixed Music in ReactiveML
Guillaume Baudart, Louis Mandel, Marc Pouzet
To cite this version:
Guillaume Baudart, Louis Mandel, Marc Pouzet. Programming Mixed Music in ReactiveML. FARM
’13 - ACM SIGPLAN Workshop on Functional Art, Music, Modeling and Design, Sep 2013, Boston,
United States. pp.11-22, ￿10.1145/2505341.2505344￿. ￿hal-00850294￿
HAL Id: hal-00850294
https://hal.inria.fr/hal-00850294
Submitted on 6 Aug 2013
HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est
archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
Programming Mixed Music in ReactiveML
GuillaumeBaudart LouisMandel MarcPouzet
E´colenormalesupe´rieuredeCachan Univ.Paris-Sud11 Univ.PierreetMarieCurie
AntennedeBretagne DI,E´colenormalesupe´rieure DI,E´colenormalesupe´rieure
DI,E´colenormalesupe´rieure INRIAParis-Rocquencourt INRIAParis-Rocquencourt
[e-mail] [e-mail] [e-mail]
Abstract
signal
Mixed music is about live musicians interacting with electronic
Antescofo
partswhicharecontrolledbyacomputerduringtheperformance.It
score
allowscomposerstouseandcombinetraditionalinstrumentswith
complex synthesized sounds and other electronic devices. There
are several languages dedicated to the writing of mixed music Antescofo
saacdsopvreaecnstc.seoAdfmsmcoounrsegicfaothlllepomewr,feortrhmaellaoAnwcnestsea:shccooowfmopelolaesncegtrruotoangimceapcnaoarutgspeilnethtdeerwareciatthcwtiiavthne ListSeenqiunegn cMearchine tempopositio
amusician.Howeverthesedomainspecificlanguagesdonotoffer n
theexpressivenessoffunctionalprogramming.
control
WeembedtheAntescofolanguageinareactivefunctionalpro-
messages
gramminglanguage,ReactiveML.Thisapproachofferstothecom- Audio
poser recursion, higher order, inductive types, as well as a sim-
plewaytoprogramcomplexreactivebehaviorsthankstothesyn- Figure1. ArchitectureoftheAntescofosystem.Continuousarrow
chronous model of concurrency on which ReactiveML is built. representpre-treatment,anddottedonesreal-timecommunications
ThisarticlepresentshowtoprogrammixedmusicinReactiveML
throughseveralexamples.
PierreBoulez,PhilippeManoury,MarcoStroppa,NewYorkPhil-
CategoriesandSubjectDescriptors D.3.2[LanguageClassifica- harmonics,BerlinPhilharmonics,andtheRadioFranceOrchestra.
tions]: Concurrent, distributed, parallel languages; H.5.5 [Infor- Figure1describestheorganizationoftheAntescofosystem.It
mationInterfacesandPresentation]:SoundandMusicComputing. iscomposedoftwodistinctsubsystems:alisteningmachineanda
sequencer.Duringaperformance,thelisteningmachineestimates
Keywords Synchronous Programming; Language Embedding; the tempo (i.e., execution speed) and the position of the live per-
MixedMusic;LiveCoding. formersinthescore.Theroleofthesequenceristousetheseinfor-
mationtotriggerelectronicactionsbysendingcontrolmessagesto
1. Introduction amusicprogrammingenvironment.e.g.,Max/MSP.2Then,theen-
vironmentusesthesemessagestohandlecomplexsoundsynthesis,
Technicalprogressessincethe1950’shaveledcomposerstowrite managelights,etc.
music mixing live performers with electronic parts. At first, per- ThelanguageofAntescofo[9]isadescriptivelanguageinspired
formers simply followed a pre-registered magnetic band. But the by classical western music notation: a score is an interleaving of
advances in computing rapidly allowed real interaction between instrumental notes and electronic actions. An original aspect of
musiciansandelectronicparts. thislanguageisthatitspecifiessynchronizationanderror-handling
DevelopedatIRCAM,Antescofo[5]1isastate-of-the-artscore strategiesfortheelectronicactions.Itmeansthatduringtheperfor-
followingsystemdedicatedtomixedmusic.Since2008,Antescofo mance,dependingonthetempovariationsandtheerrorsmadeby
hasbeenusedinthecreationofmorethan40originalmixedelec- themusicianorthelisteningmachine,thesequencerwillreactina
tronicpiecesbyworldrenownedartistsandensembles,including waywhichisdefinedbythecomposer.
ThesequencerofAntescofoisasynchronousreactivesystem:
1http://repmus.ircam.fr/antescofo itcontinuouslyreadsinputsfromthelisteningmachine,produces
outputs to the musical environment and is subject to the strong
timingrequirementrelatedtoeartolerance(typically30ms).Syn-
chronous languages [2] have been invented and are used widely3
Permissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalor forprogrammingawiderangeofcriticalcontrolsoftware:onboard
classroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributed
controloftrainsandtrackinterlocking,fly-by-wireandcockpitdis-
forprofitorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitation
onthefirstpage.CopyrightsforcomponentsofthisworkownedbyothersthanACM playsforplanes,etc.Therefore,theywouldhavebeenwellsuited
mustbehonored.Abstractingwithcreditispermitted.Tocopyotherwise,orrepublish, forprogrammingthesequencer.But,toensurestaticallythatpro-
topostonserversortoredistributetolists,requirespriorspecificpermissionand/ora grams execute in bounded time and memory, synchronous lan-
[e-mail].
FARM’13, September28,2013,Boston,MA,USA.
Copyright(cid:13)c 2013ACM978-1-4503-2386-4/13/09...$15.00. 2http://cycling74.com/
http://dx.doi.org/10.1145/2505341.2505344 3http://www.esterel-technologies.com
guageslikeLustreandEsterelintentionallyforbidthedynamiccre-
aoqtvuieoernnccooemfrpienrotRhceiesascsleitmisviewtMahtiiLcohn[1,is]w,aaenfeheaaxtvuteerneismoiofpntlheomefAethnnetteefduscnaocnftoiAolnnaantelgsluacanoggfoeu.asTgeoe- ""!!4444 ## ## ## ## ## ## ## ## ## ## $$ ## ## $$
5
OCaml4withsynchronousparallelism.Notethatweareinterested
irneltiheseolanntghueaagcetuanaldAthnetessecqoufeonlcisetre,nthineglimnkacwhiitnhe.livemusiciansstill 5"! # # # # # # # # # # # # # $ # $
Contribution of the Paper The main contribution of this paper "le[!t 1#j.a#0c,q#u(e#Fs,#4=);# 1.0#,#(#G,#4)#; 1#.0,#(A##,4)$; 1.0#, (##F,$4);
is to show, through a collection of examples, the benefit of the
1.0, (F,4); 1.0, (G,4); 1.0, (A,4); 1.0, (F,4);
embedding of Antescofo in ReactiveML for programming mixed
1.0, (A,4); 1.0, (Bf,4); 2.0, (C,5); ... ]
music.Thisembeddingcombinestheexpressivenessofthetwo.It
facilitates the creation process for the composer by using higher- Figure2. ThetraditionalFrenchmusicalroundFre`reJacquesand
ordercombinators.Itcanbeusedtoprototypenewprogramming thebeginningofitsrepresentationinourdatastructures.
constructs for Antescofo. In all our experiments, response times
were less than the reaction time of human ear, using the current
versionoftheReactiveMLcompilerandrun-time. Now that we can represent musical structures, we write func-
The paper is organized as follows. Section 2 presents Reac- tionstomanipulatethem.Forinstance,thefollowingdefinesfunc-
tiveMLthroughasimplelibraryforelectronicmusic.InSection3, tionstotransposeasequence.5
weintroducethemainfeaturesofthe Antescofo language.Then,
let move_pitch inter p =
Section4illustratesourapproachonacompleteexample.InSec-
pitch_of_int ((int_of_pitch p) + inter)
tion5,weshowhowthisworkcanbeusedtodesignnewkindsof
val move_pitch: int -> pitch -> pitch
interactionsbetweenaliveperformerandelectronicparts.Related
workisdiscussedinSection6andweconcludeonSection7.
let transpose inter sequence =
The code presented in the article is available on a companion
List.map
web site: http://reactiveml.org/farm13. Links to music or
(fun (d, p) -> d, (move_pitch inter p))
videodemosareindicatedwiththesymbol .
sequence
(cid:15) val transpose: int -> note list -> note list
2. MusicinReactiveML
Thefunctionmove_pitchtransposesasymbolicpitchbyaninter-
ReactiveML [17] is a synchronous reactive language built as an
valinterinsemi-tones.Totransposeanentiremelody,weusethe
extensionofthefunctionalprogramminglanguageOCaml.There-
higher-orderfunctionList.mapfromthestandardOCamllibrary
fore,followingtheworkofHudak[12]wecaneasilydefinemusi-
thatappliesmove_pitchtoeverynoteinasequence.
calstructures,inafunctionalprogrammingstyle.
ThissectionpresentsthelanguageReactiveMLanditsconcur-
2.2 TimeinReactiveML
rencymodelthroughasimplemusiclibrary.
ReactiveMLisasynchronouslanguagebasedonthereactivemodel
2.1 Musicdatatypes introduced by Boussinot [4]. It relies on the notion of a global
logical time. Time is viewed as a sequence of logical instants.
Traditionally, in western music, a melody is a sequence of notes,
RegularOCamlfunctionsareconsideredtobeinstantaneous,i.e.,
where a note is a sound event characterized by a pitch and a
whencalleditreturnsitsresultintheverysameinstant.Inaddition,
duration.Notescanbedefinedwiththefollowingtypes.
wecandefineprocessesthatmaylastoverseverallogicalinstants.
type pitch_class = Itisintroducedbythekeywordprocess.
| A | B | C | D | E | F | G Tostartwith,considertheprogrammingofasimplecountdown.
| As | Bs | Cs | Ds | Es | Fs | Gs Thefollowingprogramawaitsagivenamountoftimevaluegiven
| Af | Bf | Cf | Df | Ef | Ff | Gf inseconds.
type octave = int let process countdown value =
type pitch = pitch_class * octave let deadline = Unix.gettimeofday () +. value in
type dur = float while Unix.gettimeofday () < deadline do
type note = dur * pitch pause
done
The pitch is represented by a pair (pitch_class, octave), val countdown: float -> unit process
where pitch_class denotes one of the twelve semi-tones, e.g.,
A,A#,A(cid:91),B,B#...,andoctaveisaninteger.Forinstance(A,4) First, the date of the deadline is computed using the function
denotesthefamousA440Hz.Here,asinHudak’swork,Asstands gettimeofdayoftheUnixmodule6.Then,whilethecurrentdate
forAsharpandAfforAflat. if less than the deadline, the process countdown awaits the next
A classic melody is just a sequence, or list, of notes. For in- logicalinstant(pause).
stance,Figure2presentsthethemeofatraditionalFrenchmusical Communications between processes are made through signals
round:Fre`reJacquesandthebeginningofitsrepresentationinour that are values characterized by a status defined at every logical
datastructure. instant:presentorabsent.Forinstance,thefollowingprocessim-
It is often convenient to treat pitches as integers. Functions plementastandardtimersimilartotheUnixtimer.Itawaitsafirst
int_of_pitch and pitch_of_int convert a symbolic pitch,
e.g., (A,4) to an integer and vice versa (one can, for example, 5Thetypeofthefunctionsgiveninitaliccanbelessgeneralthanthe
associatethecorrespondingMIDInote,e.g.,A4=69). oneinferredbythecompiler.
6+.,*.,/.arefloating-pointaddition,multiplicationanddivisionopera-
4http://caml.inria.fr tors.
durationvalue,andthenperiodicallyemitsthesignalalarmwith To play a note, the program sends it on the signal perf. Then,
aperiodinterval.7 it waits for the delay corresponding to the note duration before
sendingthenextnote.
let rec process timer value interval alarm = Toproducesound,notesemittedonsignalperfaresenttothe
run (countdown value); audioenvironment.
emit alarm ();
run (timer interval interval alarm) let process sender perf =
val timer: loop
float -> float -> (unit, unit) event -> await perf (notes) in
unit process List.iter (fun n -> send_to_audio n) notes
end
The process timer is the interface between physical time and
val sender: (note, note list) event -> unit process
synchronous logical time. This process is non-deterministic: the
number of logical instants between two emissions of the signal This process loops infinitely: it waits for an emission on signal
alarm is not necessarily constant. Yet, if the duration of every perf and sends all received notes to the audio environment, by
logicalinstantismuchsmallerthaninterval,theerrorbetween callingthefunctionsend_to_audio.
twoemissionofthesignalalarmwillbenegligiblewithrespectto A musical performance is the parallel composition of the two
thereactiontimeofthehumanear,i.e.,30ms[10].Theimportant previousprocesses.
pointisthatwehaveisolatedthesourceofnon-determinisminthe
(cid:15)
program. Now, it is possible to express delays with respect to a 2.3 ParallelandSequentialExecution
signalwhichissupposedtobeperiodic.
Thankstothenotionofgloballogicaltimeinheritedfromthesyn-
Inthefollowing,alarmdenotesaglobalsignal.Itwillbegen-
chronousmodel,itiseasytocombineprocesses.Twoexpressions
eratedbytheprocesstimerdescribedabove,withperiodperiod
canbeevaluatedinsequence(e1; e2)orinparallel(e1 || e2).
definedasaglobalconstant.Usingthissignal,onecanwriteapro-
Inthelattercase,theyexecutesynchronously(orlockstep).There-
cessthatdoesnothingbutwaitforadurationdur.Forsimplicity,
fore,theyremainsynchronousduringanexecution.
we assume in this section that all durations are expressed in sec-
Imaginewewanttodoubleavoiceonethirdhigher(i.e.,four
onds:
semi-tones).Ofcourse,itispossibletowriteafunctionthattakes
let process wait dur = a list of notes and returns a new list of notes containing the two
let d = int_of_float (dur /. period) in voices. But, we can also use the deterministic parallel execution
for i = 1 to d do providedbyReactiveMLtoplayinparallelthevoiceanditstrans-
await alarm position.Thisconstructwillbeveryusefultocombinethemusical
done performancewithotherprocesses(seeforinstanceSection5.2).
val wait: dur -> unit process
let process double sequence =
Thisprocesswaitsfordoccurrencesofthesignalalarmwhered run (play sequence) ||
isthedurationdurexpressedasanumberofinstants. run (play (transpose 4 sequence))
InReactiveML,signalscanalsocarryvalues.Differentvalues val double: note list -> unit process
canbeemittedduringaninstant,whichistermedmulti-emission.
In ReactiveML, processes are first class citizens. Thus, it is
Inthiscase,whenthesignalisdeclared,afunctiondefininghowto
possibletowritehigherorderprocesses.Forinstance,thefollowing
combinethemultipleemissionsmustbeprovided.Here,wedeclare
codedelaystheexecutionofaprocesspbyadurationinseconds.
asignalperfwhichaccumulatesinalistthemultiplesimultaneous
emissions. let process delayed p dur =
run (wait dur);
signal perf default [] gather (fun x y -> x::y)
run p
val perf: (note, note list) event
val delayed: unit process -> dur -> unit process
The default value is set to the empty list, [], and the gathering
Finallyprocessescanbeiteratedwiththeclassicalconstructs:
function(fun x y -> x::y)appendsthevaluextotheheadof
loop/endforaninfiniteloop,for/do/doneforafinitesequential
thelisty.Thisfunctionisusedtofoldallthevaluesemittedduring
loopandfor/dopar/doneforafiniteparallelloop.
aninstantintoalist,startingfromthedefaultvalue.
With these constructs, we are able to execute the round pre-
Now,wewriteaprocessplaywhichiterativelyemitsthenotes
sentedinFigure2. Fourdifferentvoicesplaythethemeinloop.
fromasequencesequenceonthesignalperf.
Eachvoicestartstwomeasures,i.e.,8safterthepreviousone.
(cid:15)
let rec process play sequence perf =
match sequence with let process theme =
| [] -> () loop
| ((dur, pitch) as note) :: s -> run (play jacques)
emit perf note; end
run (wait dur); val theme: unit process
run (play s)
val play: let process round =
note list -> (note, note list) event -> run theme ||
unit process run (delayed theme 8.0) ||
run (delayed theme 16.0) ||
run (delayed theme 24.0)
7Inordertoavoidtimeshiftingandfloatingpointerrors,theactualimple- val round: unit process
mentationisslightlydifferent.
e e e stantaneous;theyarenotcharacterizedbyadurationbutbythede-
1 2 3
layneededbeforetheiractivation.Moreover,delaysbetweenelec-
Voice tronicactions,likenotedurations,canbespecifiedinrelativetime.
4 Thus,electro!nicpartscanfollowthespeedoftheliveperformeras
"
4 # # $ atrainedmusicianswould.
0.5 Sequences of electronic actions are linked to an instrumental
a0
group event,thetriggeringevent.Forinstance,inFigure3,actiona0is
1.0 1.0 boundtothefirstnotewithadelayof0.0.Whenthefirstnoteis
a1 a2 a3
detected,actiona issentimmediately.
0
Actionscanberegroupedintostructurescalledgroups.Groups
Figure 3. Representation of an Antescofo score. Musical notes
aretreatedasatomicactionsandareboundtoaninstrumentalevent
correspondtothemusician’spartandtheresttoelectronicactions.
andcharacterizedbyadelay.OntheexampleofFigure3,agroup
containingasequenceofthreeactionsisboundtoevente .
2
Furthermore,groupscanbearbitrarilynested.Itallowstofaith-
Thisprocesscanalsobewrittenwithaparallelloop:
fullycapturethehierarchicalstructureofamusicalpiece.Actions
let process round = containedinanestedgroupareexecutedinparallelwiththeactions
for i = 0 to 3 dopar followingthemintheembeddinggroup,notinsequence.
run (delayed theme (float (i*8))) Ascorecanbedescribedwiththefollowingtypes:
done
val round: unit process type score_event =
or, using a recursive process taking the number of voices as an { event : instr_event_label;
argument. seq : sequence; }
let rec process round nb_voices = and sequence = delay * asco_event list
if nb_voices <= 0 then ()
else begin and asco_event =
run theme || | Action of action
run (delayed (round (nb_voices - 1)) 8.0) | Group of group
end
val round: int -> unit process and action =
| Message of audio_message
We are now able to describe and execute electronic music in
| Signal of (unit, unit list) event
ReactiveML.Butwhataboutmixedmusic?
and group =
3. TowardMixedMusic
{ group_synchro: sync;
Mixedmusicisaboutinteractionbetweenalivemusicianandelec- group_error: err;
tronic parts. The Antescofo language [9] allows a composer to group_seq: sequence }
specifyelectronicpartsandhowtheyinteractwithmusiciansdur-
(cid:15)
ingaperformance.Thislanguageistheresultofaclosecollabora- and sync = Tight | Loose
tionbetweencomposersandcomputerscientists.Figure3showsa and err = Local | Global | Causal | Partial
graphicalrepresentationofaverysimpleAntescofoscore.
The type score_event represents a sequence of electronic
3.1 RelativeTimeinMusic
actionsseqboundtoaninstrumentaleventevent.Asequenceis
Inmostclassicalwesternscores,durationsanddelaysareexpressed justalistofelectronicactionscharacterizedbyadelayrelativeto
relativetothetempoi.e.,theexecutionspeedexpressedinbeatsper thetempo(seeSection3.1).
minute(bpm).OntheexampleofFigure3,thedurationofthefirst Atomicactionscanbeeithersimplecontrolmessagesdestined
twonotesissetto1.0beat,i.e.,aquarternote.Ifthecurrenttempo totheaudioenvironment,Message,orclassicReactiveMLsignals
is60bpmitmeans1s,but0.5sifthetempois120bpm.Musicians tocontrolotherprocesses,Signal.ReactiveMLsignalsareuseful
arefreetointerpretascorewithamovingtempo.Indeed,inclassi- to couple the electronic accompaniment with other reactive pro-
calwesternmusic,tempoisoneofthemostprominentdegreesof cesses(seeSection5).
freedomforinterpretation.Itpartiallyexplainsthehugedifference Theproblemisthatduringaperformance,thecomputerissup-
betweenarealinterpretationbyalivemusicianandanautomatic posedtoactasatrainedmusician,followingtheotherperformers.
executionofthesamescorebyacomputer. Thus,thespecificationofmixedmusicscoresfacestwomajorchal-
OneofthemainfeaturesoftheAntescofosystemistempoin- lenges:
ference.Duringaperformance,thelisteningmachinedecodesboth
thepositioninthescoreandtheexecutionspeed.Thetempoisnot • Duringaliveperformanceamusiciancanmakeamistake,or
estimatedfromthelastdurationalonebutratherfromalldurations worse,thelisteningmachinemayfailtorecognizeanevent.In
detected since the beginning of the performance. In this way, the bothcases,anexpectedeventismissing.Butwhathappensto
listeningmachineaddssomeinertiatotempochangeswhichcor- actionsboundtothisevent?
respondstotherealbehaviorofmusiciansplayingtogether[5,15]. • Sometimes, a sequence of actions bound to an instrumental
eventmaylastlongerthanthedurationofthetriggeringevent.
3.2 TheAntescofolanguage
Forinstance,onourexample,actionsa2anda3shouldoccur
Inthisframework,themostbasicelectronicactions,calledatomic after event e although they are triggered by event e . In this
3 2
actions are simple control messages destined for the audio envi- case,thequestionis:howshouldanelectronicpartsynchronize
ronment. Unlike traditional musical notes, atomic actions are in- withinstrumentaleventthatoccurduringitsexecution?
3.4 SynchronizationStrategies
e1 δ0 e2 Acomposerisallowedtospecifyhowactionscontainedinagroup
δ1 willsynchronizewithinstrumentaleventsthatoccurduringtheex-
group ecutionofthegroup.Currently,thelanguageproposestwodistinct
a0 a1 strategies:TightandLoose.
δ2 IfthesynchronizationattributeissettoTight,everyactioncon-
tainedinthegroupistriggeredbythemostrecentcorrespondingin-
eX1
strumentalevent.Thenearesteventiscomputedwithrespecttothe
idealtimingofthescoreregardlessoftempochanges.Thisstrat-
group partial egy is ideal when electronic parts and the musician must remain
eX1 δ1+δ2−δ0 a1 ahsarsmynocnhizraotnioonusoafsthpeospseirbfloer,mee.gr.’,sipfaarnt.eInlecthtreoenxicamvopilceeoifsFaigsuimrep3le,
if the group has a synchronization attribute set to Tight, actions
a2anda3willbetriggeredbye eventhoughtheentiregroupis
3
group causal boundtothesecondinstrumentalevent.
a0 a1 ThestrategyLooseallowsthecomposertodefinegroupsthat,
δ1+δ2−δ0 oncetriggered,onlysynchronizewiththetempo.Duetotheinertia
eX1 of the tempo inference, an electronic action contained in such a
groupandaninstrumentaleventthatseemstobesimultaneousin
group global thescoremaybedesynchronizedduringtheperformance.Indeed,
a0 a1 aperformermayaccelerateordeceleratebetweentwoevents.This
δ2 strategy is used to preserve temporal relations between actions
eX1 contained in the group. For instance, it can be very useful when
theelectronicpartistreatedasanindependentbackgroundsound.
group local
4. AFirstExample:Thehouseoftherisingsun
Asafirstexample,letustakeaclassicalfolksong,Thehouseof
Figure 4. Illustration of the four error-handling attributes on a
therisingsun.ThescoreispresentedinFigure5.Inthisexample,
simple score (on top). e and e represent instrumental events.
1 2 wewanttodefineanelectronicaccompanimentwhichfollowsthe
Supposethate ismissedande isdetected.
1 2 mainthemeplayedbyarealmusician.
First, we need to create and initia(cid:15)lize the asco environment
thatcontainstheinstrumentalscore.Thisenvironmentallowsusto
manageinputsfromthelisteningmachineandtheoutputsdestined
Error-handling and synchronization strategies depend very
fortheaudioenvironment.
much on the musical context. The Antescofo language proposes
solutions to reflect different musical behaviors. Groups are char- let asco = create_asco "rising_sun.asco" 120.
acterized by two attributes [6], a synchronization attribute (see val asco: Ascolib.asco
Section3.4)andanerrorhandlingattribute(seeSection3.3).
Tocreatetheenvironment,weneedtheinstrumentalscoreofthefu-
tureperformance,here"rising_sun.asco"andaninitialtempo,
3.3 Errorhandling typically120bpm.
Theerror-handlingattributedefinesthebehaviorofthegroupwhen 4.1 Thebasics
theexpectedtriggeringeventisabsent.Thereareseveralwaysto
Theaccompanimentisroughlydescribedinthescorebyasequence
dealwitherrorsdependingonthemusicalcontext.Wepresenthere
ofchords:Am,C,D...boundtoinstrumentalevents.Forinstance,
fourexclusiveerrorhandlingstrategies:Partial,Causal,Local
the first chord is related to the second instrumental event, the
andGlobal.Figure4illustratesthefourdifferentbehaviors.
secondtothefourthoneandsoon.
Themostbasicaccompanimentisperhapsasimpleharmoniza-
First, we need a symbolic description of chords. Like pitch
tion of the musician’s voice. In this case, if a triggering event is
describeinSection2.1,achordischaracterizedbyapitchclass:
missed,electronicpartmustkeepongoingasiftheerrorneveroc-
A,C,D...andacolor:majororminor.
curred.Thisis,forinstance,thebehaviorofapianistaccompanying
abeginner.Thus,attributesPartialandCausalaimtopreserve type color = Maj | Min
asimpleproperty:thefutureofaperformancedoesnotdependon type chord = pitch_class * color
pasterrors.
We can define the bass line of our example as a sequence of
Now, the question is: what about actions that should already
chordscharacterizedbyadelay:
have been launched when the error was detected? If the error-
handlingattributeissettoPartial,theseactionsaresimplydis- let bass = [0.0, (A, Min); 2.0, (C, Maj); ...]
carded. On the other hand, if the error handling-attribute is set val bass: (delay * chord) list
toCausal,theseactionsareimmediatelylaunched.Thisstrategy
Then,weneedtoturnthissequenceofchordsintoasequence
could be useful if some actions are used to initialize an external
ofelectronicactions.Perhapsthemostsimpleaccompanimentisto
process,e.g.,turnonthelight.
playonlytherootsofthebassline.
Moreover,agroupcanbeusedtodefineacomplexcontrolunit,
e.g,parametersneededtosynthesizedthesoundofagong.Then, let root_of_chord chord octave =
theintegrityofthegroupmustbepreserved.Thus,whenanerroris let (pitch_class, color) = chord in
detected,Globalgroupsarelaunchedwithazerodelay,whereas (pitch_class, octave)
Localgroupsarecompletelyignored. val root_of_chord: chord -> octave -> pitch
"" $$ $$%% $$ $$%% $$
## 4444 $$ A$$m%% $$ $$C%% $$!! $$D%% $$ $$ $$F%% !! Am !! C $$ $$ &&E $$%% !!
!!
55
$$%% $$ $$%% $$ $$%% $$ ''$$ ''$$ && $$%%
$$ $$%% $$
## Am !! C D $$ $$ $$F%% !! Am !! E!! !! Am
!!
Figure5. Scoreofthehouseoftherisingsun(classicalfolksong)
let only_roots chords octave dur = detected,eachchordoftheaccompanimentislinkedtoitsclosest
List.map precedinginstrumentalevent.
(fun (delay, chord) -> Thelastthingtodoistolaunchthelisteningmachineandthe
let root = root_of_chord chord octave in reactiveaccompaniment.
(delay, action_note (dur, root)))
chords let process main =
val only_roots: run (init_asco asco);
(delay * chord) list -> octave -> dur -> sequence begin
run (listener asco) ||
Given an octave, the function root_of_chord returns the pitch
run (tight_accomp) ||
correspondingtotherootofachord.Then,thefunctiononly_roots
run (sender asco)
defines the sequence of electronic actions corresponding to the
end
roots of a sequence of chords, chords. Function action_note
val main: unit process
convertsanoteintoanatomicactiondestinedfortheaudioenvi-
ronment (a value Action message where the message contains
Itfirstinitializestheenvironment.Then,theprocesslistener
thecharacteristicsofthenote).
links the listening machine of Antescofo to the environment. In
RemarkthatinFigure5,nooctaveisspecifiedonthescorefor
parallel, it executes the accompaniment part, and sends it to the
thebass.Wechoosetoinstantiatethebasswiththeoctavesetto3
audioenvironmentsender asco.
(i.e.,thefirstrootofthebassissettoA3=220Hz)andtheduration
to2.0beatswhichcorrespondstothedelaybetweentwochords.
4.2 Hierarchicalstructure
let roots = only_roots bass 3 2.0
Theharmonizationdefinedaboveisabitminimalist.Onesolution
val roots: sequence
couldbetoaddseveralothervoicesfollowingthepreviousmethod.
The next thing to do is to build the link between the in- Butwecandobetter.
strumental part and the electronic accompaniment. The process In the original song, the accompaniment is made with arpeg-
link asco evt seqlinksasequenceofactionsseqtoaninstru- giooverthebassline.Thankstofunctionalcomposition,suchan
mentaleventevt,intheenvironmentasco.Thisprocesswaitsfor accompanimentisveryeasytodefine.
theeventtobedetectedormissedandittriggersthesequence.For First,wedefinethearpeggiostylewewanttoapply:
instance,ontheexampleofFigure5,thefirstchordisboundtothe
secondinstrumentalevent.Wecanlinkthesequencerootstothis let arpeggio chord octave =
eventtoobtainabasicaccompaniment. let fond = root_of_chord chord octave in
let third =
let process basic_accomp =
match color with
run (link asco 2 roots)
| Min -> move_pitch 3 fond
val basic_accomp: unit process
| Maj -> move_pitch 4 fond
Iftheperformerdoesnotplayatconstantspeed,theaccompa- in
niment part may become desynchronized at some point. We can let fifth = move_pitch 7 fond in
easily avoid this behavior if we put the bass line inside a Tight let dur = 0.125 in
group(seeSection3.4).Moreover,itallowsustospecifyanerror- group Loose Local
handlingstrategyfortheaccompaniment.Inourcase,ifaninstru- [0.0, action_note (dur, fond);
mentaleventismissedwedonotwanttoheartheassociatedchord. 0.625, action_note (dur, third);
ThereforeweusethePartialstrategy(seeSection3.3). 0.125, action_note (dur, fifth);
0.125, action_note (dur, move_octave 1 fond);
let process tight_accomp =
0.125, action_note (dur, move_octave 1 third);
let g = group Tight Partial roots in
0.333, action_note (dur, move_octave 1 fond);
run (link asco 2 [(0.0, g)])
0.333, action_note (dur, fifth);]
val tight_accomp: unit process
val arpeggio : chord -> octave -> asco_event
Here, the function group sync err seq is a simple constructor
foragroupwithattributessyncanderrcontainingthesequence An arpeggio corresponds to a Loose Local group. Thus, an
ofactionsseq. arpeggiorelatedtoamissingeventisentirelydismissed(Local).
Notethatthesequenceboundtothesecondinstrumentalevent Besides, the synchronization strategy preserves the rhythm of
containsonlyonezerodelayelectronicaction:thegroupthatde- the arpeggio (Loose). The function move_octave is similar to
fines the entire accompaniment. When this instrumental event is move_pitchandshiftthepitchagivennumberofoctaves.
Then,wecandefinetheaccompanimentbyapplyingthefunc- let process killable k p =
tionarpeggiotothebassline. do
run p
let arpeggio_bass chords octave =
until k done
List.map
val killable:
(fun (delay, chord) ->
(’a, ’b) event -> unit process -> unit process
(delay, arpeggio chord octave))
chords Theconstructiondo/untilisnotaloopingstructure.Itstopsthe
val arpeggio_bass: executionofitsbodywhenthesignalisemitted.
(delay * chord) list -> octave -> sequence Now,wecandefineasignalkill_druminthetoplevel:
As for the basic accompaniment, we encompass the resulting signal kill_drum default () gather (fun x y -> ())
sequenceinsideaTightPartialgroup. val kill_drum: (unit, unit) event
let process arpeggio_accomp =
Insteadofsimplyexecutingthepercussionpart,weexecuteitunder
let chords = arpeggio_bass bass 3 in
thesupervisionofthekillablecombinator.
let g = group Tight Partial chords in
run (link asco 2 [0.0, g]) #run (killable kill_drum drum_accomp)
val arpeggio_accomp: unit process
Then,wecanstopthedrumpartduringtheperformancewhenever
Thisaccompanimentsillustratesthepossibilityofdefininghierar-
wewant.
chical structures in the score. Indeed, a global Tight group con-
tainsasuccessionofarpeggiosalsodefinedasgroups. emit kill_drum ()
Finallywecanreplacetheprocesstight_accompinthemain
processbyarpeggio_accomp. A more interesting example is to dynamically replace an ac-
companiment. First, let us define a signal replace on which we
4.3 LiveCoding cansendtheprocesses.
A benefit of the embedding of Antescofo in ReactiveML is the
signal replace
ability to use its read–eval–print loop, called toplevel [16]. For
default (process ())
example,ifweexecutetheexampleoftherisingsuninthetoplevel,
gather (fun x y -> process (run x || run y))
whileitisplaying,wecandefineapercussionpart(weassumehere
val replace: (unit process, unit process) event
that the environment asco has been properly initialized and that
processeslistenerandsenderarealreadyrunning):
Note that if several emissions occur during the same instant, the
let process drums_accomp = resultwillbetheparallelcompositionofalltheemittedprocesses.
let rhythm = [0.0; 0.625; 0.5; 0.5; ... ] in Usingtheprevioussignal,wecandefineanotherhigherorder
let drums = process replaceable which executes processes emitted on the
List.map (fun d -> d, action_beat) rhythm signalreplace.
in
let rec process replaceable replace p =
let accomp = group Tight Partial drums in
do
run (link asco 2 [0.0, accomp])
run p
val drums_accomp: unit process
until replace (q) ->
First, we define a sequence of delays, rhythm, to represent run (replaceable replace q)
the rhythm of the drum part. Then, we turn this rhythm into a done
sequence of electronic actions, drums. Here, action_beat is a val replaceable:
valueAction messagesuchthatwhenevertheaudioenvironment (unit process, unit process) event ->
receives one of these messages, it plays a drum beat. Finally the unit process ->
resultisaTightPartialgroupcontainingtheentirepercussion unit process
partboundtothesecondinstrumentalevent.
First, the process replaceable launches the process p. Then,
Dynamicallyduringtheperformance,wecanaskthatthispart
beplayed:8 wheneveraprocessqisemittedonsignalreplace,theexecution
ofpstopsandtheprocessqislaunchedinturn.
#run drums_accomp For instance, we can use this feature to dynamically switch
betweenthedifferentstylesofaccompaniment:basic,arpeggioor
Thisnewvoicewillsynchronizeautomaticallywiththeothers,even
drums.
ifthetriggeringeventalreadypassed,thankstotheerror-handling
strategy. #run (replaceable replace tight_accomp)
The expressiveness of the language and this dynamic aspect emit replace arpeggio_accomp
allowsustobuildmorecomplexinteractionswithmusiciansduring emit replace drums_accomp
aperformance.
Let us start with a simple example: a higher order process The first instruction launches the basic accompaniment (see
killablethatallowsaprocessgivenasargumenttobestopped Section 4.1). After a while, we can evaluate the second or the
whenasignalkisemitted.Thisprocesscanbeusedtocontrolthe third instruction to stop the electronic part and start the com-
executionofanaccompaniment. plete accompaniment, arpeggio_accomp or the percussion part,
drums_accomp.
8The #run is a toplevel directive which executes the process given as Thus, it is possible to compose, correct and interact with the
argumentinbackground. scoreduringtheperformance.
tronicactionstoaninstrumentaleventevtintheenvironmentand
loopsntimesoverthesequenceseqwhereeachiterationissepa-
ratedbyadelayperiod.Iftheperiodisshorterthantheduration
" #" " #" " " #"
! " #" #" " #" ofthepattern,theexecutionoftheloopleadstooverlappingexe-
cutionsofthepattern.
Figure6. ThemelodicfigureofSteveReich’sPianoPhase TheelectronicpartresynchronizewhenBobplaysthek-thnote
ofthesequenceastheelectronicpartisplayingthefirstnote.Thus,
weneedtotrackthek-thnoteofBob’spart.
5. ReactiveInteractions
ThemainadvantageofembeddingtheAntescofolanguageinRe- let process track k kth_note =
activeMListhatthereactiveaccompanimentisnowabletointeract let ie = instr_event asco in
withregularReactiveMLprocesses.Indeed,atomicactionsofthe loop
Antescofo language can be ReactiveML signals. This feature can await ie (evt) when (evt.index mod 12 = k) in
beusedtoprogrammusicalpieceswheretheinteractionbetween emit kth_note ()
the performer and the accompaniment is more subtle than in the end
previousexamples. val track:
int -> (unit, unit) event -> unit process
5.1 PianoPhase
PianoPhaseisapiecewrittenin1967bytheminimalistcomposer Thefunctioninstr_eventreturnsthesignalcarryingtheoutput
Steve Reich. In this piece, two pianists, Alice and Bob, begin by of the listening machine. Whenever an instrumental event is de-
playing the melodic figure presented in Figure 6 over and over tectedtheindexoftheeventinthescoreandtheestimatedtempo
againinunison.Then,oneofthepianists,letussayAlice,begins issentonthissignal.Theconstructawait ie (evt) when ...
to play her part slightly faster than Bob. When Alice plays the bindsthevaluereceivedonietoevtwhenthesignalisemitted.
first note of the melody as Bob is playing the second note, they Theexecutioncanthencontinueonlyiftheconditionofthewhen
resynchronizeforawhile,i.e.,Aliceslowsdowntoplayatthesame issatisfied.Here,ifthedetectedpositionevt.indexcorresponds
speedasBob.Theprocessisrepeated.Aliceacceleratesagainand tothek-thnoteofthesequence,thesignalkth_noteisemitted.
thenresynchronizeswhensheplaysherfirstnoteasBobisplaying The last thing to do is to compare the emissions of the two
thethird,thenthefourth,etc. signals first_note and kth_note. If two emissions are close
Ifplayingasimplemelodyatconstantspeedisrelativelyeasy enoughitemitsasignalsync.Wecansplitthisprocessintothree
(Bob’s part), Alice’s part which alternatively desynchronizes and steps.
resynchronizes is much harder to achieve. Fortunately, we can Theprocessstamp s r ascoassociatesadatetoeachemis-
programthispartasanelectronicaccompanimentofapianistthat sionofsignalsandemitstheresultonasignalr.
alwaysplaysatconstantspeed,hereBob.
Thedifficultycomesfromthefactthatwecannoteasilycom- let process stamp s r =
puteaprioriwhenAliceandBobwillresynchronize.Eveninthe loop
originalscore,thecomposeronlyspecifiesboundsonthenumber await s;
ofiterationsofthesequenceduringthedesynchronization(between emit r (date asco)
fourandsixteentimes).Howcanweprogramthisbehavior? end
Let us start with a simple process that plays the twelve note val stamp:
(cid:15)
melodyatagivenspeedandsendsasignalfirst_noteeachtime (unit, unit) event -> (delay, delay) event ->
thefirstnoteisplayed.ThiscodecorrespondstoAlice’spart. unit process
let process melody evt n delay first_note =
The function date returns a date relative to the tempo. It corre-
let pattern =
spondstotheelapseddelaysincethebeginningoftheperformance.
group Tight Partial
Thereforetheprocessstampisresponsivetotheexecutionspeed.
[0.0, action_note (delay, (E,4));
Then,thefollowingprocessemitsasignalsynceachtimethe
0.0, action_signal first_note;
lastvaluesemittedonsignalsr1andr2areapproximatelyequal,
delay, action_note (delay, (Ff,4));
i.e.,wheneverthedifferencebetweenthetwovaluesislessthana
delay, action_note (delay, (B,4));
constanteps.
...]
in
let process spy r1 r2 sync eps =
let seq = [0.0, pattern]
loop
let period = 12.0 *. delay in
await (r1 \/ r2);
run (link_and_loop asco evt period n seq)
let t1 = last ?r1
val melody:
and t2 = last ?r2 in
instr_event_label -> int -> delay ->
if abs_float (t1 -. t2) < eps then
(unit, unit) event -> unit process
emit sync ()
First, we define a simple Tight Partial group that contains end
the melodic pattern presented in Figure 6. The second action, val spy:
introduced with the function action_signal sends the signal (float, float) event -> (float, float) event ->
first_note.Sincethedelayofthisactionissetto0.0,thesignal (unit, unit) event -> float -> unit process
willbeemittedsimultaneouslywiththefirstnoteofthesequence.
Then we compute the period of the pattern which is the dura- Theconstructawait (r1 \/ r2)awaitsforoneofthetwosig-
tionofthetwelvenotemelody,i.e.,12.0 *. delay.Theprocess nalsr1orr2tobeemitted.Theexpressionlast ?r1evaluatesto
link_and_loop evt n period seq links a sequence of elec- thelastvalueemittedonsignalr1.
Finally we can combine the previous processes to achieve the alteringitsbehavior.Forinstance,itisrelativelyeasytoprograma
desiredbehavior. graphicalinterfaceforthepreviousexample.Thissectionissimply
aproofofconcept.Wedonotclaimthatthisinterfaceisespecially
let process compare s1 s2 sync eps =
aesthetic.
signal r1 default -.eps gather (fun x y -> x) in
Let us start with a very simple process that draws a rectangle
signal r2 default -.eps gather (fun x y -> x) in
atagivenposition,Left,RightorFull,eachtimeasignalsis
run (stamp s1 r1) ||
emitted.
run (stamp s2 r2) ||
run (spy r1 r2 sync eps) let process draw_sig s pos color =
val compare: loop
(unit, unit) event -> (unit, unit) event -> await s;
(unit, unit) event -> float -> unit process draw_rect pos color;
run (wait asco 0.2);
Thisprocess,associatesadatetoeachemissionofsignalss1and Graphics.clear_graph ()
s2takenasarguments.Resultsareemittedonsignalsr1andr2, end
respectively. Then, in parallel, we launch the process spy on r1 val draw_sig:
andr2,whichemitsthesignalsyncwheneveranemissionofs1 (unit, unit) event -> position -> color ->
iscloseenoughtoanemissionofs2.Sincetheprocessstampis unit process
responsivetothetempo,thetolerancerepresentedbytheconstant
eps increases when the tempo decelerates and decreases when The process wait waits for a duration specified relative to the
it accelerates. It corresponds to the behavior of actual musicians tempooftheenvironment.Thus,theinterfaceisresponsivetothe
playingtogether. speedoftheperformance.
Signalsr1andr2areinitializedinordertoavoidinitialization Then,wecanwriteagraphicobserverthatreactstotheemis-
conflicts.Indeed,ifasignalhasneverbeenemitted,thelastvalue sionoftwosignalss1ands2byalternatingbetweentwomodes
isthedefaultvalue.Besides,multipleemissionsareinthiscontext triggeredbythesignalsm1andm2.
very unlikely since the entire sequence is played between two
let process graphic_observer s1 s2 m1 m2 =
emissionoffirst_noteorkth_note.Therefore,thecombination loop
function chooses arbitrarily one of the values emitted during the
do
sameinstant.
Graphics.clear_graph ();
Finallytheentirepieceisdescribedbythefollowingcode.
run (draw_sig s1 Full Graphics.green)
let piano_phase sync desync first_note kth_note = until m1 done;
let rec process piano_phase k = do
let ev = last_event asco in Graphics.clear_graph ();
run (melody ev 4 0.25 first_note); begin
emit desync (); run (draw_sig s1 Left Graphics.blue) ||
do run (draw_sig s2 Right Graphics.red)
let ev = last_event asco in end
run (melody (ev+1) 16 0.246 first_note) || until m2 done
run (track k kth_note) || end
run (compare first_note kth_note sync 0.05) val graphic_observer:
until sync done; (unit, unit) event -> (unit, unit) event ->
run (piano_phase ((k + 1) mod 12)) (unit, unit) event -> (unit, unit) event ->
in unit process
piano_phase 1 Inthefirstmode,wedrawagreenrectangleontheentiregraphic
val piano_phase: windoweachtimethesignals1isemitted.Whenanemissionof
(unit, unit) event -> (unit, unit) event -> signalm1occurs,weswitchtothesecondmode.Then,wedrawa
(unit, unit) event -> (unit, unit) event -> bluerectangleontheleftsideofthewindoweachtimethesignal
unit process s1isemitted,andaredrectangleontherightsideifthesignals2
isemitted.Then,werestartthewholeprocesswhenthesignalm2
First,weplaythesequencefourtimeswiththedurationofthenotes
isemitted.
setto0.25(asixteenthnote)i.e.,synchronouslywiththepianist.
We can run the process graphic_observer with signals
Duringthisphase,theloopislinkedtothelastdetectedinstrumen-
first_note, kth_note, desync and sync in parallel with the
talevent:last_event asco.Thenweemitthesignaldesyncand
processpiano_phasetoobtainasimplegraphicalvisualizationof
starttoplaythesequenceslightlyfasterwiththenextinstrumental
PianoPhase.Duringthesynchronousphase,greenflashesindicate
event (the duration of the notes is set to 0.246). In parallel, we
that Alice, i.e., the electronic part, is playing the first note of the
compareemissionsofsignalsfirst_noteandkth_note.When
sequence.Duringthedesynchronization,wecanseeredflasheson
theyarecloseenough(hereepsissetto0.05),thesignalsyncis
therighteachtimeBob,i.e.,theactualpianist,isplayingthek-th
emitted.Thenweincrementk andrestartthewholeprocess.The
note,andblueonesonthelefteachtimetheaccompanimentpart
numberofiterationsinthedesynchronizedmodeissetto16.After
isplayingthefirstnote.
that,ifthetwopianistshavenotresynchronized,theelectronicpart
stops.
6. RelatedWork
5.2 SynchronousObserver
Comparison with the actual Antescofo language We focused
In ReactiveML, signals are broadcast communication channels. here on a subset of the actual language of Antescofo. However,
Therefore, one can easily write synchronous observers [11], i.e., constructs that we left aside can be easily encoded using our li-
processesthatlistentotheinputsandoutputsofaprocesswithout brary.Forinstance,anoriginalcontrolstructurewasaddedtothe