Table Of ContentIntroduction
byToddM.Fay
DirectMusicissoftwarethatallowscomposers,sounddesigners,andaudioengineersto
createdigitalmusicalcontentwithinteractive,variable,andadaptiveplaybackproperties.
Whatdoesthatmean?Moreover,whocares?ChancesarethatJoeMusicianisnotterribly
familiarwiththeterms"interactive,""variable,"and"adaptive"astheyrelatetoaudio
production—andforgoodreason;musicalpiecesfeaturingthesepropertieshaveyetto
enterthemusicalmainstream.Well,weareheretochangeallofthat…
DirectMusic in Interactive Media Development (aka
Computer/Video Games)
Whilenotfamiliartermsinmainstreammusicproductionormotionpicturepost,interactive,
variable,andadaptiveaudioplaybackaremainstaytermsintheworldofgameaudio
production.Games,likeothervisualmedia,useaudioasameansofcommunicating
informationandmanipulatingaudienceemotion.However,therearesomepitfallsinherentto
themediumthataudioproducersneedtodeal with,namelynonlinearity,variable
environments,repetition,storagespace,andprocessingpowerlimitations.Thesepotential
roadblocksrequireuniquecreativeandtechnicalsolutions.DirectMusicexists,inpart,to
assistaudioproducersindealingwiththeseuniqueproblems.
DirectMusicispartofDirectXAudio,theaural componentsofasuiteofsoftwaretools(or
"library")collectivelyreferredtoasDirectX.MicrosoftcreatedDirectXtogiveprogrammers
powerful toolstocreategamesforWindowsPCs.Andtheysucceeded;DirectXmade
WindowsthepremierPCplatformforgames.DirectXAudioconsistsoftwocomponentparts
namedDirectMusicandDirectSound.Atfirst,DirectXhadonlytheDirectSoundlibrary.
ProgrammersusedDirectSoundforbasicplaybackofagame'saudioassets.DirectMusic
wasintroducedlaterontogivecomposersand sounddesignersmorecontrolovertheir
contributiontothegame.Today,thetwoarereferredtocollectivelyasDirectXAudio.We
focusmainlyonDirectMusicinthistextbecauseitiseasiertoworkwithfromtheperspective
ofacomposerandsounddesignerandisineffectmorepowerful.Justunderstandthat
DirectSoundcameintoexistencebeforeDirectMusic,andDirectMusicwasbuilt,inpart,on
topof DirectSound.Thisisalreadymorethanyouneedtoknow,solet'savoidenteringinto
anymoredetailshere,asthereisabookonthesubject(theoneyou'reholding).
Greater Control for Audio Producers
Intheearlydaysof gamedevelopment,audioproducerswerelargelydependentuponaudio
programmers(softwareengineers)togettheirsoundsandmusicintotheirgame,aprocess
knownasintegration.DirectMusicrelievesmuchofthisdependency.UsingDirectMusic
Producer,anaudioproducercanplaceall ofhisorherwaves,sequencedata,samplesets,
andeveninstructionsonhowaudioshouldbeplayedinteractively,adaptively,andwith
variation.Thesumoftheaudioproducer'sworkissavedinafilecalledaSegment(.sgt
files).ASegmentistoanadaptive,interactive,andvariablescoreasawaveand/orMIDIfile
istoalinearscore.Becauseofthecompletenessoftheinformationanddatastoredina
DirectMusicSegment,theroleoftheprogrammerinintegratingisreducedsignificantly.
Gonearethedaysof aproject'saudioprogrammerscrewing withvolumelevelsortriggering
soundstothewrongevents.Theprogrammerdoesn'tneedtoknowhowtheaudiogets
playedback;heonlytellsthegametoplaythepieceofmusic,andwhateveraudiodataand
parameterswerespecifiedbythecomposerareusedatplayback.
Thisisusefulforaudioproducerscreatingstand-alonemusicwithDirectMusic.Theycan
saveall oftheirsamples,waves,sequencedata,andinstructionsforadaptation,variation,
andinteractivity(ifany)intheSegment.ThelistenerthenloadsthefileintotheirDirectMusic
playerprogram,andvoílà—beautifulmusic!
Therewill,of course,alwaysbe someamountofinformationthatprogrammersand
composersmustcommunicatetoeachother.Programmersmustletthecomposerknow
howmuchmemoryhecanuseforsample,wave,andsequencedataorwhetherhecan
stream wavedataoffoftheharddiskorCD/DVD.Theprogrammergenerallyneedstoknow
howtotransitionfrom onepieceofmusictothenext(measure,beat,markerboundaries,
andsoon).Theprogrammertypicallyimplementsthingsliketempochangesbasedongame
state.However,acomposer(orprogrammer)caninsteadtakeadvantageoftheDirectX
AudioScriptinglanguageforgreatercontrolovertheseaspectsofplaybackfromthe
programmer.Ascriptinglanguageprovidestheaudioproducerwiththeabilitytocreate
simpleeasy-to-read"programs"(knownasroutines)thataprogrammercanthenrunbased
onprogrammaticevents.Thelanguageisdesignedtobesimpletolearnbutstill havealot
offlexibilityandpowerinwhatitcandoforaudioproducers.Wediscussthismorelater.
DirectMusic for Music Production
Theapplicationsforinteractive,variable,andadaptiveaudio,particularlymusic,extend
beyondcomputerandvideogames.ThepotentialforDirectMusictoalterthewaywelisten
todigitalmusicinourhomes,cars,andanywhereelsewhere wecansmuggleapersonal
musical playerisveryexciting.Musicwrittenwithinteractivity,adaptability,anddynamism
allowsaudiencestotakeamoreactiveroleintheirlisteningexperiences.Arecordingartist's
nextsinglecouldbeaDirectMusicmodulethatlistenerscanintuitivelyremixwith,introduce
newmusical partsandeffectsto,orsettoreacttochangesintheircomputingenvironment.
Itcouldevenbemadetoneverplaybackthesamewaytwice,iflistenerssochose!
Admittedly,noteveryoneisinterestedinhavingsuchanactiveroleinmusiclistening.
Passivelistenerscanstill enjoyenhancedmusicalexperiencesbysettingtheirDirectMusic
songstoplaybackaccordingtopackagedtemplates."Factorysettings"couldprovide
alternateinstrumentation,allowingajazzpiecetotakeonmoreof aworld,rock,orhip-hop
feelforinstance,evenswitchingbackandforthbetweenstylesandinstrumentsets.Other
templatescouldrangefromintelligent"concerthall"or"arena"reverberationsettingsusing
built-ineffectsprocessingtocompletelyself-generatedperformancesbasedonalooseset
ofcriteriacreatedfromscratchwitheveryplay.
Thereareplentyoftoolsonthemarketthatallowjustaboutanyonewithafewhundred
buckstobegincreatingmusiconhiscomputer.Thedifferencesbetweenyouraverageaudio
editingormultitrackingprogram andDirectMusicaretheinteractive,adaptive,andvariable
abilitiesthatyouwill notfindanywhereelse(atleastnottothisdegree).Remember,these
kindsofsoundtracks,songs,andsoundscapesareforthemostpartunexploredterritory.We
assureyou,theterraininthisterritoryisunlikeanyyou'veexperiencedbefore,eitherasa
composer,aperformer,anengineer,oralistener.
Atthetimeofthisbook'spublication,Microsoftwasn'tpromotingDirectMusicasa
mainstreammusicproductiontool.DirectMusicwasvirtuallyunknowntothemillionsof
music-producinghobbyistsandprofessionalslookingfornewwaystodistinguishthemselves
amongtheirpeersandaudiences.Onlyafewmembersof asmallcommunitywereusing
DirectMusicforaudioproductionoutsideofgames.Oursolutiontobringinginteractive,
adaptive,andvariablemusictothemasses?Educateaudioproducersonhowtouse
DirectMusictocreatenewsonicandmusical experiences.ThemoreDirectMusicsongs
created,themorethatpeoplecanbegintoexperimentandappreciatethesenewmusical
experiences.
DirectMusicisthechosenplatformforthisneo-musicalproductioncrusadeforthree
reasons.First,itsdepth.Thereisnot,toourknowledge,anyotherinteractive/adaptiveaudio
solutionascomprehensiveasDirectMusic.Secondly,itsreach.AnyonewithaWindows
machineandDirectX6.1orhighercanenjoymusiccreatedforDirectMusic.Oh,andnumber
three:It'sfree!Wewishwecouldadd"easeandconvenienceof use"toourlistofreasonsto
loveDirectMusic,butifthatwerethecase,youwouldnothavemuchuseforthisfinetext!
DirectMusicProducer(theprogram MicrosoftgivesyoutocreateDirectMusiccontent)isnot
user-friendlyproductionsoftware.InallfairnesstothedesignersatMicrosoft,DirectMusic
Producer(DMP)isn'tcommercialsoftware,anditwasnevermeanttobe.It'sabit
convolutedasaninterface,anditcertainlyisn'tprettytolookat.Addaslewofalien
conceptsinaudioproduction,andyouhaveasignificantbarriertoentry.Upuntilnow,there
wasn'tevenabookonhowtogetaroundinDMP,letaloneDirectMusic.Thegoodnewsis
thatsinceDMPissimplyaninterfaceforproducerstowork withDirectMusic,someone
somewherecouldbuildamoreuser-friendlyprogramtoreplaceDMP,withoutsacrificingany
ofDirectMusic'spower.Ofcourse,it'spurespeculationastoifthatmayeverhappen…In
themeantime,DMPgetsthejobdone.Heck,weoftenrelyonanothersequencer/multitrack
andeditingsoftwarepackageforthebulkof ourproductionwork—onlyusingDMPwhen
we wanttoapplysomeof DirectMusic'suniquefunctionalitytoourproductions(dynamism,
interactivity,and/oradaptability).
Withoutadoubt,thesinglebiggestdrawbacktoproducingmusicusingDirectMusicisthe
lackofacustomizable,standardized,andwidelyavailableplayerprogram.See,DirectMusic
filesarespecialinthattheycannotjustbeopenedinWindowsMediaPlayer,Real Player,
Winamp,orwhateverothermusicplayerpeopleusedayinanddayout.Theyhavetobe
playedbackusingaprogram speciallywrittentoplayDirectMusicfiles.Truthbetold,WMP,
Winamp,RealPlayer,andtheirpeerscouldsupportDirectMusicfiles,buttheyjustdon't.
Maybesomedaytheywill,butfornowDirectMusicplaybackoptionsarelimitedtolaunching
DirectMusicProducerorwritingyourownplayer.Luckilyforall ofus,Mr.TodorFay(notto
beconfusedwithToddM.Fay!)decidedtocreateJones.JonesisaDirectMusicplayer.You
canloadaDirectMusicfileintoitandplayitback.Jonesisfree,andyoucandistributeitas
youseefit! Sogetitouttoyouraudiencesotheycanlistentoyournewmusical creations!
ThereismoreonusingJoneslater…
Excited?Good! ReadthisbookanddigintowhatDirectMusichastoofferyou!Thewisdom
inthesepagesandthesoftwareofferyouagrandopportunityforyoutodistinguishyourart
fromthatofyourstrictlylinearaudio-producingpeers.
DirectX Audio for Software Engineers
Thisisshortandsweet.DirectX9AudioExposed:InteractiveAudioDevelopmentcontainsa
comprehensivetutorial-stylelookattheinnerworkingsoftheDirectXAudiolibraries.
Everythingyou'dliketoknowabouttheDirectMusicAPIiswithinthesepages.TodorFay,
theoriginatorofthecornerstoneDirectMusictechnology,haslaidoutmuchofhiswisdomfor
allof ustolearnfrom.Sotakealookattheexamplesanddiscussionsandgetsomeideas
abouthowyoucanbuildyourownsoftwareapplicationsupontheDirectXAudiobackbone.
Welookforwardtoseeing/hearing/usingwhatyoucomeupwith!
Unit I: Producing Audio with DirectMusic
Chapter List
Chapter1:DirectMusicConcepts
Chapter2:LinearPlayback
Chapter3:Variation
Chapter4:InteractiveandAdaptiveAudio
Chapter5:DirectMusicProducer
Chapter6:WorkingwithChordTracksandChordMaps
Chapter7:DirectXAudioScripting
Chapter 1: DirectMusic Concepts
Overview
ScottSelfon
DirectMusicandDirectMusicProducer(thetoolusedtoauthormusicandsoundscapesfor
DirectMusic)introducesomenewconceptsinaudioproduction.Learntoappreciatewhat
follows,andyouwill seethatthereisanewmusicalworldwaitingtobeconquered.Butfirst,
aquickexplanationofterms….
Weusetheterm"audioproducer"throughoutthisbook.Anaudioproducerisanyonewho
createsaudiotobeusedaspartofaDirectMusicproject.Thiscanbeasounddesigner,
producer,recordingengineer,composer;wedon'tcare.Ifyourfocusiscreatingaudiotobe
placedinaproject,youareanaudioproducer.Ifyourjobisfocusedonintegratingsounds
intoaprojectusingcode,buildingtools/extensionstoDirectXAudio,ordevelopingplayback
software,oryouareagameprogrammer,thenyoushouldperkupwhenyouseetheterm
"programmer."
Interactivity
"Interactivity"isboththemostoverusedandmisusedtermingameaudio.Often,when
someonespeaksof interactiveaudio,heisreferringeithertoadaptiveaudio(audiothat
changescharacteristics,suchaspitch,tempo,instrumentation,etc.,inreactiontochanges
ingameconditions)ortotheartandscienceof gameaudioingeneral.Abetterdefinitionfor
interactiveaudioisanysoundeventresultingfrom actiontakenbytheaudience(oraplayer
inthecaseofagameorasurferinthecaseof awebpage,etc.).Whenagameplayer
pressesthe"fire"button,hisweaponsoundsoff.Thesoundofthegunfiringisinteractive.If
aplayerringsadoorbellinagameworld,theringingofthebellisalsoaninteractivesound.
If someonerollsthemouseoveranobjectonawebsiteandtriggersasound,thatsoundis
interactive.Basically,anysoundevent,whetheritbeaone-off,amusicalevent,orevenan
ambientpadchange,ifitcomesintoplaybecauseof somethingtheaudiencedoes(directly
orindirectly),itisclassifiedasinteractiveaudio.
Variability
Aninterestingsideeffectoftheperformanceofrecordedmusicproductionistheabsenceof
variationinplayback.SongsonaCDplaybackthesameeverysingletime.Musicwrittento
takeadvantageof DirectMusic'svariabilitypropertiescanbedifferenteverytimeitplays.
Variationisparticularlyusefulinproducingmusicforgames,sincethereisoftenalittlebitof
musicthatneedstostretchovermanyhoursof gameplay.Chapter21discusses
applicationsofvariationinmusicproductionoutsidetherealmof games.
Theimperfectionoftheliving,breathingmusiciancreatesthehumanelementoflivemusical
performances.Humansareincapableofreproducingamusical performancewith100
percentfidelity.Therefore,everytimeyouhearabandplayyourfavoritesong,nomatter
howmuchtheypractice,itdiffersfromthelasttimetheyplayeditlive,howeversubtlethe
differences.Repetitionisperceivedasbeingsomethingunnatural (notnecessarily
undesirable,butunnaturalnonetheless)andiseasilydetectablebythehumanear.
Variabilityalsoplaysaroleinsongform andimprovisation.Again,whenasongiscommitted
toarecording,ithasthesameformandsoloseverytimesomeoneplaysthatrecording.
However,whenperformedbymusiciansatalivevenue,theymaychoosetoaltertheform
orimproviseinawaythatisdifferentfromthatusedintherecording.
DirectMusicallowsacomposertoinjectvariabilityintoapreparedpieceof digital audio,
whetheraviolinconcertooranaudiodesignmodeledtomimicthesoundsof aSouth
Americanrainforest.Composersandsounddesignerscanintroducevariabilityondifferent
levels,fromtheinstrumentlevel(alteringcharacteristicssuchasvelocity,timbre,pitch,etc.)
tothesonglevel (manipulatingoverallinstrumentationchoices,musicalstylechoices,song
form,etc.).UsingDirectMusic'spowerofvariability,composerscancreatestand-alone
piecesofmusicthatreinventthemselveseverytimethelistenerplaysthem,creatingavery
differentlisteningexperiencewhenjuxtaposedtoamixed/masteredversionofthesame
music.Composersalterthereplayvalueoftheircompositionsaswell byallowingtheirmusic
toreinventitselfuponeverylisteningsession.
Avoidanceofaudiocontentrepetitioningamesisoftenimportant.Whenaskedaboutmusic
forgames,someoneoncesaid,"Atnotimeinhistoryhavesofewnotesbeenheardso
manytimes."Repetitionisarguablythesinglebiggestdeterrenttotheenjoymentofaudio
(bothsoundeffectsandmusic)ingames.Unliketraditional linearmedialikefilm,thereare
typicallynosettimesordurationsforspecificgameevents.A"scene"mighttakefive
minutesinoneinstanceandhoursinanother.Furthermore,thereisnoguaranteethat
particulareventswill occurinaspecificorder,willnotrepeat,orwillnotbeskippedentirely.
Coupledwiththemodeststoragespace(alsoknownasthe"footprint")budgetedforaudio
onthemediaandinmemory,thisleavestheaudioproducerinabitofaquandary.Forthe
issueof underscore,agametitlewithhoursofgameplaymightonlybebudgetedforafew
minutesworthoflinearmusic.Theaudiodirectormustdevelopcreativewaystokeepthis
musicfresh—alternativeversions,versionbranching,andsoon.Audioprogrammerscan
investigateandimplementthesesolutionsusingDirectMusic.
Astoeventstriggeringambience,dialog,andspecificsoundeffects,thesemayrepeat
numeroustimes,addingthechallengeof avoidingthekindofobviousrepetitionthatcan
spoil agame'srealismfortheplayer.Aswediscussinmoredetail,DirectMusicprovides
numerousmethodsforhelpingtoavoidrepetition.Onthemostbasiclevel,audio
programmerscanspecifyvariationsinpitchandmultipleversionsof wave,note,and
controllerdata.Evengamecontent(specificscriptedeventsinthegameforinstance)can
specifyorderingsforplayback(suchasshuffling,norepeats,andsoon)thatDirectMusic
tracksasthegameprogresses.Usingadvancedfeatures,chordprogressionscanmaintain
numerouspotential progressionpaths,allowingalimitedamountof sourcematerial to
remainfreshevenlonger.
