Table Of Content

Introduction byToddM.Fay DirectMusicissoftwarethatallowscomposers,sounddesigners,andaudioengineersto createdigitalmusicalcontentwithinteractive,variable,andadaptiveplaybackproperties. Whatdoesthatmean?Moreover,whocares?ChancesarethatJoeMusicianisnotterribly familiarwiththeterms"interactive,""variable,"and"adaptive"astheyrelatetoaudio production—andforgoodreason;musicalpiecesfeaturingthesepropertieshaveyetto enterthemusicalmainstream.Well,weareheretochangeallofthat… DirectMusic in Interactive Media Development (aka Computer/Video Games) Whilenotfamiliartermsinmainstreammusicproductionormotionpicturepost,interactive, variable,andadaptiveaudioplaybackaremainstaytermsintheworldofgameaudio production.Games,likeothervisualmedia,useaudioasameansofcommunicating informationandmanipulatingaudienceemotion.However,therearesomepitfallsinherentto themediumthataudioproducersneedtodeal with,namelynonlinearity,variable environments,repetition,storagespace,andprocessingpowerlimitations.Thesepotential roadblocksrequireuniquecreativeandtechnicalsolutions.DirectMusicexists,inpart,to assistaudioproducersindealingwiththeseuniqueproblems. DirectMusicispartofDirectXAudio,theaural componentsofasuiteofsoftwaretools(or "library")collectivelyreferredtoasDirectX.MicrosoftcreatedDirectXtogiveprogrammers powerful toolstocreategamesforWindowsPCs.Andtheysucceeded;DirectXmade WindowsthepremierPCplatformforgames.DirectXAudioconsistsoftwocomponentparts namedDirectMusicandDirectSound.Atfirst,DirectXhadonlytheDirectSoundlibrary. ProgrammersusedDirectSoundforbasicplaybackofagame'saudioassets.DirectMusic wasintroducedlaterontogivecomposersand sounddesignersmorecontrolovertheir contributiontothegame.Today,thetwoarereferredtocollectivelyasDirectXAudio.We focusmainlyonDirectMusicinthistextbecauseitiseasiertoworkwithfromtheperspective ofacomposerandsounddesignerandisineffectmorepowerful.Justunderstandthat DirectSoundcameintoexistencebeforeDirectMusic,andDirectMusicwasbuilt,inpart,on topof DirectSound.Thisisalreadymorethanyouneedtoknow,solet'savoidenteringinto anymoredetailshere,asthereisabookonthesubject(theoneyou'reholding). Greater Control for Audio Producers Intheearlydaysof gamedevelopment,audioproducerswerelargelydependentuponaudio programmers(softwareengineers)togettheirsoundsandmusicintotheirgame,aprocess knownasintegration.DirectMusicrelievesmuchofthisdependency.UsingDirectMusic Producer,anaudioproducercanplaceall ofhisorherwaves,sequencedata,samplesets, andeveninstructionsonhowaudioshouldbeplayedinteractively,adaptively,andwith variation.Thesumoftheaudioproducer'sworkissavedinafilecalledaSegment(.sgt files).ASegmentistoanadaptive,interactive,andvariablescoreasawaveand/orMIDIfile istoalinearscore.Becauseofthecompletenessoftheinformationanddatastoredina DirectMusicSegment,theroleoftheprogrammerinintegratingisreducedsignificantly. Gonearethedaysof aproject'saudioprogrammerscrewing withvolumelevelsortriggering soundstothewrongevents.Theprogrammerdoesn'tneedtoknowhowtheaudiogets playedback;heonlytellsthegametoplaythepieceofmusic,andwhateveraudiodataand parameterswerespecifiedbythecomposerareusedatplayback. Thisisusefulforaudioproducerscreatingstand-alonemusicwithDirectMusic.Theycan saveall oftheirsamples,waves,sequencedata,andinstructionsforadaptation,variation, andinteractivity(ifany)intheSegment.ThelistenerthenloadsthefileintotheirDirectMusic playerprogram,andvoílà—beautifulmusic! Therewill,of course,alwaysbe someamountofinformationthatprogrammersand composersmustcommunicatetoeachother.Programmersmustletthecomposerknow howmuchmemoryhecanuseforsample,wave,andsequencedataorwhetherhecan stream wavedataoffoftheharddiskorCD/DVD.Theprogrammergenerallyneedstoknow howtotransitionfrom onepieceofmusictothenext(measure,beat,markerboundaries, andsoon).Theprogrammertypicallyimplementsthingsliketempochangesbasedongame state.However,acomposer(orprogrammer)caninsteadtakeadvantageoftheDirectX AudioScriptinglanguageforgreatercontrolovertheseaspectsofplaybackfromthe programmer.Ascriptinglanguageprovidestheaudioproducerwiththeabilitytocreate simpleeasy-to-read"programs"(knownasroutines)thataprogrammercanthenrunbased onprogrammaticevents.Thelanguageisdesignedtobesimpletolearnbutstill havealot offlexibilityandpowerinwhatitcandoforaudioproducers.Wediscussthismorelater. DirectMusic for Music Production Theapplicationsforinteractive,variable,andadaptiveaudio,particularlymusic,extend beyondcomputerandvideogames.ThepotentialforDirectMusictoalterthewaywelisten todigitalmusicinourhomes,cars,andanywhereelsewhere wecansmuggleapersonal musical playerisveryexciting.Musicwrittenwithinteractivity,adaptability,anddynamism allowsaudiencestotakeamoreactiveroleintheirlisteningexperiences.Arecordingartist's nextsinglecouldbeaDirectMusicmodulethatlistenerscanintuitivelyremixwith,introduce newmusical partsandeffectsto,orsettoreacttochangesintheircomputingenvironment. Itcouldevenbemadetoneverplaybackthesamewaytwice,iflistenerssochose! Admittedly,noteveryoneisinterestedinhavingsuchanactiveroleinmusiclistening. Passivelistenerscanstill enjoyenhancedmusicalexperiencesbysettingtheirDirectMusic songstoplaybackaccordingtopackagedtemplates."Factorysettings"couldprovide alternateinstrumentation,allowingajazzpiecetotakeonmoreof aworld,rock,orhip-hop feelforinstance,evenswitchingbackandforthbetweenstylesandinstrumentsets.Other templatescouldrangefromintelligent"concerthall"or"arena"reverberationsettingsusing built-ineffectsprocessingtocompletelyself-generatedperformancesbasedonalooseset ofcriteriacreatedfromscratchwitheveryplay. Thereareplentyoftoolsonthemarketthatallowjustaboutanyonewithafewhundred buckstobegincreatingmusiconhiscomputer.Thedifferencesbetweenyouraverageaudio editingormultitrackingprogram andDirectMusicaretheinteractive,adaptive,andvariable abilitiesthatyouwill notfindanywhereelse(atleastnottothisdegree).Remember,these kindsofsoundtracks,songs,andsoundscapesareforthemostpartunexploredterritory.We assureyou,theterraininthisterritoryisunlikeanyyou'veexperiencedbefore,eitherasa composer,aperformer,anengineer,oralistener. Atthetimeofthisbook'spublication,Microsoftwasn'tpromotingDirectMusicasa mainstreammusicproductiontool.DirectMusicwasvirtuallyunknowntothemillionsof music-producinghobbyistsandprofessionalslookingfornewwaystodistinguishthemselves amongtheirpeersandaudiences.Onlyafewmembersof asmallcommunitywereusing DirectMusicforaudioproductionoutsideofgames.Oursolutiontobringinginteractive, adaptive,andvariablemusictothemasses?Educateaudioproducersonhowtouse DirectMusictocreatenewsonicandmusical experiences.ThemoreDirectMusicsongs created,themorethatpeoplecanbegintoexperimentandappreciatethesenewmusical experiences. DirectMusicisthechosenplatformforthisneo-musicalproductioncrusadeforthree reasons.First,itsdepth.Thereisnot,toourknowledge,anyotherinteractive/adaptiveaudio solutionascomprehensiveasDirectMusic.Secondly,itsreach.AnyonewithaWindows machineandDirectX6.1orhighercanenjoymusiccreatedforDirectMusic.Oh,andnumber three:It'sfree!Wewishwecouldadd"easeandconvenienceof use"toourlistofreasonsto loveDirectMusic,butifthatwerethecase,youwouldnothavemuchuseforthisfinetext! DirectMusicProducer(theprogram MicrosoftgivesyoutocreateDirectMusiccontent)isnot user-friendlyproductionsoftware.InallfairnesstothedesignersatMicrosoft,DirectMusic Producer(DMP)isn'tcommercialsoftware,anditwasnevermeanttobe.It'sabit convolutedasaninterface,anditcertainlyisn'tprettytolookat.Addaslewofalien conceptsinaudioproduction,andyouhaveasignificantbarriertoentry.Upuntilnow,there wasn'tevenabookonhowtogetaroundinDMP,letaloneDirectMusic.Thegoodnewsis thatsinceDMPissimplyaninterfaceforproducerstowork withDirectMusic,someone somewherecouldbuildamoreuser-friendlyprogramtoreplaceDMP,withoutsacrificingany ofDirectMusic'spower.Ofcourse,it'spurespeculationastoifthatmayeverhappen…In themeantime,DMPgetsthejobdone.Heck,weoftenrelyonanothersequencer/multitrack andeditingsoftwarepackageforthebulkof ourproductionwork—onlyusingDMPwhen we wanttoapplysomeof DirectMusic'suniquefunctionalitytoourproductions(dynamism, interactivity,and/oradaptability). Withoutadoubt,thesinglebiggestdrawbacktoproducingmusicusingDirectMusicisthe lackofacustomizable,standardized,andwidelyavailableplayerprogram.See,DirectMusic filesarespecialinthattheycannotjustbeopenedinWindowsMediaPlayer,Real Player, Winamp,orwhateverothermusicplayerpeopleusedayinanddayout.Theyhavetobe playedbackusingaprogram speciallywrittentoplayDirectMusicfiles.Truthbetold,WMP, Winamp,RealPlayer,andtheirpeerscouldsupportDirectMusicfiles,buttheyjustdon't. Maybesomedaytheywill,butfornowDirectMusicplaybackoptionsarelimitedtolaunching DirectMusicProducerorwritingyourownplayer.Luckilyforall ofus,Mr.TodorFay(notto beconfusedwithToddM.Fay!)decidedtocreateJones.JonesisaDirectMusicplayer.You canloadaDirectMusicfileintoitandplayitback.Jonesisfree,andyoucandistributeitas youseefit! Sogetitouttoyouraudiencesotheycanlistentoyournewmusical creations! ThereismoreonusingJoneslater… Excited?Good! ReadthisbookanddigintowhatDirectMusichastoofferyou!Thewisdom inthesepagesandthesoftwareofferyouagrandopportunityforyoutodistinguishyourart fromthatofyourstrictlylinearaudio-producingpeers. DirectX Audio for Software Engineers Thisisshortandsweet.DirectX9AudioExposed:InteractiveAudioDevelopmentcontainsa comprehensivetutorial-stylelookattheinnerworkingsoftheDirectXAudiolibraries. Everythingyou'dliketoknowabouttheDirectMusicAPIiswithinthesepages.TodorFay, theoriginatorofthecornerstoneDirectMusictechnology,haslaidoutmuchofhiswisdomfor allof ustolearnfrom.Sotakealookattheexamplesanddiscussionsandgetsomeideas abouthowyoucanbuildyourownsoftwareapplicationsupontheDirectXAudiobackbone. Welookforwardtoseeing/hearing/usingwhatyoucomeupwith! Unit I: Producing Audio with DirectMusic Chapter List Chapter1:DirectMusicConcepts Chapter2:LinearPlayback Chapter3:Variation Chapter4:InteractiveandAdaptiveAudio Chapter5:DirectMusicProducer Chapter6:WorkingwithChordTracksandChordMaps Chapter7:DirectXAudioScripting Chapter 1: DirectMusic Concepts Overview ScottSelfon DirectMusicandDirectMusicProducer(thetoolusedtoauthormusicandsoundscapesfor DirectMusic)introducesomenewconceptsinaudioproduction.Learntoappreciatewhat follows,andyouwill seethatthereisanewmusicalworldwaitingtobeconquered.Butfirst, aquickexplanationofterms…. Weusetheterm"audioproducer"throughoutthisbook.Anaudioproducerisanyonewho createsaudiotobeusedaspartofaDirectMusicproject.Thiscanbeasounddesigner, producer,recordingengineer,composer;wedon'tcare.Ifyourfocusiscreatingaudiotobe placedinaproject,youareanaudioproducer.Ifyourjobisfocusedonintegratingsounds intoaprojectusingcode,buildingtools/extensionstoDirectXAudio,ordevelopingplayback software,oryouareagameprogrammer,thenyoushouldperkupwhenyouseetheterm "programmer." Interactivity "Interactivity"isboththemostoverusedandmisusedtermingameaudio.Often,when someonespeaksof interactiveaudio,heisreferringeithertoadaptiveaudio(audiothat changescharacteristics,suchaspitch,tempo,instrumentation,etc.,inreactiontochanges ingameconditions)ortotheartandscienceof gameaudioingeneral.Abetterdefinitionfor interactiveaudioisanysoundeventresultingfrom actiontakenbytheaudience(oraplayer inthecaseofagameorasurferinthecaseof awebpage,etc.).Whenagameplayer pressesthe"fire"button,hisweaponsoundsoff.Thesoundofthegunfiringisinteractive.If aplayerringsadoorbellinagameworld,theringingofthebellisalsoaninteractivesound. If someonerollsthemouseoveranobjectonawebsiteandtriggersasound,thatsoundis interactive.Basically,anysoundevent,whetheritbeaone-off,amusicalevent,orevenan ambientpadchange,ifitcomesintoplaybecauseof somethingtheaudiencedoes(directly orindirectly),itisclassifiedasinteractiveaudio. Variability Aninterestingsideeffectoftheperformanceofrecordedmusicproductionistheabsenceof variationinplayback.SongsonaCDplaybackthesameeverysingletime.Musicwrittento takeadvantageof DirectMusic'svariabilitypropertiescanbedifferenteverytimeitplays. Variationisparticularlyusefulinproducingmusicforgames,sincethereisoftenalittlebitof musicthatneedstostretchovermanyhoursof gameplay.Chapter21discusses applicationsofvariationinmusicproductionoutsidetherealmof games. Theimperfectionoftheliving,breathingmusiciancreatesthehumanelementoflivemusical performances.Humansareincapableofreproducingamusical performancewith100 percentfidelity.Therefore,everytimeyouhearabandplayyourfavoritesong,nomatter howmuchtheypractice,itdiffersfromthelasttimetheyplayeditlive,howeversubtlethe differences.Repetitionisperceivedasbeingsomethingunnatural (notnecessarily undesirable,butunnaturalnonetheless)andiseasilydetectablebythehumanear. Variabilityalsoplaysaroleinsongform andimprovisation.Again,whenasongiscommitted toarecording,ithasthesameformandsoloseverytimesomeoneplaysthatrecording. However,whenperformedbymusiciansatalivevenue,theymaychoosetoaltertheform orimproviseinawaythatisdifferentfromthatusedintherecording. DirectMusicallowsacomposertoinjectvariabilityintoapreparedpieceof digital audio, whetheraviolinconcertooranaudiodesignmodeledtomimicthesoundsof aSouth Americanrainforest.Composersandsounddesignerscanintroducevariabilityondifferent levels,fromtheinstrumentlevel(alteringcharacteristicssuchasvelocity,timbre,pitch,etc.) tothesonglevel (manipulatingoverallinstrumentationchoices,musicalstylechoices,song form,etc.).UsingDirectMusic'spowerofvariability,composerscancreatestand-alone piecesofmusicthatreinventthemselveseverytimethelistenerplaysthem,creatingavery differentlisteningexperiencewhenjuxtaposedtoamixed/masteredversionofthesame music.Composersalterthereplayvalueoftheircompositionsaswell byallowingtheirmusic toreinventitselfuponeverylisteningsession. Avoidanceofaudiocontentrepetitioningamesisoftenimportant.Whenaskedaboutmusic forgames,someoneoncesaid,"Atnotimeinhistoryhavesofewnotesbeenheardso manytimes."Repetitionisarguablythesinglebiggestdeterrenttotheenjoymentofaudio (bothsoundeffectsandmusic)ingames.Unliketraditional linearmedialikefilm,thereare typicallynosettimesordurationsforspecificgameevents.A"scene"mighttakefive minutesinoneinstanceandhoursinanother.Furthermore,thereisnoguaranteethat particulareventswill occurinaspecificorder,willnotrepeat,orwillnotbeskippedentirely. Coupledwiththemodeststoragespace(alsoknownasthe"footprint")budgetedforaudio onthemediaandinmemory,thisleavestheaudioproducerinabitofaquandary.Forthe issueof underscore,agametitlewithhoursofgameplaymightonlybebudgetedforafew minutesworthoflinearmusic.Theaudiodirectormustdevelopcreativewaystokeepthis musicfresh—alternativeversions,versionbranching,andsoon.Audioprogrammerscan investigateandimplementthesesolutionsusingDirectMusic. Astoeventstriggeringambience,dialog,andspecificsoundeffects,thesemayrepeat numeroustimes,addingthechallengeof avoidingthekindofobviousrepetitionthatcan spoil agame'srealismfortheplayer.Aswediscussinmoredetail,DirectMusicprovides numerousmethodsforhelpingtoavoidrepetition.Onthemostbasiclevel,audio programmerscanspecifyvariationsinpitchandmultipleversionsof wave,note,and controllerdata.Evengamecontent(specificscriptedeventsinthegameforinstance)can specifyorderingsforplayback(suchasshuffling,norepeats,andsoon)thatDirectMusic tracksasthegameprogresses.Usingadvancedfeatures,chordprogressionscanmaintain numerouspotential progressionpaths,allowingalimitedamountof sourcematerial to remainfreshevenlonger.