Investigating the role of musical genre in human perception of music stretching resistance

1.3 MB·

by  Jun Chen

Investigating the role of musical genre in human
∗
perception of music stretching resistance
Jun Chen+,1 and Chaokun Wang*,1
1SchoolofSoftware,TsinghuaUniversity,Beijing,100084,P.R.China.
[email protected]
*[email protected]
ABSTRACT
7
1
0
Tostretchamusicpiecetoagivenlengthisacommondemandinpeople’sdailylives,e.g.,inaudio-videosynchronizationand
2
animationproduction. However,itisnotalwaysguaranteedthatthestretchedmusicpieceisacceptableforgeneralaudience

n sincemusicstretchingsuffersfrompeople’sperceptualartefacts. Over-stretchingamusicpiecewillmakeituncomfortablefor
a humanpsychoacoustichearing. Theresearchonmusicstretchingresistanceattemptstoestimatethemaximumstretchability
J
ofmusicpiecestofurtheravoidover-stretch. Ithasbeenobservedthatmusicalgenrescansignificantlyimprovetheaccuracy

2 ofautomaticestimationofmusicstretchingresistance,buthowmusicalgenresarerelatedtomusicstretchingresistancehas
1
neverbeenexplainedorstudiedindetailintheliterature. Inthispaper,thecharacteristicsofmusicstretchingresistanceare

comparedacrossdifferentmusicalgenres. Itisfoundthatmusicstretchingresistancehasstrongintra-genrecohesivenessand
]
M inter-genrediscrepanciesintheexperiments. Moreover,theambiguityandthesymmetryofmusicstretchingresistanceare
alsoobservedintheexperimentalanalysis. Thesefindingsleadtoanewmeasurementonthesimilaritybetweendifferent
M musicalgenresbasedontheirmusicstretchingresistance. Inaddition,theanalysisofvariance(ANOVA)alsosupportsthe
findingsinthispaperbyverifyingthesignificanceofmusicalgenreinshapingmusicstretchingresistance.
.
s
c
[

1
v Introduction
4
7 Music stretching resistance (abbr. MSR) which describes the acceptable range of time stretching rate of music piece for
2 people’spsychoacoustichearing,1,2consistsoftheminimumcompressingrate,denotedasα (0.0<α <1.0),andthe
min min
3
maximum elongating rate, denoted as α (1.0<α <2.0).3 The research into MSR is of broad interest in fields like
0 max max
time-scalemodificationofspeech,1,2 musicresizing,4–6 dynamicmusicre-scaling7 aswellasotherfieldsrelatedtohuman
.
1 perceptionofmusicandpsychoacoustichearing. Thecomputationalmethod3whichestimatesMSRbyincorporatingsound
0
features(e.g.spectralanalysis,timbre,pitch)andmusicalgenres,hasshownthatmusicalgenreismuchmoreimportantin
7
1 affectingMSRcomparedwithsoundfeaturesliketimbre,pitchandrhythm. However,thereisstillnoin-depthresearchto
: investigatetherelationshipbetweenMSRandmusicalgenrestothebestofourknowledge.
v
Generally speaking, the existence of MSR can be attributed to the process of receiving and recognizing accelerated
i
X and decelerated sounds where people would make positive or negative reaction to accept/reject the changes based on the
r satisfactionoftheirpsychoacoustichearing. Basically,MSRisrelatedtotheperceptionofmusicandtheartefactsofdigital
a
signalprocessing. Humanperceptionofmusichasmuchincommonwithhumanrecognitionofnaturallanguages,where
thestructures(syntax/harmony),thevocabularies(words/chords),thetonalproperties(inflection/timbre)andthetemporal
features(prosody/rhythm)areshared.8 Meanwhile,thefunctionalmagneticresonanceimaging(fMRI)showsgreaterneuronal
activityinthevoice-selectiveregionsofparticipantswhentheyarelisteningtovocalsoundsthantonon-vocalsounds,9which
indicatesthatpeoplemayalsohavedifferentdegreesofsensitivitytowardsmusicwith/withoutlyrics,orwithsparse/dense
lyrics. ThisdifferencemaycontributetothecreationofMSRsincestretchingoperationschangethedensityoflyrics(e.g.words
perminute),too. Thecompressingorelongatingoperations1,2,4–6onagivenmusicpiecechangethetemposwhilepreserving
thepitchfeatures,1,2,10whichleadstoarangeofacceptabilityonuserpreferredspeeds11aswellasusertolerablespeedranges.
ThesearetheevidenceswhyMSRexistsforgeneralaudience.
Thedynamicattendingtheories12–15positthattheinternalmechanism,likeaclock,atimekeeperoranoscillatorinside
humanbeings, resonatesandsynchronizeswiththeperiodicityofstretchedmusicpiecestoenabletheaudiencetofollow
changesintempo. WithouttheknowledgeofMSR,however,perceptualartefacts6aremorelikelytooccuranddegradethe
auditoryexperienceofgenerallistenerswhenamusicpieceisstretchedatarateoutofitsacceptablestretchingrange,i.e.,
∗Ifyouareinterestedinthecollecteddata,pleasecontactDr.Chen:[email protected].
overlycompressedoroverlyelongated. Mostlistenersarelikelytoidentifythebasetempoofmusicaround120BPM(beatsper
minute),andtheaccelerationorthedecelerationinspeedusuallyinducesmoreambiguityintheidentificationoftempo.16,17
Tosomeextent,thena¨ıveapproximationofacceptablestretchingrange,i.e.,±20%(namelyα =0.80,α =1.20),18is
min max
baselessandfailstoutilizethefeaturesofmusicpieces,consideringthattheperceptionoftempoappropriatenessforagiven
musicpieceisdeterminedbyitscontents.19 Oneofthecategoricalfeaturesofmusicisgenrewhichincorporatescultural
backgroundsandemotionsofartists,andcharacterizesthesimilaritybetweenmusicpieces.20 Meanwhile,musicalgenresare
alsorelatedtomusiccontentfeatures,e.g.,timbre,pitchandrhythm,sothattheautomaticgenreclassificationalgorithms20–23
couldwork,whichmakesgenreapotentialfactorinstudyingMSR.
InChen’spreviouswork,3MSRvaluesarediscretizedaslabelsalongtheaxisofstretchingrate. TheestimationofMSRis
performedbyclassifyingthelabelofthegivenmusicpieceusingitssoundfeatures(spectralanalysis,timbre,pitchandtempo)
andmusicalgenrewiththemachinelearningtechniques. Itisobservedthatmusicalgenrehaslargercontributiontoimprove
theclassificationaccuracycomparedwithsoundfeatures. ButhowmusicalgenreisrelatedtoMSRhasneverbeenstudiedin
detail. WebelievethatitisnecessarytofurtherexploretherelationshipbetweenMSRandmusicalgenres.
Inthispaper,weinvestigatetheimportantroleofmusicalgenresinshapingMSR.WefindthatMSRtendstobeconstant
withsmallfluctuationwithinagivenmusicalgenre. Thesignificanceofmusicalgenresintheanalysisofvariancesubstantiates
theexistenceofinter-genrediscrepancyandintra-genrecohesivenessinMSR,i.e.,MSRvaluesarewidelydiscrepantamong
differentgenresandareinherentlycohesivewithinasamegenre. TheambiguityofMSRisalsodiversifiedfordifferentmusical
genres. Besides,theMSRvaluesofagivenmusicalgenrearesymmetricintermsoftherangeboundaries(i.e.,mean)aswell
astheambiguityofrangeboundaries(i.e.,standarddeviation). Theregressionlinesfromoriginaltemposofmusicpiecesto
MSRvalues(α andα )arealmosthorizontalwithinagivenmusicalgenreintheexperiments,whichfurtherindicates
min max
thatMSRhaslittlecorrelationwithtempounlikethatwithmusicalgenre. Besides,wealsoanalyzetheMSR-basedsimilarity
betweenmusicalgenresbycomputingtheoverlappingareaofcoveringregionsontheα –α coordinatesystem. This
min max
newmeasuremeantofsimilarityoffersanewperspectiveonmusicalgenresbasedonhumanperceptionofmusicstretching
resistance. MSRisapsychoacousticreflectionofhumanperceptionofmusic,11 andthestudyonMSRalsoshedsnewlighton
content-awaremusicadaption4–6anddynamicattendingtheory.12–15
Methods
Participants
Werecruited17collegestudentsasparticipantsintheexperiments. Theseparticipantsrangedinagefrom18to25yearsold,
with29.4%femaleand70.6%male. Mostofthemwerenon-musiciansexceptfortwoparticipantswhohadreceivedpiano
educationformorethan3yearsbythetimeofexperiments. Ithasbeenprovedthatlisteners(musiciansornon-musicians)can
makeconsistentjudgmentsonwhethermusicpiecesareplayedoverlyfastoroverlyslow.8,19 Thecompositionofparticipants
inourexperimentsissimilartotherealsituationinourdailylives. Theparticipantswereselectedamongthepeoplewho
enjoyedlisteningtomusicandwerewillingtospendatleasthalfanhoureverydaytoconducttheexperiments.Theexperiments
lastforaboutonemonthsothattheexperimentalresultswouldbelessinfluencedbytheshort-termchangesofparticipants’
physicalormentalstates,suchasmoods(happy,sad),time(morning,evening),locations(home,workplace),andweather
(rainy,sunny). Sincetheparticipantswereexpectedtoconductthelisteningexperimentsforalongperiodoftime,weexcluded
those short-term participants who could not make through the one-month experiments to get the results from the final 17
participants. Althoughthefinalnumberofparticipantsisnotverylarge,wetriedtominimizetheimpactofpersonalpreferences
byincreasingtheoverlapofmusicpiecesthatdifferentparticipantslistened,andmajority-votingtheresultstheyreported. All
theparticipantswerepaidalittlebitfortheirendeavorwhentheexperimentsfinished.
ExperimentalSettings
Weusedthecollectionof894songsfrom3inthisstudy. Thesesongsidenticallycover11musicalgenresasshowninTable1.
Thesesongswererandomlycrawledfromamusicwebsite1,andthegenresofthesesongswereannotatedbyreferringtothe
metadataofthesesongsaswellasthegenretaxonomyfromWikipediaandsomepopularmusicwebsites.
Allthesongsinthecollectionwerestretchedintimedomainusingthesynchronizedoverlap-add(SOLA)method24toavoid
pitchshift,whichwasimplementedintheSoundTouchlibrary2. Eachsongwasstretchedintodifferentversionsofdiscrete
stretchingratesbetween(0.00,2.00)witharatestep0.02. Thatis, eachsonghas49compressedversionswithstretching
ratesin{0.98,0.96,...,0.02}aswellas49elongatedversionswithstretchingratesin{1.02,1.04,...,1.98}. Pleasenotethatno
musicpiecewithstretchingratebeyond2.00willbeacceptablefromourexperiencesincetheelongationwouldhavedestroyed
themusicalstructuretoomuchandmadetheelongatedmusicpiecesounduncomfortable.
1http://www.top100.cn
2http://www.surina.net/soundtouch
2/8
Figure1. TheerrorbarsofMSRdistributionof11differentmusicalgenres.
Althoughtherearesomeotherdifferentmusicstretchingmethods,4–7wechooseSOLA24inthisworkbecauseSOLAis
amorefundamentalworkinthemusicstretchingliteratureandituniformlystretchesmusicpieces. Therefore,theresults
reportedinthispaperareaboutuniformlymusicstretching.
Procedure
Eachparticipantwasdeliveredatleastonepackageof20randomsongsfromourcollectionaswellasthe98stretchedversions
(49compressed+49elongated)foreachsong. Thegenrecompositionineachsongpackageisrandom,andsomeparticipants
weredeliveredmoresongpackagessincetheywerefasterinconductingthelisteningexperiments. Theparticipantswere
askedtolistentothesestretchedversionsoneafteranother,andjudgewhethereachstretchedversionisacceptable. They
couldchoosetheorderofstretchingratestolistenintheirpreferredwaysaslongastheycouldgivejudgementonα and
min
α . Weofferedtheparticipantsafewgeneraljudgingcriteria,e.g.,speedacceptance,lyricsdensityacceptanceandoverall
max
listeningacceptance. ThedegradationofsoundqualityafterstretchusingSOLAmethodalsoturnsouttobeanimportant
factortoinfluencepeople’spsychoacousticacceptance. Forexample,overelongationwillmakethesoundinterruptivewhile
overcompressionwillmessuptheaudiosignalsandmakeitsoundnoisy. Besidesthegivenjudgingcriteria,itisuptothe
participantstomaketheirownjudgementbasedonthecomforttheyfeelinthelistening. Thoughthetotalnumberofstretched
versionsofeachsongwasnotsmall,theparticipantdidnothavetolistentoallofthem,andtheyonlyneededtolocatethe
minimumandthemaximumacceptablestretchingratesofeachsongintheirpackage(s). Tofurtherhelptheparticipantsto
judgetheacceptanceofthestretchedversions,wealsodevelopedanewmusicplayerforthem,whichsupported‘one-click’
switchbetweentwostretchedversionsofasamesongatthesameposition,e.g.,30%ofthesong.25 Withthismusicplayer,it
wouldbeeasierfortheparticipantstojudgewhetherthelisteningversionisover-stretchedatthecurrentlyplayingsegment
comparedwiththeoriginalversionortheotheracceptableversions.
Whentheparticipantscompletedthetasksinthedeliveredpackage(s),(s)heinputtheresults,i.e.,theα andtheα of
min max
eachsonginthepackage,intoourexperimentalwebpages. Specifically,theparticipantsenterthevaluesofα (e.g. 0.82)
min
andα (e.g. 1.26)ofthesongsintheirpackage(s)throughtextfieldsonthewebpage. Theparticipantswereallowedto
max
correcttheirresultsbeforethefinalsubmissionbytheendoftheone-monthexperiment. Thus,theminimumcompressingrate
α andthemaximumelongatingrateα ofeachsonginourcollectionwereobtainedforlateranalysis.
min max
Results and Discussion
Weperformedtheanalysisofvariance(ANOVA)ontheMSRvaluesobtainedfromthelisteningexperiments. MSRvalues
representα andα ofeachsongreportedbytheparticipants. TheresultsshowthatMSRvaluesaresignificantlyaffected
min max
bymusicalgenres(F-test,alphalevelsarethe11genres,α : F(10,883)=74.683,P<0.001,α : F(10,883)=56.407,
min max
P<0.001). Next,therelationshipbetweenMSRandmusicalgenresisdiscussedindetailbasedontheexperimentalresults.
BasicMSRProperties
Fig.1illustratestheerrorbars(meansandstandarddeviations)ofMSRvaluesforthe11musicalgenres(thestatisticsare
showninTable1),fromwhichwecandrawthefollowingfourbasicconclusionsaboutMSR:
3/8
Genre #.Piece α α Slopeα Slopeα
min max min max
Pop 87 0.735±0.069 1.246±0.066 2.78×10−4 −5.30×10−5
Rock 95 0.742±0.083 1.235±0.064 6.85×10−6 −1.45×10−4
EasyListening 83 0.594±0.097 1.355±0.086 −3.15×10−4 6.75×10−5
Folk 80 0.698±0.057 1.283±0.056 −4.29×10−5 −1.47×10−4
Latin 68 0.702±0.066 1.287±0.056 −1.34×10−4 −1.25×10−5
Country 86 0.692±0.047 1.294±0.050 1.73×10−6 1.79×10−4
Hip-hop&Rap 82 0.789±0.036 1.207±0.036 6.98×10−5 3.12×10−6
R&B 90 0.729±0.045 1.259±0.040 −2.77×10−4 −1.71×10−5
Jazz&Blues 78 0.703±0.049 1.282±0.059 −4.11×10−5 −1.24×10−4
Classical 73 0.578±0.085 1.376±0.079 −4.50×10−4 4.89×10−4
Electronic 72 0.610±0.090 1.346±0.076 −7.79×10−5 5.31×10−5
Table1. ThemeansanddeviationsoftheMSRvaluesofdifferentmusicalgenres,aswellastheslopesofregressionlines
fromtheoriginaltempoofamusicpiecetoα /α withineachmusicalgenre.
min max
• Inter-GenreDiscrepancy: WidediscrepanciesinMSRvaluesunderdifferentgenresareobserved. Thepositionandthe
stretchofMSRofmusicalgenresarequitedifferent. Forinstance,EasyListening,ElectronicandClassicalmusichavea
widerstretchingrange(theintervalbetweenthemeanofα andthatofα ),whileHip-hop&RapandR&Bexhibit
min max
anarrowerstretchingreach. Itisinlinewiththecharactereffect(fastandslow)ofmusicpiecesonhumanpreferred
speedsandrangesofacceptability.11
• Intra-GenreCohesiveness: MSRvaluesaresubstantiatedtobecohesiveunderagivengenreaccordingtothesignifi-
canceofmusicalgenresintheanalysisofvariance.
• Ambiguity: TheambiguityofMSRisgreaterforEasyListening,ElectronicandClassicalmusicpiecesseenfromthe
standarddeviations. Thelargerthestandarddeviationis, thelargerambiguitytheMSRofamusicalgenreis. This
probablyresultsfromtherhythmicfeaturesofthesemusicalgenressincetheremayhardlybeanyfixedrhythmicpatterns
fortheseaforementionedgenres,forinstance,apieceofpianomusicorviolinmusic. Onthecontrary,songslikeR&B,
Hip-hop&Rapusuallyfollowasolidtempothroughoutthewholemusicpiece.
• Symmetry: Underagivengenre,MSRtendstobesymmetricbetweenα andα ,onboththerangeboundaries
min max
(mean)andtheambiguityofrangeboundaries(standarddeviation). Thispropertycomesfromthesymmetriccriteriathat
listenersusedtojudgeα andα . Supposedly,theoptimaltempoofagivensongshouldoccurnearitsbaseone. As
min max
aresult,ifasongisover-compressedandsoundsuncomfortable,itismorelikelythattheelongatedonewiththesame
shiftofstretchingrateincreasewillalsosounduneasy,andviceversa.
Intra-GenreLinearRegressionWithTempo
Thestretchingoperationsonagivenmusicpiecewillleadtoaninverselyproportionalrelationship,t = to,wherer isthe
s r
stretchingratebetweentherange(0.00,2.00),t andt arethetemposoftheoriginalmusicpieceandthatofthestretched
o s
versionmeasuredbybeatperminute,respectively. Evidently,t isfixedforagivenmusicpiece. Thus,t meetstheupper
o s
limitwhenrequalsα ,whilet reachesthefloorboundarywhenrequalsα . Soastoidentifytherelationshipbetween
min s max
boundarytemposandMSR,thelinearregressionisperformedfrombasetempostoα andα inthemusiccollection,
min max
respectively. Thisistostudywhetherornotsongswithdifferenttemposunderagivenmusicalgenrewouldgenerallyhave
differentα andα . Theregressionlinesarealmosthorizontalunderallgenressincetheirslopesareveryclosetozero
min max
(Table1). Forexample,thesteepestslopeinTable1is−4.89×10−4ofα ofClassicalMusic. Sincetheoriginaltempoofa
max
musicpiecemostlyvariesbetween[0,200]BPM,whichcanonlycauselessthan0.1biasintheα fromothermusicpieces
max
ofthisgenre. Consequently,foragivenmusicalgenre,musicpieceswithdifferentbasetemposusuallyhavesimilarMSR,
whichverifiestheintra-genrecohesivenessofMSRfromanotherpointofview. Underagivengenre,thelimitsofrarefixed,
andthustheupperandthefloorboundariesoft varyaccordingtothevalueoft ofagivenmusicpiece. Thefactthatthe
s o
regressedslopeswithineachmusicalgenrearealmostzeroisalsoasolidproofthatMSRhaslittlecorrelationwithtempo
unlikethatwithmusicalgenre.
MSR-basedMusicalGenreSimilarity
TofurtherinvestigatetherelationshipbetweenMSRandmusicalgenres,weillustratethescatterplotsoftheMSRvalueswithin
eachmusicalgenreinFig.2. Eachpointinthesepanelsrepresentsapair(α ,α ),whileeachpointmaycorrespondtoa
min max
4/8
Figure2. ThescatterplotsoftheMSRvaluedistributionwithineachmusicalgenre. Theabscissaandtheordinateofeach
pointinthepanelsrepresentthevaluesofα andα ,respectively. (A)istheassemblyofthepointsfrom(B)to(L).
min max
fewsongssharingthesameMSRvalues. Obviously,thecoveredregionofthepointswithineachmusicalgenrediversifies.
For example, the points in Fig. 2d cover a wide range which means the ambiguity/variance of MSR in this group is high,
however,thepointsinFig.2hareveryclosewitheachotherspreadinginasmallrange,andthustheambiguity/varianceof
MSRinthisgroupisrelativelylowerincontrast. TheassemblyofthesepointsinFig.2aalsoshowsthedifferenceintheMSR
valuedistributionofdifferentmusicalgenres,whichinspiresusthatwecandistinguishdifferentmusicalgenresbasedontheir
coveringareaofthepointsontheα −α coordinatesystem.
min max
WeusearectangletomarktheedgesoftheMSRregion(a.k.a.MSRrectangle)ontheα -α coordinatesystemfora
min max
givengenre. Therectangleislocatedbythefollowingfourcoordinates:
(Mean(α )±StdDev(α ),Mean(α )±StdDev(α )), (1)
min min max max
where Mean(α ) and StdDev(α ) represent the mean and the standard deviation of α in the given musical genre,
min min min
respectively. SoareMean(α )andStdDev(α ). ThesevaluesareshowninTable1. Fig.3ashowstheMSRrectangleof
max max
Popmusic. ThelargertheMSRrectangleis,themoreambiguityofMSRthegivenmusicalgenrehas. TheMSRrectangle
dividestheα −α coordinatesysteminto9partsonthefirstquadrantasillustratedinFig.3a. Part9canbeconsidered
min max
asthe ‘safe’stretching areaof thegivenmusical genresince thepoints inthis regionare withinthe acceptablestretching
range. Parts1,2,3,4,7arethe‘dangerous’stretchingareasofthegivenmusicalgenrebycontrast. Furthermore,Parts5,6,8can
thereforebeconsideredasthetransitionareasfromthe‘dangerous’stretchingregiontothe‘safe’stretchingregion. Ifthe
musicstretchingtasks1,2,4–6 areperformedwithinthe‘safe’region,thestretchedresultswillbethemostlikelytobeaccepted
bygeneralaudience. Onthecontrary,ifstretchedinthe‘dangerous’region,theresultswillusuallybeunacceptableforgeneral
audience. SincedifferentmusicalgenresleadtodifferentMSRrectangles,theseMSRrectanglescanbeusedtodistinguish
differentmusicalgenresfromtheperspectiveofMSRwhichhasneverbeenstudiedintheliterature.
TheareaofMSRrectangle,i.e.,4×StdDev(α )×StdDev(α )canbeusedtomeasuretherelativeMSRambiguityof
min max
musicalgenres. Moreover,therelationshipofMSRrectanglesfromdifferentmusicalgenresfallsintothreecategories: (1)
5/8
Figure3. TheMSR-basedsimilarityofmusicalgenres. (A)TheMSRrectangleofPopmusiccontainingpointsonthe
α −α coordinatesystem. (B)AnexampleofthepossiblerelationshipbetweenMSRrectangles. (C)Theillustrationof
min max
theMSRrectanglesofallmusicalgenres.
Genre Pop Rock EasyListening Folk Latin Country Hip-hop&Rap R&B Jazz&Blues Classical Electronic
Pop 1.0 0.713 0.021 0.323 0.334 0.219 0.159 0.395 0.346 0.0 0.032
Rock 1.0 0.018 0.255 0.259 0.168 0.244 0.339 0.275 1.7e-4 0.025
EasyListening 1.0 0.082 0.092 0.088 0.0 0.005 0.063 0.658 0.748
Folk 1.0 0.808 0.675 0.002 0.344 0.822 0.024 0.113
Latin 1.0 0.625 0.009 0.351 0.692 0.031 0.125
Country 1.0 0.0 0.223 0.648 0.024 0.125
Hip-hop&Rap 1.0 0.042 0.0 0.0 0.0
R&B 1.0 0.380 0.0 0.014
Jazz&Blues 1.0 0.010 0.092
Classical 1.0 0.492
Electronic 1.0
Table2. TheMSR-basedsimilaritybetweenmusicalgenres. Thesimilaritymatrixissymmetric,andonlyhalfthematrixis
shown. Theinclusionrelationshipisinboldmodeandtheexclusionrelationshipisinitalicmode. Theothersareinthe
intersectionrelationship.
Inclusion—onerectangleistotallyincludedinanotherrectangle,ortworectanglesareexactlythesame;(2)Exclusion—two
rectanglesaretotallydisjoint;(3)Intersection—tworectangleshavethecommonoverlapaswellasthedisjointpart. Fig.3b
showsanexampleofthepossiblerelationshipbetweenmusicalgenresbasedontheirMSRrectangles. Rect1isintersected
withRect2,whileRect3isincludedinRect2andexcludedfromRect1. Theoverlapsofrectanglesarefilledwithblack,and
wecancomputetheoverlapratiousingtheJaccardsimilarity:
Rect ∧Rect
i j
Sim(Rect,Rect )= . (2)
i j
Rect ∨Rect
i j
Sim(Rect,Rect )measureshowmuchsimilaritythemusicalgenresthatRect andRect represent,sharefromtheperspective
i j i j
ofMSR.TherelationshipsofMSRrectanglesofallmusicalgenresareshowninFig.3c. Wecanseethatalltheinclusion,the
exclusionandtheintersectionrelationshipexistamongthemusicalgenrespresentedinthispaper.
Next,wecomputedtheMSR-basedsimilaritybetweenmusicalgenresusingEq.2,andtheresultsareshowninTable2.
Sincethesimilaritymatrixissymmetric,onlyhalfthematrixisshown. Obviously,thesimilaritybetweentwomusicalgenres
whoseMSRrectanglessatisfytheexclusionrelationshipiszero(initalicmode). Theinclusionrelationship(inboldmode)
isfoundthatR&B⊆Pop,R&B⊆Rock,andHip-hop&Rap⊆Rockwhere⊆istheinclusionoperator. Mostoftherelationship
observed is intersection. However, the ratio of overlap in the intersection relationship varies between different pairs of
musicalgenres. TheMSR-basedsimilarityoffersanewlooktoexploretherelationshipbetweenmusicalgenres. Wecan
seefromTable2thathighMSR-basedsimilarityisobservedbetweensomemusicalgenrepairs,e.g.,Pop-Rock,Folk-Latin,
Folk-Jazz&Blues. Thiskindofsimilaritymaybeverydifficultorevenimpossibletostudyusingmeta-data,audiocontentor
culturalbackgroundssinceMSR-basedsimilarityisrelatedtohumanpsychoacousticperceptionofmusicaswellaspeople’s
degreesofself-adaptiontochangesofmusicsuchastempo,eventdensityandlyricsdensity.
WebelievethatourworkhasmadeprospectiveattemptsinstudyingtherelationshipbetweenMSRandmusicalgenres,and
moreissuesremaintobefurtherinvestigatedinthefuture. OurfindingsontheMSRholdnotonlyondigitalmusicrecordings
whicharestretchedusingcomplexsignalprocessingalgorithms.1,2,4–6 Itcanalsobeappliedinlivemusicperformance,for
example,theaccelerationandthedecelerationoflivepianoorviolinperformancestoaccompanysingersinaconcert.
6/8
References
1. Verhelst,W.&Roelands,M. Anoverlap-addtechniquebasedonwaveformsimilarity(wsola)forhighqualitytime-scale
modificationofspeech. InIEEEinternationalconferenceonacoustics,speechandsignalprocessing,554–557(1993).
2. Verhelst,W. Overlap-addmethodsfortime-scalingofspeech. SpeechCommunication30,207–221(2000).
3. Chen,J.&Wang,C. Automaticmusicstretchingresistanceclassificationusingaudiofeaturesandgenres. IEEESignal
ProcessingLetters20,1249–1252(2013).
4. Liu, Z., Wang, C., Wang, J., Wang, H. & Bai, Y. Adaptive music resizing with stretching, cropping and insertion.
MultimediaSystem19,359–380(2013).
5. Liu, Z., Wang, C., Bai, Y., Wang, H.&Wang, J. Musiz: agenericframeworkformusicresizingwithstretchingand
cropping. InACMMultimedia,523–532(2011).
6. Liu,Z.,Wang,C.,Guo,L.,Bai,Y.&Wang,J. Lydar: alyricsdensitybasedapproachtonon-homogeneousmusicresizing.
InIEEEinternationalconferenceonmultimediaandexpo,310–315(2010).
7. Wenner,S.,Bazin,J.-C.,Sorkine-Hornung,A.,Kim,C.&Gross,M. Scalablemusic: Automaticmusicretargetingand
synthesis. Eurographics32,345–354(2013).
8. Brennan, D.&Stevens, C. Theeffectofpitch, tempoandproportionalpitchandtempomanipulationonmemoryof
familiarmusicalexcerpts. InInternationalconferenceonmusicperceptionandcognition,1771–1778(2006).
9. Berlin,P.,Zatorre,R.J.,Lafaille,P.,Ahad,P.&Pike,B. Voice-selectiveareasinhumanauditorycortex. Nature403,
309–312(2000).
10. Madison,G.&Paulin,J. Ratingsofspeedinrealmusicasafunctionofbothoriginalandmanipulatedbeattempo. Journal
oftheAcousticalSocietyAmerica128,3032–3040(2010).
11. Bisesi,E.&Vicario,G.B. Psychoacousticaspectsofthespeedofmelodyperformance. InInternationalconferenceof
studentsofsystematicmusicology,7–11(2008).
12. Large,E.&Palmer,C. Perceivingtemporalregularityinmusic. MathematicalBehavior26,1–37(2002).
13. Jones,M.&Boltz,M. Dynamicattendingandresponsestotime. PsychologicalReview96,459–491(1989).
14. Drake,C.,Jones,M.&Baruch,C. Thedevelopmentofrhythmicattendinginauditorysequences: attunement,referent
period,focalattending. Cognition77,251–288(2000).
15. Large,E.&Jones,M. Thedynamicsofattending: howpeopletracktime-varyingevents. PsychologicalReview106,
119–159(1999).
16. Moelants,D.&Mckinney,M.F. Tempoperceptionandmusicalcontent: Whatmakesapiecefast,slowortemporally
ambiguous? InInternationalconferenceonmusicperceptionandcognition,558–562(2004).
17. Mckinney,M.F.&Moelants,D. Deviationsfromtheresonancetheoryoftempoinduction. InInternationalconferenceon
interdisciplinarymusicology,124–125(2004).
18. Lee, E., Nakra, T.M.&Borchers, J. You’retheconductor: arealisticinteractiveconductingsystemforchildren. In
Internationalconferenceonnewinterfacesformusicalexpression,68–73(2004).
19. Quinn,S.&Watt,R. Theperceptionoftempoinmusic. Perception35,267–280(2006).
20. Scaringella,N.,Zoia,G.&Mlynek,D. Automaticgenreclassificationofmusiccontent: asurvey. IEEESignalProcessing
Magazine23,133–141(2006).
21. Li,T.,Ogihara,M.&Li,Q. Acomparativestudyoncontent-basedmusicgenreclassification. InSIGIR,282–289(2003).
22. Tzanetakis,G.,Essl,G.&Cook,P. Automaticmusicalgenreclassificationofaudiosignals. IEEETransactiononSpeech
andAudioProcessing10,293–302(2002).
23. Bagci,U.&Erzin,E. Automaticclassificationofmusicalgenresusinginter-genresimilarity. IEEESignalProcessing
Letters14,521–524(2007).
24. Roucos,S.&Wilgus,A. Highqualitytime-scalemodificationforspeech. InIEEEinternationalconferenceonacoustics,
speechandsignalprocessing,493–496(1985).
25. Chen,J.&Wang,C. RESIC:Atoolformusicstretchingresistanceestimation. InMultiMediaModeling,LectureNotein
ComputerScience,386–389(2014).
7/8
Acknowledgements
Wewouldliketothankallthevolunteerswhoparticipatedinthelisteningexperimentsfortheircontributionswhichformthe
basisofthispaper.
8/8