Discrete cosine and sine transforms: general properties, fast algorithms and integer approximations

353 Pages·2007·1.75 MB·English

by  Vladimir Britanak

Prelims-P373624.tex 7/8/2006 13:3 Pageix
Preface
Since the book, Discrete Cosine Transform by K. R. Rao and P.Yip (Academic Press,
Boston) was published in 1990, the discrete cosine transform (DCT) has increasingly
attractedtheattentionofscientific,engineeringandresearchcommunities.TheDCTisused
inmanyapplicationsandindatacompressioninparticular.Thisisduetothefactthatthe
DCThasexcellentenergy-packingcapabilityandalsoapproachesthestatisticallyoptimal
Karhunen–Loévetransform(KLT)indecorrrelatingasignal.Thedevelopmentofvarious
fastalgorithmsfortheefficientimplementationoftheDCTinvolvingrealarithmeticonly,
further contributed to its popularity. In the last several years there have been significant
advancesanddevelopmentsinboththeoryandapplicationsrelatingtotransformprocessing
ofsignals. Inparticular, digitalprocessingmotivatedtheinvestigationofotherformsof
DCTs for their integer approximations. International standards organizations (ISO/IEC
andITU-T)haveadoptedtheuseofvariousformsoftheintegerDCT.Atthesametime,
the investigation of other forms of discrete sine transforms (DSTs) has made a similar
impact.Thereisthereforeaneedtoextendthecoveragetoincludethesetechniques.This
bookisaimedatdoingjustthat.
Theauthorshaveretainedmuchofthebasictheoryoftransformsandtransformprocessing,
since the basic mathematics remains valid and valuable. The theory and fast algorithms
oftheDCTs,aswellasthosefortheDSTs,aredealtwithingreatdetail.Thereisalsoan
appendixcoveringsomeofthefundamentalmathematicalaspectsunderlyingthetheory
oftransforms.ItisnoexaggerationtosaythatapplicationsusingDCTarenumerousand
itiswiththisinmindthattheauthorshavedecidednottoincludeapplicationsexplicitly.
Readers of this book will either have practical problems requiring the use of DCT, or
want to examine the more general theory and techniques for future applications. There
isnopracticalwayofcomprehensivelydealingwithallpossibleapplications. However,
it must be emphasized that implementation of the various transforms is considered an
integralpartofourpresentation.Itistheauthors’hopethatreaderswillnotonlygainsome
understandingofthevarioustransforms,butalsotakethisknowledgetoapplytowhatever
processingproblemstheymayencounter.
ThebookDiscreteCosineandSineTransforms:Generalproperties,Fastalgorithmsand
IntegerApproximationsisaimedatboththenoviceandtheexpert.Theferventhopesand
aspirationsoftheauthorsarethatthelatestdevelopmentsinthegeneralDCT/DSTfield
ix
Prelims-P373624.tex 7/8/2006 13:3 Pagex
x Preface
furtherleadintoadditionalapplicationsandalsoprovidetheincentiveandinspirationto
further modify/customize these transforms with the overall motivation to improve their
efficiencieswhileretainingthesimplicityinimplementations.
V.Britanak
P.C.Yip
K.R.Rao
February2006
Prelims-P373624.tex 7/8/2006 13:3 Pagexi
Acknowledgments
Manymanyhoursandeffortshavebeenspentbehindthecomputerinpreparingandtyping
electronicformofthemanuscript,inanalyticalderivationofvariousmatrixfactorizations,
drawingthefiguresofsignalflowgraphsandverifyingbycomputerprograms.Thisbook
istheresultoflong-termassociationofthethreeauthors,V.Britanak,P.YipandK.R.Rao.
Special thanks go to their respective families for their support, perseverance and under-
standing.Weappreciatealsothecontinuedencouragementandmanyhelpfulsuggestions
ofourcolleaguesandfriendsinthedepartmentsoftheinstituteanduniversities.Especially,
theleadingauthorwishestothankhislove,Gulinka,forherpatience,understandingand
encouragementduringtheyearsofpreparationofthisbook.Finally,itisalsoappropriate
toacknowledgeherethefinancialsupportprovidedbySlovakScientificAgencyVEGA,
ProjectNo.2/4149/24.
TheauthorshavebeenhonoredtohaveworkedwithAcademicPressInc.ElsevierScience
onthisproject.Theencouragement,supportandunderstandingforthedelayedcompletion
of the book provided by the publishing editorial staff at Materials Science Department,
andinparticular,ChristopherGreenwellandDr.JonathanAgbenyega,havebeengreatly
appreciated.
xi
Prelims-P373624.tex 7/8/2006 13:3 Pagexiii
List of Acronyms
1-D One-Dimensional
2-D Two-dimensional
3-D Three-dimensional
BinDCT BinaryarithmeticDCT
BinDST BinaryarithmeticDST
CMT C-MatrixTransform
CPU CentralProcessorUnit
DCT DiscreteCosineTransform
DFT DiscreteFourierTransform
DHT DiscreteHartleyTransform
DLU DLUmatrixfactorizationorDLUcomputationalstructure
DPCM DifferentialPulseCodeModulation
DST DiscreteSineTransform
DTT DiscreteTrigonometricTransform
DUL DULmatrixfactorizationorDULcomputationalstructure
DWT DiscreteWTransform
EOT Even/OddTransform
FCT FourierCosineTransform
FFCT FastFourierCosineTransform
FFT FastFourierTransform
FRDCT FractionalDiscreteCosineTransform
FRDFT FractionalDiscreteFourierTransform
FRDST FractionalDiscreteSineTransform
FST FourierSineTransform
GCMT GeneralizedC-MatrixTransform
GCT GeneralizedChenTransform
GDFT GeneralizedDiscreteFourierTransform
GDHT GeneralizedDiscreteHartleyTransform
HA HalfsampleAntisymmetric
HS HalfsampleSymmetric
ICT IntegerCosineTransform
IEC InternationalElectrotechnicalCommission
IEEE InstituteofElectricalandElectronicsEngineers
IntDCT IntegerDCT
xiii
Prelims-P373624.tex 7/8/2006 13:3 Pagexiv
xiv ListofAcronyms
INTDCT IntegerDCT
IntDST IntegerDST
ISO InternationalOrganizationforStandardization
IST IntegerSineTransform
ITU-T InternationalTelecommunicationStandardizationSector
JPEG2000 JointPhotographicExpertsGroup
KLT Karhunen–LoéveTransform
LDCT LosslessDCT
LDST LosslessDST
LDU LDU matrixfactorization
LU LU matrixfactorization
LUL LULmatrixfactorizationorLULcomputationalstructure
MDCT ModifiedDiscreteCosineTransform
MDL Multi-Dimensionalcomputationalstructure
MDST ModifiedDiscreteSineTransform
MLT ModulatedLappedTransform
MPEG MovingPictureExpertsGroup
MSE MeanSquareError
PCA PrincipalComponentAnalysis
PLUS PLUSmatrixfactorization
POS PointsOfSymmetry
QR QRmatrixfactorization
RDCT ReversibleDCT
RDST ReversibleDST
SCT SymmetricCosineTransform
SignDCT SignedDCT
SMT S-MatrixTransform
SOPOT SumOfPowersOfTwo
SST SymmetricSineTransform
SVD SingularValueDecomposition
ULD ULDmatrixfactorizationorULDcomputationalstructure
ULU ULUmatrixfactorizationorULUcomputationalstructure
VLSI VeryLarge-ScaleIntegration
WA WholesampleAntisymmetric
WHT Walsh–HadamardTransform
WS WholesampleSymmetric
Ch01-P373624.tex 7/8/2006 12:52 Page1
CHAPTER1
Discrete Cosine and Sine Transforms
1.1 Introduction
Since the publication of original book [1] more than 15 years ago many new con-
tributions/extensions/modifications/updates/improvements to the origin, theoretical and
practical aspects of the discrete cosine transforms (DCTs) and discrete sine transforms
(DSTs) have been developed.Although the original book [1] has focused almost exclu-
sivelyonthefastalgorithmsandapplicationsoftheDCToftypeII(DCT-II)whichhas
becometheheartofmanyestablishedinternationalimage/videocodingstandards[2],since
thenotherformsoftheDCTandDSThavebeeninvestigatedindetail.Thecompletesetof
DCTsandDSTs,calledthediscretetrigonometrictransforms,hasfoundanumberofdigital
signalprocessingapplications.Amongthem,forexample,theDCT/DSToftypeIV(DCT-
IV/DST-IV)andDCT-II/DST-IIareusedfortheefficientimplementationoflappedorthog-
onaltransforms[6]andperfectreconstructioncosine/sinemodulatedfilterbanks(known
asmodifieddiscretecosine/sinetransforms(MDCTs/MDSTs)orequivalentlymodulated
lappedtransforms(MLTs)[6])forhigh-qualitytransform/subbandaudiocoding.
ThecompletesetofDCTsandDSTsconstitutingtheentireclassofdiscretesinusoidaluni-
tarytransformsispresentedincludingtheirdefinitions,generalmathematicalproperties,
relationstotheKarhunen–Loèvetransform(KLT),withtheemphasisonfastalgorithms
andintegerapproximationsfortheirefficientimplementationsintheintegerdomain.The
DCTs and DSTs are real-valued transforms that map integer-valued signals to floating-
pointcoefficients.OneoftheimportantissuesfortheapplicabilityofDCTsandDSTsis
the existence of fast algorithms that allow their efficient computation.Although the fast
algorithmsreducethecomputationalcomplexitysignificantly,theystillneedfloating-point
operations.Toeliminatethefloating-pointoperations,methodsofintegerapproximations
havebeenproposedtoconstructandflexiblygenerateafamilyofintegertransformswith
arbitraryaccuracyandperformance.Theintegertransformscurrentlyrepresentthemod-
erntransformtechnologiesforlosslesstransform-basedcoding.TheintegerDCTs/DSTs
withlow-costandlow-poweredimplementationcanreplacethecorrespondingreal-valued
transformsinwirelessandsatellitecommunicationsystemsaswellasportablecomputing
applications.
1
Ch01-P373624.tex 7/8/2006 12:52 Page2
2 DiscreteCosineandSineTransforms
ThebookcoversvariouslatestdevelopmentsinDCTsandDSTsinaunifiedway,anditis
essentiallyadetailedexcursiononorthogonal/orthonormalDCTandDSTmatrices,their
matrixfactorizationsandintegerapproximations. Itishopedthatthebookwillserveas
anexcellentreferenceindevelopingintegerDCTsandDSTsaswellasaninspirationfor
furtheradvancedresearch.
1.2 Organizationofthebook
The book is organized in terms of chapters starting with this introductory chapter; each
chapterhasitsownlistofgeneralreferencesandappendices.
Chapter2coversdefinitionsandgeneralpropertiesofclassicalintegraltransforms,Fourier
cosinetransformandFouriersinetransform.Thegeneralpropertiesofthesecontinuous
transforms such as inversion, linearity, shift in time/frequency, differentiation in time/
frequency, asymptotic behavior, integration in time/frequency and convolution in time
togetherwithexamplesofintegraltransformsforselectedcontinuousfunctionsarepre-
sentedinSections2.2–2.5.AlltheDCTsandDSTsarenotsimplydiscretizedversionsof
the corresponding integral continuous transforms rather, the discretized cosine and sine
functionsformthebasisfunctionsforanentirefamilyofDCTsandDSTs,andareactually
eigenfunctions(oreigenvectors)ofcertaintridiagonalmatrixforms.Thisissueisaddressed
inSections2.6and2.7.DCTsandDSTspossessnicemathematicalpropertiessuchasuni-
tarity,linearity,scalingandshiftintime,and,inparticular,convolutionpropertieswhich
arediscussedindetailinSections2.8and2.9.
KLT an optimal transform from a statistical viewpoint is defined in Chapter 3 (Section
3.2)alongwiththedemonstrationoftheasymptoticequivalenceofDCT-IandDCT-IIto
KLTinSection3.3.Section3.4addressestheasymptoticequivalenceofdifferenttypesof
correlationmatricesandtheirorthonormalrepresentationsleadingtoageneralprocedure
for generating certain discrete unitary transforms for a given class of signal correlation
matrices.
For the DCT and DST to be viable, feasible and practical, the fast algorithms for their
efficient implementation in terms of reduced memory, implementation complexity and
recursivity are essential. The fast algorithms for both one- and two-dimensional (1-D,
2-D,respectively)DCTs/DSTsarethemainthrustinChapter4.InSection4.2,thedefini-
tions,propertiesofandrelationsbetweenDCTsandDSTsarefirstpresented,followedby
presentationoftheexplicitformsoforthonormalDCTandDSTmatricesforN=2,4and
8 in Section 4.3. The fast 1-D rotation-based algorithms for the computation of DCTs
andDSTsbasedonthe(recursive)sparsematrixfactorizationsofthecorrespondingDCT
and DST matrices and represented by the generalized signal flow graphs are discussed
in Section 4.4. The matrix factorizations reveal various interrelations between different
versionsoftheDCTandDST.Theseselectedfastalgorithmsareveryconvenientincon-
structingintegerapproximationsofDCTsandDSTs.Section4.5analyzesexisting2-Dfast
DCT/DSTalgorithmsandsuggestsasimplemethodforgenerating2-DdirectDCT/DST
algorithmsfromthecorresponding1-Dones.
AsintegerversionsoftheDCT/DSThaveattractedtheattentionofresearchersresulting
insubstantialsimplificationintheirimplementationwhilestillmaintainingperformance
Ch01-P373624.tex 7/8/2006 12:52 Page3
DiscreteCosineandSineTransforms 3
nearlyequaltotheirearlierversions,itisonlylogicalthatthisarenabefocusedinmuch
detail and depth in Chapter 5. Section 5.2 presents the basic material from linear alge-
bra,theoryofmatricesandmatrixcomputationswhichisfundamentalforunderstanding
the approximation methods. In order to evaluate the approximation error between the
approximatedandoriginaltransformmatrixandtomeasuretheperformanceofresulting
approximatedtransformusedindatacompression,sometheoreticalcriteriaaredefinedin
Section5.3.Finally,variousdevelopedmethodsanddesignapproachestointegerapprox-
imation of the DCT and DST are detailed in Section 5.4. More recent developments in
designinglosslessDCTs,invertibleintegerDCTsandreversibleDCTsincludingthelatest
developmentsarediscussedinSections5.5and5.6.
Allchaptersendwithasummary,problems/exercisesandreferences.Problems/exercises
reflectthecontentsofthecorrespondingchaptersandareintendedforthereaderintermsof
refresh/review/reinforcetheircontents.Extensivedefinitions,principles,properties,signal
flowgraphs,derivations,proofsandexamplesareprovidedthroughoutthebookforproper
understandingofthestrengthsandshortcomingsofthespectrumofcosine/sinetransforms
andtheirapplicationindiversedisciplines.
1.3 Appendices
AppendicesA.1 throughA.3 review the important basic concepts of linear algebra such
asvectorspaces(AppendixA.1),matrixeigenvalueproblem(AppendixA.2)andmatrix
decompositions (AppendixA.3) in the form of definitions and theorems with exercises/
problemsattheend.Deterministicaswellasrandomsignals,theirclassificationandrepre-
sentationsarediscussedinAppendixA.4.AnumberofexamplesarelistedinAppendices
toillustratetheuseofbasicconceptsinpracticalapplications.
1.4 References
To retain the connectivity among the chapters of the book as much as possible, each
chapter in the book includes its own list of references related to the discussed sub-
ject. Therefore, some references may appear in the lists of references of chapters more
thanonce.
1.5 Additionalreferences
Anextensivelistofadditionalreferenceshavebeenappendedtothischapter.Noclaimfor
completenessofthislistismade.Additionalreferences,althoughnotcitedinsubsequent
chapters,reflectthevariousrecent/latestdevelopmentsintheefficientimplementationsof
DCTsandDSTs,mainly1-D,2-D,3-Dandingeneral,multi-dimensionalfastDCT/DST
algorithmsforthetimeperiodfrom1989/1990uptonow.Theysupplementthecompre-
hensivelistofreferencesrelatedtoDCTsandDSTsintheoriginalbooks[1,2].Thus,this
bookandbooks[1,2]covercompletelythetheoreticaldevelopments,algorithmichistory
ofDCTsandDSTsincludingtherecentactiveresearchtopics.
Ch01-P373624.tex 7/8/2006 12:52 Page4
4 DiscreteCosineandSineTransforms
For clarity, the additional references are classified into the following categories with
guidelines:
•
OtherbooksdiscussingDCTs,DSTsandKLT[3–9]
TherecentpublishedbooksdiscussboththetheoreticalandpracticalaspectsofDCTs
andDSTsincludingtheKLT.
•
Fast1-Dradix-2DCT/DSTalgorithms[10–67]
Thiscategoryisfurthersubdividedintothreeparts:fastalgorithmsforcomputation
of DCT-I, -II, -III, -IV and corresponding DST-I, -II, -III, -IV [10–32], fast DCT
algorithmsonly[33–59]andfastDSTalgorithmsonly[60–67].
•
Fastdirect2-DDCT/DSTalgorithms[68–87]
This category includes the direct 2-D radix-2 DCT/DST algorithms, and direct
even/prime-length2-Dalgorithmsbasedoncyclicconvolutionsandcircularorskew-
circularcorrelations.Since2-DDCT/DSTkernelsareseparable,the2-DDCT/DST
computation can simply be realized by the so-called row–column method which
sequentiallyusesanyfast1-DDCT/DSTalgorithmonrowsandcolumnsoftheinput
datamatrix.Ingeneral,many1-DDCT/DSTalgorithmscanbeextendedtothedirect
2-Dcaseusinga2-Ddecompositionprocess.
•
Fastdirect3-Dandmulti-dimensionalDCT/DSTalgorithms[88–97]
Thehigher-dimensionalDCT/DSTalgorithmscanbeobtainedbythesimilarmethods
asthoseof2-DDCT/DSTones.
•
Fast even/odd/composite-length, prime-factor, radix-q and mixed-radix DCT/
DSTalgorithms[98–126]
ThelimitationcommontomostfastDCT/DSTalgorithmsisthatN mustbeapower
of2(radix-2DCT/DSTalgorithms).Inpractice,varioussequencelengthsotherthan
apowerof2mayoccur.Todealwithsuchsequencelengths,newfasteven/odd-length
(N isaneven/oddinteger),composite-length(N=p·q,wherepandqarerelatively
primes), prime-factor, radix-q (N=qn, whereq isanoddinteger)andmixed-radix
(N=2n·q, where q=3,5,6,7,9, ...) DCT/DST algorithms have been proposed.
Even/odd-lengthandprime-factorDCT/DSTalgorithmscanbedirectlymappedinto
thecorrespondingeven/odd-lengthandprime-factorcomplex-valuedorreal-valued
FFTmodules,ortheyarebasedonshortercyclic/skew-cyclicconvolutionsandskew-
circular correlations. The algorithms for sequence lengths other than 2n need quite
differentmethodsfortheirderivation,andgenerallytheyhaveahighercomputational
complexityandhavemorecomplexstructure.
•
FastpruningDCTalgorithms[127–134]
ThestandardDCT(radix-2)algorithmsinherentlyassumethatthelengthsofinput
and output data sequences are equal. However, in many applications such as data
compression, the most important information about the signal is kept by the low-
frequency DCT coefficients. Therefore, from N coefficients (N being the length of
datasequence)onlyN (N <N)lowest-frequencycoefficientsneedtobecomputed.
1 1
Suchamethodwhereonlyasubsetoftheoutputcoefficientsisutilizedtoaccelerate
the computation is referred to as “pruning”. Therefore the algorithms, called fast
pruningDCTalgorithms,havebeendevelopedjustforthispurpose.Ingeneral,thefast
DCTalgorithmtobeprunedmustbedefinedbyasimplestructuredrecursivematrix
factorizationofthetransformmatrixandrepresentedbytheregularsignalflowgraph.
Ch01-P373624.tex 7/8/2006 12:52 Page5
DiscreteCosineandSineTransforms 5
•
DCT/DSTcomputationbyrecursivefilterstructures[135–149]
AclassofalgorithmsforarbitrarylengthforwardandinverseDCT/DSTcomputations
are recursive algorithms where DCT/DST kernels are converted to regular regres-
sive structures based on sinusoidal recursive formulae, or recurrence formulae for
Chebyshevpolynomials(ofthesecondandthirdkind),orClenshaw’srecurrencefor-
mula.Althoughtheserecursivealgorithmsarenotefficientintermsofcomputational
complexity,regressivestructuresprovidesimpleandefficientschemesfortheparallel
VLSIimplementationofthevariablelengthDCTs/DSTs.
•
FractionalDCTsandDSTs[150–152]
Recently,thefractionalDCTs(FRDCTs)andfractionalDSTs(FRDSTs)forDCT-II,
symmetric cosine and symmetric sine transforms have been introduced. The defi-
nitions of FRDCTs and FRDSTs are based on eigen decompositions (eigenvalues
andeigenvectors)ofthecorrespondingDCTandDSTmatrices;orsimplybyother
words, FRDCTs and FRDSTs are defined through the “fractional” real powers of
DCTandDSTmatrices.ItisthesameideaasthatofthefractionaldiscreteFourier
transform(FRDFT).TheinvestigationofFRDCTandFRDST,theirgeneralproper-
tiesarerecentlyanactiveandinterestingresearchtopic.Openproblemsinvolvethe
rigorousdefinitionsofFRDCTsandFRDSTsforotherformsofDCTandDST,study
oftheirgeneralproperties,matrixrepresentationsand,inparticular,fastalgorithms
fortheirpracticalimplementations[152].
•
FastquantumalgorithmsforDCTsandDSTs[153]
Quantumcomputinghasrecentlybecomeanexcitingareaofemergingdigitalsignal
processing applications. A classical computer does not allow to calculate N-point
DCTsorDSTs,whereN=2n,inlessthanlineartime.Thistriviallowerboundisno
longervalidforaquantumcomputer.Infact,itispossibletorealizeN-pointDCTs
and DSTs with as little as O(log 2N) operations on a quantum computer, whereas
2
the all known fast DCT/DST algorithms realized on a classical computer require
O(Nlog N)operations.BasedonexistingefficientquantumcircuitsfortheDFT,the
2
(extremely)fastquantumDCT/DSTalgorithmscanbederivedandimplementedon
anumberofquantumcomputingtechnologies.
Webelievethattheadditionalreferences,althoughnotusedinthebook,willbeavaluable
andusefulsourceforthereaderinher/hisfurtherstudyoradvancedresearchorinsolving
specificproblemsintheareaofDCT/DSTapplications.
References
[1] K. R. Rao and P.Yip, Discrete Cosine Transform: Algorithms, Advantages, Applications,
AcademicPress,BostonMA,1990.
[2] K.R.RaoandJ.J.Hwang,TechniquesandStandardsforDigitalImage/Video/AudioCoding,
Prentice-Hall,UpperSaddleRiver,NJ,1996.
OtherbooksdiscussingDCTs,DSTsandKLT
[3] A.K.Jain,FundamentalsofDigitalImageProcessing,Prentice-Hall,EnglewoodCliffs,NJ,
1989.