Fast Fourier transform algorithms with applications [PhD thesis]

340 Pages·2008·1.38 MB·English

by  Todd Mateer

FAST FOURIER TRANSFORM ALGORITHMS WITH APPLICATIONS
A Dissertation
Presented to
the Graduate School of
Clemson University
In Partial Fulfillment
of the Requirements for the Degree
Doctor of Philosophy
Mathematical Sciences
by
Todd Mateer
August 2008
Accepted by:
Dr. Shuhong Gao, Committee Chair
Dr. Joel Brawley
Dr. Neil Calkin
Dr. Kevin James
ABSTRACT
This manuscript describes a number of algorithms that can be used to quickly
evaluate a polynomial over a collection of points and interpolate these evaluations
back into a polynomial. Engineers define the “Fast Fourier Transform” as a method
of solving the interpolation problem where the coefficient ring used to construct the
polynomials has a special multiplicative structure. Mathematicians define the “Fast
Fourier Transform” as a method of solving the multipoint evaluation problem. One
purpose of the document is to provide a mathematical treatment of the topic of
the “Fast Fourier Transform” that can also be understood by someone who has an
understanding of the topic from the engineering perspective.
Themanuscriptwillalsointroduceseveralnewalgorithmsthatefficientlysolve
the multipoint evaluation problem over certain finite fields and require fewer finite
field operations than existing techniques. The document will also demonstrate that
these new algorithms can be used to multiply polynomials with finite field coefficients
with fewer operations than Sch¨onhage’s algorithm in most circumstances.
A third objective of this document is to provide a mathematical perspective
of several algorithms which can be used to multiply polynomials whose size is not a
power of two. Several improvements to these algorithms will also be discussed.
Finally, the document will describe several applications of the “Fast Fourier
Transform” algorithms presented and will introduce improvements in several of these
applications. In addition to polynomial multiplication, the applications of polynomial
division with remainder, the greatest common divisor, decoding of Reed-Solomon
error-correcting codes, and the computation of the coefficients of a discrete Fourier
series will be addressed.
ii
DEDICATION
I dedicate this work to my wife Jennifer and my children Nathan, Laura,
Jonathan, and Daniel. In terms of our family, the progress of my graduate research
program has been measured through a collection of fifty quarters produced by the
United States mint over roughly the same ten year period while I completed my
graduate studies. I look forward to placing the final quarter on our “doctor school”
map at the end of this year (2008) when I anticipate being finished with several
publications related to the research presented in this manuscript.
I have really treasured this time to be at home with my family while my
children were young and for their “company” and “support” while completing this
project. Much of this dissertation was written with little children in my lap or by
my side as my wife and I worked together to get through these first few years of
parenthood. I consider this time to be more valuable than the degree for which this
dissertation was written.
iii
ACKNOWLEDGMENTS
There are many people who deserve recognition for their role in my education
whichhasledmetothispointinmyacademiccareer. Iwillattempttobeascomplete
as possible here, knowing that are likely several people that I have left out.
Obviously, my advisor Shuhong Gao and committee members deserve mention
for being willing to mentor me through this process. This was an especially more
challenging task given the fact that we were separated geographically during the
entire writing of this dissertation. In fact, my advisor and I only saw each other two
times in the three years it took to complete the research program. I would also like
to thank Shuhong Gao for teaching a computer algebra class in 2001 which got me
interested in this topic for my doctoral studies.
I would also like to thank several anonymous reviewers who read over this
entire manuscript several times. There are likely more places where the document
can be improved and if there is an interest, I will make revisions to the manuscript
as these places are pointed out to me.
I would also like this opportunity to thank the teachers which taught me
the subjects related to this research program. In particular, I would like to thank
Rebecca Nowakowski, James Payne, Dale McIntyre, Yao-Huan Xu, Jenny Key, and
Joel Brawley for equipping me with the algebra and complex variables background to
takeonthisassignment. Also,TimothyMohr,FrankDudaandRobertMuellertaught
me the signals analysis which introduced me to the “engineering perspective” of the
FFT. InclassestaughtbyJoeChurmandRobertJamison, Ilearnedhowthematerial
of this dissertation is closely related to basic music theory. Through the instruction of
Curt Frank, Jim Kendall, Fred Jenny, Michelle Claus, and James Peterson I learned
iv
thecomputerscienceandprogrammingskillsneededforthisresearchproject. Finally,
I would like to thank my father Robert Mateer who taught me the trigonometry that
is so fundamental to the topic of this dissertation. While I turned out to be an
algebraist instead of following in his footsteps and making analysis (i.e. Calculus) my
specialty, I still hope to become as good of a teacher as my father someday.
...but there is a God in heaven who reveals mysteries.
Daniel 2:28a
v
TABLE OF CONTENTS
Page
TITLE PAGE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
CHAPTER
1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 The mathematician’s perspective . . . . . . . . . . . . . . . . . . 3
1.2 Prerequisite mathematics . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Operation counts of the algorithms . . . . . . . . . . . . . . . . . 5
1.4 Multipoint polynomial evaluation . . . . . . . . . . . . . . . . . . 7
1.5 Fast multipoint evaluation . . . . . . . . . . . . . . . . . . . . . 9
1.6 Lagrangian interpolation . . . . . . . . . . . . . . . . . . . . . . 15
1.7 Fast interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.8 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 23
2. MULTIPLICATIVE FAST FOURIER TRANSFORM ALGORITHMS 24
2.1 The bit reversal function . . . . . . . . . . . . . . . . . . . . . . 25
2.2 Classical radix-2 FFT . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 Twisted radix-2 FFT . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4 Hybrid radix-2 FFTs . . . . . . . . . . . . . . . . . . . . . . . . 35
2.5 Classical radix-4 FFT . . . . . . . . . . . . . . . . . . . . . . . . 36
2.6 Twisted radix-4 FFT . . . . . . . . . . . . . . . . . . . . . . . . 41
2.7 Radix-8 FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.8 Split-radix FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.9 Modified split-radix FFT . . . . . . . . . . . . . . . . . . . . . . 54
2.10 The ternary reversal function . . . . . . . . . . . . . . . . . . . . 59
2.11 Classical radix-3 FFT . . . . . . . . . . . . . . . . . . . . . . . . 60
2.12 Twisted radix-3 FFT . . . . . . . . . . . . . . . . . . . . . . . . 66
2.13 Hybrid radix-3 FFTs . . . . . . . . . . . . . . . . . . . . . . . . 73
vi
Table of Contents (Continued)
Page
2.14 Radix-3 FFT for symbolic roots of unity . . . . . . . . . . . . . . 75
2.15 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 77
3. ADDITIVE FAST FOURIER TRANSFORM ALGORITHMS . . . . 78
3.1 Von zur Gathen-Gerhard additive FFT . . . . . . . . . . . . . . 78
3.2 Wang-Zhu-Cantor additive FFT . . . . . . . . . . . . . . . . . . 87
3.3 Shifted additive FFT . . . . . . . . . . . . . . . . . . . . . . . . 94
3.4 Gao’s additive FFT . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.5 A new additive FFT . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.6 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 115
4. INVERSE FAST FOURIER TRANSFORM ALGORITHMS . . . . . 116
4.1 Classical radix-2 IFFT . . . . . . . . . . . . . . . . . . . . . . . . 116
4.2 Twisted radix-2 IFFT . . . . . . . . . . . . . . . . . . . . . . . . 120
4.3 Other multiplicative IFFTs . . . . . . . . . . . . . . . . . . . . . 122
4.4 Wang-Zhu-Cantor additive IFFT . . . . . . . . . . . . . . . . . . 129
4.5 A new additive IFFT . . . . . . . . . . . . . . . . . . . . . . . . 132
4.6 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 139
5. POLYNOMIAL MULTIPLICATION ALGORITHMS . . . . . . . . . 141
5.1 Karatsuba multiplication . . . . . . . . . . . . . . . . . . . . . . 141
5.2 Karatsuba’s algorithm for other sizes . . . . . . . . . . . . . . . . 145
5.3 FFT-based multiplication . . . . . . . . . . . . . . . . . . . . . . 146
5.4 Sch¨onhage’s algorithm . . . . . . . . . . . . . . . . . . . . . . . . 148
5.5 FFT-based multiplication using the new additive FFT algorithm 156
5.6 Comparison of the multiplication algorithms . . . . . . . . . . . 158
5.7 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 160
6. TRUNCATED FAST FOURIER TRANSFORM ALGORITHMS . . . 162
6.1 A truncated FFT algorithm . . . . . . . . . . . . . . . . . . . . . 164
6.2 An inverse truncated FFT algorithm . . . . . . . . . . . . . . . . 167
6.3 Illustration of truncated FFT algorithms . . . . . . . . . . . . . 175
6.4 Truncated algorithms based on roots of xN 1 . . . . . . . . . . 177
−
6.5 Truncated algorithms based on roots of xN x . . . . . . . . . . 179
−
6.6 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 180
7. POLYNOMIAL DIVISION WITH REMAINDER . . . . . . . . . . . 181
7.1 Classical division . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
vii
Table of Contents (Continued)
Page
7.2 Newton division . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
7.3 Newton divison using the multiplicative FFT . . . . . . . . . . . 188
7.4 Newton divison for finite fields of characteristic 2 . . . . . . . . 192
7.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 194
8. THE EUCLIDEAN ALGORITHM . . . . . . . . . . . . . . . . . . . . 195
8.1 The Euclidean Algorithm . . . . . . . . . . . . . . . . . . . . . . 195
8.2 The Extended Euclidean Algorithm . . . . . . . . . . . . . . . . 199
8.3 Normalized Extended Euclidean Algorithm . . . . . . . . . . . . 209
8.4 The Fast Euclidean Algorithm . . . . . . . . . . . . . . . . . . . 210
8.5 Algorithm improvements due to the Fast Fourier Transform . . . 226
8.6 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 231
9. REED-SOLOMON ERROR-CORRECTING CODES . . . . . . . . . 234
9.1 Systematic encoding of Reed-Solomon codewords . . . . . . . . . 235
9.2 A transform of the Reed-Solomon codeword . . . . . . . . . . . . 237
9.3 Decoding of systematic Reed-Solomon codewords . . . . . . . . . 240
9.4 Pseudocode and operation count of the simple decoding algorithm 246
9.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 249
10.FURTHER APPLICATIONS AND CONCLUDING REMARKS . . . 251
10.1 Computing the coefficients of a discrete Fourier series . . . . . . 251
10.2 Fast multipoint evaluation: revisited . . . . . . . . . . . . . . . . 255
10.3 Fast interpolation: revisited . . . . . . . . . . . . . . . . . . . . . 257
10.4 Other research areas involving FFT algorithms . . . . . . . . . . 258
10.5 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . 260
APPENDICES
A. Master equations for algorithm operation counts . . . . . . . . . . . . 263
B. Operation count: split-radix FFT . . . . . . . . . . . . . . . . . . . . 274
C. Additional details of the modified split-radix FFT . . . . . . . . . . . 278
D. Complex conjugate properties . . . . . . . . . . . . . . . . . . . . . . 285
E. Proof of the existence of the Cantor basis . . . . . . . . . . . . . . . . 287
F. Taylor shift of a polynomial with finite field coefficients . . . . . . . . 294
G. Taylor expansion of a polynomial with finite field coefficients at xτ . . 298
H. Additional recurrence relation solutions for additive FFT algorithms . 302
I. Operation count: Karatsuba’s multiplication algorithm. . . . . . . . . 305
J. Operation count: Sch¨onhage’s algorithm . . . . . . . . . . . . . . . . . 307
viii
Table of Contents (Continued)
Page
K. Karatsuba’s algorithm in FFT-based multiplication using the new ad-
ditive FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
L. Reischert’s multiplication method . . . . . . . . . . . . . . . . . . . . 311
M. Two positions on future polynomial multiplication algorithm perfor-
mance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
N. Complexity of truncated algorithms . . . . . . . . . . . . . . . . . . . 317
O. Alternative derivation of Newton’s Method . . . . . . . . . . . . . . . 319
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
ix
LIST OF TABLES
Table Page
5.1 AdditioncostcomparisonbetweenSch¨onhage’salgorithmandFFT-based
multiplication using the new additive FFT algorithm . . . . . . . . . 160
L.1 Operation counts of “pointwise products” involved in Reischert’s multi-
plication method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
x