Parallel Additive Fast Fourier Transform Algorithms

98 Pages·2016·1.63 MB·English

Parallel Additive Fast Fourier
Transform Algorithms
Matan Hamilis
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
Parallel Additive Fast Fourier
Transform Algorithms
Research Thesis
Submitted in partial fulfillment of the requirements
for the degree of Master of Science in Computer Science
Matan Hamilis
Submitted to the Senate
of the Technion — Israel Institute of Technology
Tamuz 5776 Haifa July 2016
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
The Research Thesis Was Done Under The Supervision of Prof. Eli Ben-Sasson and
Prof. Mark Silberstein, in the Faculty of Computer Science.
Some results in this thesis have been published as articles by the author and research
collaborators in conferences and journals during the course of the author’s doctoral
research period, the most up-to-date versions of which being:
ICS 16’ E. Ben-Sasson, M. Hamilis, M. Silberstein, and E. Tromer, Fast
Multiplication in Binary Fields on GPUs via Register Cache, Proceedings
th
of the 30 International Conference on Supercomputing, ACM 2016.
E. Ben-Sasson and I. Ben-Tov and A. Chiesa and A. Gabizon and D. Genkin
and M. Hamilis and E. Pergament and M. Riabzev and M. Silberstein
and E. Tromer and M. Virza, Computational integrity with a public
random string from quasi-linear PCPs, cryptology ePrint Archive, Report
2016/646.
Acknowledgements
First, I would like to thank my advisors, Eli and Mark, for their infinite patience and
for wholeheartedly answering every question I had in any time. For cultivating my
curiosity by posing new challenges, for giving me the tools to face these challenges and
for pointing out possible research directions along the way.
I would like to thank my family as well, for constantly pushing me to saturate my
uncontainable thirst for knowledge, for their inexpressible support and wisdom and for
helping me to make one step at a time towards my dreams.
I am thankful for having my friends inside and outside the Technion for being
there beside me along the way, for sharing their paths of lives and experiences with
mine and for helping me maintaining a balanced, exciting and gladdening life. The
importance of your interest in my work is immeasurable and without it, achieving the
same results would have been impossible. For those who were listening to me without
even understanding what I was talking about just so I would feel comfortable about my
achievements and for those who did understand me for their constructive commentary
and critique.
Last but not least I would like to thank my students along the years, which undoubt-
edly I have learned from more than they did from me.
The Generous Financial Help Of The Hilda and Manasche Ben-Shlomo Fellowship And
The Technion Is Gratefully Acknowledged.
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
Contents
List of Figures
List of Algorithms
Abstract 1
1 Introduction 3
2 Preliminaries 9
2.1 Finite Extension Fields’ Elements and Bases . . . . . . . . . . . . . . . . 9
2.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Polynomial Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Normal Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Theoretical Discussion 13
3.1 Fast Multiplication in GF(2n) . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.1 Generalization for Optimized Multiplication in k-Gapped Finite
Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.2 Finding a k-Gapped polynomial . . . . . . . . . . . . . . . . . . . 17
3.2 Generalizing Gao & Mateer’s Additive FFT for affine subspaces . . . . . 18
3.2.1 Taylor Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.2 Additive FFT in Binary Fields Over Affine Subspaces . . . . . . 19
4 CPU 23
4.1 Finite Field Arithmetics . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.1 Element Representation on CPU . . . . . . . . . . . . . . . . . . 23
4.1.2 Finite Field Library API . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.3 Implementation of multiplication in GF(264) . . . . . . . . . . . 24
4.2 Parallel FFT and inverse FFT implementation . . . . . . . . . . . . . . 25
5 GPU - Introduction of Register Cache 27
5.1 Introduction of GPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.2 Intra-warp register cache . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2.1 Example: 1D k-stencil . . . . . . . . . . . . . . . . . . . . . . . . 31
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
5.2.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6 GPU - Finite Field Multiplication 37
6.1 Sequential finite field multiplication . . . . . . . . . . . . . . . . . . . . . 37
6.1.1 The CPU CLMUL instruction . . . . . . . . . . . . . . . . . . . . . 37
6.1.2 Sequential polynomial multiplication . . . . . . . . . . . . . . . . 38
6.2 Parallel polynomial multiplication . . . . . . . . . . . . . . . . . . . . . 39
6.2.1 Bit slicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.2.2 Parallel polynomial multiplication using chunks . . . . . . . . . . 40
6.3 Polynomial multiplication using register cache . . . . . . . . . . . . . . . 41
6.4 Extending to polynomials of larger degrees . . . . . . . . . . . . . . . . 43
6.4.1 Performance comparison of the different designs . . . . . . . . . 44
6.4.2 Application to larger fields . . . . . . . . . . . . . . . . . . . . . 45
6.4.3 Using shared memory only for the output . . . . . . . . . . . . . 46
7 Implementation of the FFT algorithm on GPU 49
7.1 Outline of the Implementation . . . . . . . . . . . . . . . . . . . . . . . 49
7.2 Set Up for GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7.3 Shift Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7.4 Taylor Expansion Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.5 Shuffle Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.6 Linear Evaluation Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.7 Merge Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8 Performance 63
8.1 FFT and Inverse FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
9 Conclusion and Open Questions 69
9.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
9.2 Some open questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Hebrew Abstract i
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
List of Figures
5.1 Input distribution in 1-stencil computation . . . . . . . . . . . . . . . . 32
5.2 Speedup obtained from coarsening in the computation of 1−Stencil and
7 − Stencil for register cache and shared memory implementation . . . . 35
5.3 Speedup of the shuffle-based k-Stencil implementation over the shared
memory-based implementation as a function of k . . . . . . . . . . . . . 35
6.1 Illustration of the access pattern of the multiplication algorithm for
GF(24) with W =4. Each frame encloses the indexes of rows in A and B
accessed for computing the respective rows ci specified on the top. Tid
denotes the thread index in the warp. . . . . . . . . . . . . . . . . . . . 39
6.2 Polynomial addition in 4-bit chunks. Computing the output chunk
requires 3 bit-wise XORs, each performing 4 concurrent ⊕ operations. . 40
7.1 Storage of Coefficients of Input FFT Polynomial in Chunks . . . . . . . 49
7.2 Outline of the FFT Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 51
7.3 Outline of a Single Split Iteration . . . . . . . . . . . . . . . . . . . . . . 51
7.4 Outline of the Taylor Expansion Algorithm . . . . . . . . . . . . . . . . 54
7.5 Applying Permutation σd on a Chunks’ Elements . . . . . . . . . . . . . 57
7.6 Applying Permutation π8 on a Chunks’ Elements . . . . . . . . . . . . . 58
7.7 Linear Evaluation Phase Applied Over a Single Chunk . . . . . . . . . . 59
8.1 Speedup of register cache multiplication in GF(264) and GF(232) over
NTL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
8.2 Speedup over NTL for varying field sizes . . . . . . . . . . . . . . . . . . 65
8.3 Finite field multiplication performance for GF(2N) where N is not a
power of 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
8.4 Comparison of GPU and a single threaded CPU implementation for FFT 66
8.5 Comparison of GPU and a single threaded CPU implementation for
inverse FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016
Technion - Computer Science Department - M.Sc. Thesis MSC-2016-15 - 2016