probabilistic boolean logic, arithmetic and architectures

156 Pages·2008·1.38 MB·English

PROBABILISTIC BOOLEAN LOGIC, ARITHMETIC
AND ARCHITECTURES
A Thesis
Presented to
The Academic Faculty
by
Lakshmi Narasimhan Barath Chakrapani
In Partial Fulfillment
of the Requirements for the Degree
Doctor of Philosophy in the
School of Computer Science, College of Computing
Georgia Institute of Technology
December 2008
PROBABILISTIC BOOLEAN LOGIC, ARITHMETIC
AND ARCHITECTURES
Approved by:
Professor Krishna V. Palem, Advisor Professor Trevor Mudge
School of Computer Science, College Department of Electrical Engineering
of Computing and Computer Science
Georgia Institute of Technology University of Michigan, Ann Arbor
Professor Sung Kyu Lim Professor Sudhakar Yalamanchili
School of Electrical and Computer School of Electrical and Computer
Engineering Engineering
Georgia Institute of Technology Georgia Institute of Technology
Professor Gabriel H. Loh Date Approved: 24 March 2008
College of Computing
Georgia Institute of Technology
To my parents
The source of my existence, inspiration and strength.
iii
ACKNOWLEDGEMENTS
आचायातर् ्पादमाद(cid:419)े पादं िशंयः ःवमेधया।
पादं सॄ(cid:442)चा(cid:464)र(cid:223)यः पादं कालबमेण च॥
“One fourth (of knowledge) from the teacher, one fourth from self study, one fourth
from fellow students and one fourth in due time” 1
Many people have played a profound role in the successful completion of this disser-
tation and I first apologize to those whose help I might have failed to acknowledge. I
express my sincere gratitude for everything you have done for me.
I express my gratitude to Professor Krisha V. Palem, for his energy, support and
guidance throughout the course of my graduate studies. Several key results per-
taining to the semantic model and the properties of probabilistic Boolean logic were
due to his brilliant insights. In this sense, the results pertaining to the semantic
model and the properties of probabilistic Boolean logic, were developed in a truly
collaborative manner. I express my sincere gratitude to Professor Zvi Kedem for his
enthusiasm and insightful feedback on the logic aspects of this work. I thank my
dissertation committee member, Professor Sudhakar Yalamanchili for his patience,
support and guidance during my stay at Georgia Tech. My dissertation committee
member, Professor Gabriel Loh provided me invaluable feedback and advice on my
research and writing. Given their standing in the microelectronics community and
their busy schedules, I consider myself very fortunate to have had the guidance and
encouragement of Professor James Meindl and Dr. Ralph Cavin. I thank my disser-
tation committee members, Professor Trevor Mudge and Professor Sung Kyu Lim for
1Ancientsubhashita, usuallyattributedtoHaradatta’scommentaryonApastambaDharmasutra
iv
their encouragement and suggestions. I wish to thank Professor Vincent Mooney for
his interest and encouragement. I wish to express my sincere and profound gratitude
to Dr. Ranjani Parthasarathi for stimulating my interest in computer science and
encouraging me to pursue graduate studies.
I would like to acknowledge the funding support from Defense Advanced Research
Projects Agency (DARPA). Dr. Robert Graybill supported our research, right from
its infancy, and enabled us to pursue it through a DARPA seedling in 2002. I wish
to thank Dr. Shekhar Borkar, Dr. Vivek De and Dr. Keith Bowman for their
enthusiastic support and guidance and acknowledge Intel Corporation for the funding
which supported in part, the research reported in this dissertation.
I benefited greatly from the stimulating environment at the Georgia Institute
of Technology, the California Institute of Technology and Rice University. I thank
everyone who granted me an opportunity to work at these institutions.
I thank my colleagues Dr. Pinar Korkmaz and Suresh Cheemalavagu for their
friendship and collaboration. A large part of this dissertation is synergistic with and
complements Pinar’s ideas and foundations. I thank Dr. Bilge Akgul, Jason George
and Bo Marr for their friendship and collaboration on many joint papers. I wish to
thank my friends Dr. Rodric Rabbah, Yogesh Chobe, Tushar Kumar and Dr. Bal-
asubramanian Seshasayee for the many stimulating conversations and eclectic ideas.
I thank my friends Romain Cledat, Jaswanth Sreeram and Rick Copeland for their
friendship. I thank my friends Karthik Sankaranarayanan, Easwaran Raman, Mad-
husudanan Seshadri and Niranjan Venkatraman for their friendship over the many
years, their standard of academic excellence and their personal integrity. I wish to
thank Kiruthika for her friendship, patience, interesting conversations and company.
Words cannot express my gratitude for the sacrifices, patience, trust and support
ofmymotherSaroja, myfatherChakrapani, mysisterManjulaandmybrother-in-law
Srivats.
v
TABLE OF CONTENTS
DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
I INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Reading Guide and Roadmap . . . . . . . . . . . . . . . . . . . . . 9
II HISTORICAL PERSPECTIVE AND BACKGROUND. . . . . . . . . . 11
2.1 Logic, Computing and Probabilistic Algorithms . . . . . . . . . . . 11
2.2 Frequentist Interpretation of Probability and
Probability in Logics . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.1 The Frequentist Interpretation of Probability . . . . . . . . 13
2.2.2 Probability in Logics . . . . . . . . . . . . . . . . . . . . . . 15
2.3 Thermodynamics and the Energy Cost of Computing . . . . . . . . 16
2.3.1 The Energy Cost of Probabilistic Computing . . . . . . . . 20
2.4 Current Technology Challenges . . . . . . . . . . . . . . . . . . . . 22
2.4.1 Probabilistic Complementary Metal Oxide Semiconductor
Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4.2 Techniques for Energy Efficient and Error-free Computing . 25
2.4.3 The Trade off between Energy, Error and Quality of Solution 26
2.4.4 Theoretical Approaches to Computing in the Presence of Faults 27
2.4.5 Practical Approaches to Computing In the Presence of Faults 28
III A PROBABILISTIC BOOLEAN LOGIC AND ITS MEANING . . . . . 30
3.1 Probabilistic Boolean Logic and Well-Formed Formulae . . . . . . . 31
3.1.1 Boolean Logic Preliminaries . . . . . . . . . . . . . . . . . . 32
3.1.2 The Operational Meaning of Probabilistic Boolean Operators 33
vi
3.1.3 Probabilistic Boolean Formulae and their Truth Tables . . . 34
3.2 The Event Set Semantics of Probabilistic Boolean Logic . . . . . . 35
3.2.1 A Frequentist View of Probabilistic Boolean Logic . . . . . 36
3.2.2 An Interpretation of a Probabilistic Boolean Formula for a
Fixed Assignment Through Event Sets . . . . . . . . . . . . 39
3.3 A Formal Model for Probabilistic Boolean Logic . . . . . . . . . . . 44
IV PROPERTIES OF PROBABILISTIC BOOLEAN LOGIC . . . . . . . . 49
4.1 Classical Identities That are Preserved . . . . . . . . . . . . . . . . 49
4.2 Identities that are not Preserved . . . . . . . . . . . . . . . . . . . 50
4.2.1 Associativity . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2.2 Distributivity . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3 Degree of Non-Associativity . . . . . . . . . . . . . . . . . . . . . . 52
4.3.1 Balanced Binary and Linear Probabilistic Boolean Formula 53
4.3.2 An Upper bound on the Probability of Unsatisfiability of a
Balanced Binary Probabilistic Boolean Formula . . . . . . . 55
4.3.3 A Lower bound on the Probability of Unsatisfiability of a
Linear Probabilistic Boolean Formula . . . . . . . . . . . . . 58
4.3.4 The Degree of Non-associativity of Probabilistic Boolean Logic 60
V PROBABILISTIC BOOLEAN LOGIC AND MODELS OF COMPUTING 62
5.0.5 Thermodynamic Separation of Implicitly and Explicitly
Probabilistic Gates and The Circuit Model of Computation 62
5.0.6 Energy Considerations For Realizing Probabilistic and
Randomized Boolean Operators . . . . . . . . . . . . . . . . 66
5.0.7 Extending to Computational Model with State . . . . . . . 66
VI PROBABILISTIC ARCHITECTURES . . . . . . . . . . . . . . . . . . 69
6.1 Probabilistic System on a Chip Architectures . . . . . . . . . . . . 70
6.2 Energy and Performance Metrics for Probabilistic System on a Chip
Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.2.1 Performance and Energy Modeling of Probabilistic System
on a Chip Architectures . . . . . . . . . . . . . . . . . . . . 73
6.3 A Co-design framework . . . . . . . . . . . . . . . . . . . . . . . . 74
vii
6.3.1 A Brief Description of the Applications of Interest . . . . . 75
6.3.2 Application Level Gains . . . . . . . . . . . . . . . . . . . . 77
6.3.3 An Analysis of Gains . . . . . . . . . . . . . . . . . . . . . . 78
6.3.4 Implementation Independent Characteristics Influencing
Co-design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.3.5 Implementation Dependent Characteristics Influencing
Co-design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.4 A Comparison of Implicitly Probabilistic and Explicitly Random
Circuits in the System on a Chip Context . . . . . . . . . . . . . . 88
6.5 The Suite of Applications, Partitioning, Optimization and psoc
Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.6 Some Practical Considerations . . . . . . . . . . . . . . . . . . . . 95
6.6.1 Reducing Multiple Voltage Levels . . . . . . . . . . . . . . . 95
6.6.2 Quality of Randomization . . . . . . . . . . . . . . . . . . . 98
VII PROBABILISTIC ARITHMETIC . . . . . . . . . . . . . . . . . . . . . 100
7.1 Abstracting a Mathematical Model . . . . . . . . . . . . . . . . . . 101
7.1.1 A Mathematical Model for Deterministic Addition . . . . . 102
7.1.2 A Mathematical Model for Probabilistic Addition . . . . . . 105
7.2 Cost and Magnitude of Error of Probabilistic Addition . . . . . . . 106
7.2.1 Error Magnitude of Probabilistic Addition . . . . . . . . . . 107
7.3 Relative Magnitude of Error . . . . . . . . . . . . . . . . . . . . . . 110
7.4 Some Practical Considerations . . . . . . . . . . . . . . . . . . . . 115
7.4.1 Truncation in Probabilistic Arithmetic . . . . . . . . . . . . 117
7.5 Case Study of Probabilistic Arithmetic in Digital Signal Processing 119
VIII REMARKS AND FUTURE DIRECTIONS . . . . . . . . . . . . . . . . 121
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
viii
LIST OF TABLES
1 Identities of pbl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2 The algorithms of interest, applications based on these algorithms and
the core probabilistic step for each algorithm . . . . . . . . . . . . . . 76
3 Maximum and minimum epp gains of pcmos over the baseline im-
plementation where the implementation I has a StrongARM sa-1100
host and a pcmos based co-processor . . . . . . . . . . . . . . . . . . 78
4 Application level flux, maximum and minimum epp gains of pcmos
over the baseline implementation where the implementation I has a
StrongARM sa-1100 host and a pcmos based co-processor . . . . . . 79
5 The epp gain of pcmos over sa-1100 and over cmos for the core
probabilistic step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6 The probability parameters required by the application, and the com-
position tree for generating them using two voltage levels . . . . . . . 98
ix
LIST OF FIGURES
1 Following Palem [139], (a) deterministic one bit switching functions
and (b) their probabilistic counterparts with probability parameter
(probability of correctness) p . . . . . . . . . . . . . . . . . . . . . . . 21
2 The models from [2] of (a) a pcmos switch and (b) representation of
digital values 0 and 1 and the probability of error for a pcmos switch 24
3 (a) The relationship between energy per switching step and probability
ofcorrectnessforanandgateandaninverterat90nmtechnology, and
(b) the same relationship at 65nm technology (from [101]) . . . . . . 25
4 A Boolean truth table for the formula (((x∧y)∨(x∧z))∨(y ∧z)) . 33
5 A probabilistic Boolean truth table for the pbf (((x∧ y)∨ (x∧ z))∨
1 1 1 1
(y ∧ z)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3/4
6 (a) A frequentist interpretation of a sentence (1 ∨ 0) ==r 1 in pbl
3
4
throughaninfinitesequenceofeventsand(b)asuccinctrepresentation
of this sequence as an event set . . . . . . . . . . . . . . . . . . . . . 36
3 1
7 (a) The event set for the valid sentence (0∨ 0) ==4 0 and (0∨ 0) ==4 1
3 3
4 4
(b) three valid sentences and their event sets for the three remaining
assignments to (x∨ y) . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3
4
r0 r00
8 (a)EventsetE of(1∨ 0) == 1(b)eventsetE of(1) === 1(c)
S0,I0 3 S00,I00
4
˜ r
E = E ×E (d)constructingtheeventsetfor((1∨ 0)∨ 1) == 1
S0,I0 S00,I00 3 5
4 6
˜
from E. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
9 (a) A linear pbf over n variables in syntactic form (b) as a tree struc-
ture illustrating the linear form (c) a reassociation of the same pbf (d)
its balanced binary representation in tree form . . . . . . . . . . . . 54
10 (a) A transition function encoded as a transition truth table (b) a
probabilistic circuit which computes this transition truth table (c) an
equivalent randomized Boolean circuit which computes the transition
truth table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
11 The canonical psoc architecture . . . . . . . . . . . . . . . . . . . . . 71
12 The conventional implementation alternatives for an application . . . 72
13 The energy and performance modeling methodology for each compo-
nent of a psoc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
14 Gain and flux for Bayesian network of various sizes . . . . . . . . . . 80
x