Cipher alphabets

160 Pages·1998·7.18 MB·English

Acquisitions and Acquisitions et
Bibliographie Services services bibliographiques
395 Wellington Street 395, rue Wellington
Ottawa ON K1A ON4 Ottawa ON K1A ON4
Canada Canada
Yow fi& Votre rdldmnce
Our file Notre rdiérenu)
The author has granted a non- L'auteur a accordé une licence non
exclusive licence allowing the exclusive permettant à la
National Library of Canada to Bibliothèque nationale du Canada de
reproduce, loan, distribute or sell reproduire, prêter, distribuer ou
copies of this thesis in microform, vendre des copies de cette thèse sous
paper or electronic formats. la forme de microfiche/film, de
reproduction sur papier ou sur format
électronique.
The author retains ownership of the L'auteur conserve la propriété du
copyright in this thesis. Neither the droit d'auteur qui protège cette thèse.
thesis nor substantial extracts h m i t Ni la thèse ni des extraits substantiels
may be printed or otherwise de celle-ci ne doivent être imprimés
reproduced without the author's ou autrement reproduits sans son
permission. autorisation.
Abstract
Solving a cryptogram is non-trivial although many people can generally learn to
do it with logical deduction aided by the ability to readily recognize words in the
language. Despite these and other qualities possessed by the human solver, only the
presence of regularities in the enciphered message or cryptogram makes cryptanalysis
possible.
In t his t hesis an Automated CryptanaZysis Emzllator ( ACE) was designed, devel-
oped and implemented for the cryptanalysis of classical cryptograms (Monoalphabetic
und polyalphabetic Substitution ciphers). Three principal algorithms constitute the
backbone of the system. The first attempts a reconstruction of the Specific Key used
to generate the cipher message. In the second algorithm, the technique of singular
value decomposition is used to isolate the vowels and consonants in the coded mes-
sage. In the third algorithm the problem of breaking the coded message is represented
as a probabilistic labeling problem. Every cipher letter is assigned probabilities for
representing plaintext letters. These probabilities are updated in parallel for al1 ci-
pher let ters using joint letter probabilities. Iterating the updat ing scheme results in
improved estimates that finally lead to solving the cryptogram.
The emulating aspect of the cryptanalysis process via a graphical interface was
accomplished as planned, without any major problems. Six ciphers of different lengths
were subjected to the cryptanalysis processes of ACE. The ciphertext were 335, 383,
467, 531, 660 and 1140 characters long. One hundred percent cipher reconstruction
was achieved only when the specific cipher key was found. The technique of singular
value decomposition and the probabilistic relaxation algorithm performed well in soine
test cases and very poorly in others.
Contents
A bs t ract
List of Tables
List of Figures viii
Acknowledgments x
1 Introduction 1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Background 1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Motivation 4
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Thesis Overview 4
2 Literature Review 5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1 Data Security 5
. . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Cryptographic Systems 6
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Number Theory 7
. . . . . . . . . . . . . . . . . . . . . . .
2.3.1 Modular Arithmetic 8
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 Types of Ciphers 9
. . . . . . . . . . . . . . . . . . . . . . .
2.4.1 Substitution Ciphers 9
. . . . . . . . . . . . . . . . . . . . . .
2.4.2 Transposition Ciphers 10
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 Types of Alphabets 10
. . . . . . . . . . . . . . . . . . . .
2.5.1 Normal or Plain alphabet 10
. . . . . . . . . . . . . . . . . . . . . . . . . .
2.5.2 Cipher alphabet 11
. . . . . . . . . . . . . . . . . . . .
2.5.3 Types of cipher alphabets 11
. . . . . . . . . . . . . . . . . . . . . . . . .
2.6 Monoalphabetic ciphers 12
. . . . . . . . . . . . . . . . . . . . . . . . . .
2.7 Polyalphabetic ciphers 13
. . . . . . . . . . . . . . .
2.8 The Singular Value Decomposition (SVD) 17
3 The Cryptanalysis Process 19
. . . . . . . . . . . . . . . . . . . . . . .
3.1 Determination of Language 20
. . . . . . . . . . . . . . . . . .
3.2 Determination of the General system 21
. . . . .
3.2.1 Monoalphabetic and Polyalphabetic characterization 23
. . . . . . . . . . . . . . . . . . .
3.3 Reconstruction of the Specific Key 25
. . . . . . . . . . . . . . .
3.4 Reconstruction of the plaintext equivalent 26
4 Implementation of the Algorithms 27
. . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1 The Data Structures 27
. . . . . . . . . . . . . . . . . . . . . .
4.2 Cipher Classification Criteria 29
. . . . . . . . . . . . . . . . . . . .
4.3 Poly to monoalphabetic reduction 31
4.3.1 Determination of cipher alphabets . . . . . . . . . . . . . . . . 31
4.3.2 The Coefficient of Variation (CV) . . . . . . . . . . . . . . . . 31
4.3.3 Matching alphabet distributions . . . . . . . . . . . . . . . . . 35
4.4 Reconstruction Algorithm 1: The SVD
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Routine 36
4.4.1 General observations . . . . . . . . . . . . . . . . . . . . . . . 36
. . . . . . . . . . . . . . . . . . . . . . . . .
4.4.2 The SVD Routine 37
4.4.3 Classification of Vowels and Consonants . . . . . . . . . . . . 37
4.4.4 Identities of Classified Letters . . . . . . . . . . . . . . . . . . 38
. . . . . . . . . . . . . . . . . . . . . . .
4.4.5 Experimental Results 40
4.5 Reconstruction Algorithm 2: The Relaxation Algorithm . . . . . . . . 46
. . . . . . . . . . . . . . . . . . .
4.6 Probabilistic Relaxation Algorithm 47
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6.1 Assumptions 48
. . . . . . . . . . . . . . . . . . .
4.6.2 Initialization of Probabilities 48
. . . . . . . . . . . . . . . . . . . . . .
4.6.3 Updating Probabilities 51
. . . . . . . . . . . . . . . .
4.6.4 The Trigram Language Statistics 56
. . . . . . . . . . . . . . . . . . . . . . .
4.6.5 Experimental Results 57
5 Development and Testing of ACE 59
. . . . . . . . . . . . . . . . . . . . . . . .
5.1 Visual Basic: An Overview 60
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 The Design Process 62
. . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 The Program Structure 63
. . . . . . . . . . . . . . . . . . . . . . . .
5.3.1 The Entry Section 63
. . . . . . . . . . . . . . . . . . . .
5.3.2 The Development Section 64
. . . . . . . . . . . . . . .
5.3.3 The On-line Documentation Section 74
. . . . . . . . . . . . . . . . . . . . . . . .
5.4 The User Interface Design 76
. . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Testing and Debugging 78
. . . . . . . . . . . . . . . . . . . . .
5.5.1 Code writing and testing 78
. . . . . . . . . . . . . . . . . . . . . . . .
5.5.2 Application testing 79
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.6 Results 81
6 Conclusion and Future Work 83
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1 Conclusion 83
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 FutureWork 84
Appendices
1 Monogram statistics
II Adjusted (SBJ ) trigram statistics
.
III Experimental runô of ACE
List of Tables
. . . . . . . . . . . . .
Vigenere square with direct standard alphabet 16
. . . . . . . . . . . . . . .
A synoptic tree for classical cipher analysis 22
Expected ICvalues and corresponding cipher alphabets l . . . . . . . 25
Digram frequency matrix of the first 1. 140 characters in Chapter 1 of
1431 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
(xi)
(Uii) and singular vectors and their corresponding let ter frequencies . 42
(Ui2) and (K2) singular vectors . Alternating signs identifies vowels(V).
. . . . . . . . . . . . . . . . . . . . .
consonants(C) and don't care(X)
Extrapolated frequencies F(a)a nd discrimination factors fa for cipher
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vowels
. . . . . . . . . . . .
Comparable decision factors f+ for cipher vowels
. . . . . . . .
Decision table for plaintext equivalents of cipher vowels
Extrapolated frequencies F(a)a nd discrimination factors fa for cipher
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
consonants
Comparable decision factors fa8 for cipher consonants . . . . . . . . .
Decision table for plaintext equivalents of cipher consonants . . . . . .
4.10 A summary of the resulting output tables . . . . . . . . . . . . . . . .
4.11 Contents of array P-prime[] after the fourth probabilistic relaxation
iteration . The * " indicates zero occurrence of a cipher symbol . . .
"
5.1 Cipher reconstruction times (CRT) for ACE experimental runs us-
ing the SVD algorithm using machinel(1BM Pl66 model with 32MB
RAM) and machinel(1BM PSI2 model 90 XP 486). . . . . . . . . . .
5.2 Cipher reconstruction times (CRT) for ACE experimental runs using
the probabilistic relaxation algorithm using machine1 (IBM P 166 model
with 32MB RAM) and machine2(IBM PSI2 model 90 XP 486). . . .
1.1 The frequency counts of English letters in a text of 5,000,000 characters
11.1 Adjusted (SBJ) trigrams statistic. . . . . . . . . . . . . . . . . . . . .
vii
List of Figures
. . . . . . . . . . . . . . . . . . . . .
A classical information channel
. . . . . . . . . . . . . .
Outline of a complete cryptographic system
. . . . . . . . . . . . . . . . . . .
The principle of modular arithmetic
A flow chart summarizing the cipher classification algorithm . The en-
. . . . . . . . .
closed region constitutes the function GeneralsystemO
Algorithm for computing Coefficient of Variation (CV). . . . . . . . .
. . . . . . . . . . . . . . . .
Algorithm for selecting the truc key length
The binomial distribution algorit hm . . . . . . . . . . . . . . . . . . .
Algorithm to compute likelihood vector . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
The product updating rule algorithm
Program structure of the entry section . . . . . . . . . . . . . . . . . .
Program structure of the encryption subsection of the development
section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
Program structure of analysis subsection 1 of the development section
.
Program structure of the analysis subsection 2 of the development section
Program structure of the analysis subsection 3 of the development section .
ACE screen(a) shows the file menu of the entry section . . . . . . . . .
ACE screen(b) shows the edit menu of the entry section . . . . . . . .
ACE screen(c) shows the setup menu of the entry section . . . . . . .
ACE screen(d) shows the monoalphabetic systems menu of the encryp-
tion subsection of the development section . . . . . . . . . . . . . . . .
...
Vlll
5.10 ACE screen(e) shows the polyalphabetic systems menu of the encryp-
. . . . . . . . . . . . . . .
tion subsection of the development section. 72
5.11 ACE screen(f) shows the analysis menu of the analysis subsection of
. . . . . . . . . . . . . . . . . . . . . . . . .
the development section. 72
5.12 ACE screen(g) shows the solution menu of the analysis subsection of
. . . . . . . . . . . . . . . . . . . . . . . . .
the development section. 73
5.13 ACE screen(h) shows the help menu of the on-line documentation section. 74
5.14 Flow of work in the conception and development of the on-line docu-
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
mentation section. 75
5.15 Enclosed regions represent the scope of application of ACE. . . . . . SO