Table Of Content

Decrypted Secrets Friedrich L. Bauer Decrypted Secrets Methods and Maxims of Cryptology Fourth, Revised and Extended Edition With 191 Figures, 29 Tables, and 16 Color Plates 123 Dr.rer.nat.Dr.èssc.h.c.Dr.rer.nat.h.c.mult.FriedrichL.Bauer ProfessorEmeritusofMathematicsandComputerScience MunichInstituteofTechnology DepartmentofComputerScience Boltzmannstr.3 85748Garching,Germany ACMComputingClassification(1998):E.3,D.4.6,K.6.5,E.4 MathematicsSubjectClassification(1991):94A60,68P25 LibraryofCongressControlNumber:2006933429 ISBN-10 3-540-24502-2 SpringerBerlinHeidelbergNewYork ISBN-13 978-3-540-24502-5 SpringerBerlinHeidelbergNewYork ISBN 3-540-42674-4 3rded.SpringerBerlinHeidelbergNewYork Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerial isconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broad- casting,reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationof thispublicationorpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLaw ofSeptember9,1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfrom Springer.ViolationsareliableforprosecutionundertheGermanCopyrightLaw. SpringerisapartofSpringerScience+BusinessMedia springer.com ©Springer-VerlagBerlinHeidelberg1997,2000,2002,2007 Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnot imply, even in the absence of a specific statement, thatsuch names are exempt from the relevant protectivelawsandregulationsandthereforefreeforgeneraluse. CoverDesign:Design&ConceptE.Smejkal,Heidelberg ColorPhotos:ReinhardKrause,DeutschesMuseumMünchen Typesetting:BytheauthorinTEX Production:LE-TEX,Jelonek,Schmidt&VöcklerGbR,Leipzig Printedonacid-freepaper 33/3100YL 543210 Preface Towards the end of the 1960s, under the influence of the rapid development of microelectronics, electromechanical cryptological machines began to be replaced by electronic data encryption devices using large-scale integrated circuits. This promised more secure encryption at lower prices. Then, in 1976, DiffieandHellmanopenedupthenewcryptologicalfieldofpublic-key systems. Cryptography, hithertocloakedinobscurity, wasemergingintothe publicdomain. Additionally,ENIGMArevelationsawokethepublicinterest. Computer science was a flourishing new field, too, and computer scientists became interested in several aspects of cryptology. But many of them were notwellenoughinformedaboutthecenturies-longhistoryofcryptologyand the high level it had attained. I saw some people starting to reinvent the wheel, and others who had an incredibly naive belief in safe encryption, and I became worried about the commercial and scientific development of professional cryptology among computer scientists and about the unstable situation with respect to official security services. ThispromptedmetoofferlecturesonthissubjectattheMunichInstituteof Technology. The first series of lectures in the winter term 1977/78, backed by the comprehensive and reliable book The Codebreakers (1967) by David Kahn, was held under the code name ‘Special Problems of Information Theory’ and therefore attracted neither too many students nor too many suspicious people from outside the university. Next time, in the summer term of 1981, my lectures on the subject were announced under the open title ‘Cryptology’. This was seemingly the first publicly announced lecture series under this title at a German, if not indeed a Continental European, university. Theseriesoflectureswasrepeatedafewtimes,andin1986/87lecturenotes wereprintedwhichfinallydevelopedintoPartIofthisbook. Activeinterest on the side of the students led to a seminar on cryptanalytic methods in the summer term of 1988, from which Part II of the present book originated. The1993firstedition(inGerman)ofmybookKryptologie,althoughwritten mainly for computer science students, found lively interest also outside the field. It was reviewed favorably by some leading science journalists, and the publisher followed the study book edition with a 1995 hardcover edition under the title Entzifferte Geheimnisse [Decrypted Secrets], which gave me the opportunity to round out some subjects. Reviews in American journals recommendedalsoanEnglishversion,whichledin1997tothepresentbook. It has become customary among cryptologists to explain how they became acquainted with the field. In my case, this was independent of the Second World War. In fact, I was never a member of any official service—and I VI Preface consider this my greatest advantage, since I am not bound by any pledge of secrecy. Ontheotherhand,keepingeyesandearsopenandreadingbetween the lines, I learned a lot from conversations (where my scientific metier was a good starting point), although I never know exactly whether I am allowed to know what I happen to know. Itallstartedin1951,whenItoldmyformerprofessor of formal logic at Munich University, Wilhelm Brit- zelmayr, of my invention of an error-correcting code forteletypelines1. Thiscausedhimtomakeawrong association, and he gave me a copy of Sacco’s book, whichhadjustappeared2. Iwaslucky, foritwasthe best book I could have encountered at that time— although I didn’t know that then. I devoured the book. Noticing this, my dear friend and colleague Paul August Mann, who was aware of my acquain- tance with Shannon’s redundancy-decreasing encod- LuigiSacco(1883–1970) ing, gave me a copy of the now-famous paper by Claude Shannon called Communication Theory of Secrecy Systems3 (which in those days as a Bell Systems Technical Report was almost unavailable in Germany). I was fascinated by this background to Shannon’s information theory, which I was already familiar with. This imprinted my interest in cryptology as a subfield of coding theory and formal languages theory, fields that held my academic interest for many years to come. Strange accidents—or maybe sharper observation—then brought me into contact with more and more people once close to cryptology, starting with Willi Jensen (Flensburg) in 1955, Karl Stein (Munich) in 1955, Hans Rohr- bach,mycolleagueatMainzUniversity,in1959,aswellasHelmutGrunsky, Gisbert Hasenja¨ger, and Ernst Witt. In 1957, I became acquainted with Erich Hu¨ttenhain (Bad Godesberg), but our discussions on the suitability of certain computers for cryptological work were in the circumstances limited by certain restrictions. Among the American and British colleagues in nu- merical analysis and computer science I had closer contact with, some had been involved with cryptology in the Second World War; but no one spoke aboutthat,particularlynotbefore1974,theyearwhenWinterbotham’sbook TheUltraSecretappeared. In1976,IheardB.RandallandI.J.Goodreveal somedetailsabouttheColossiinasymposiuminLosAlamos. Asascience- oriented civilian member of the cryptology academia, my interest in cryptology was then and still is centered on computerized cryptanalysis. Other aspectsofsignalsintelligence(‘SIGINT’),forexample,trafficanalysisanddi- rectionfinding,arebeyondthescopeofthisbook;thesameholdsforphysical devices that screen electromechanical radiation emitted by cipher machines. 1 DBPNo.892767,applicationdateJanuary21,1951. 2 GénéralLuigiSacco,ManueldeCryptographie. Payot,Paris1951. 3 BellSystemsTechnicalJournal28,Oct.1949,pp.656–715. Preface VII Cryptology is a discipline with an international touch and a particular ter- minology. It may therefore be helpful sometimes to give in this book some explanations of terms that originated in a language other than English. Thefirstpartofthisbookpresentscryptographicmethods. Thesecondpart coverscryptanalysis, aboveallthefactsthatareimportantforjudgingcryp- tographic methods and for saving the user from unexpected pitfalls. This follows from Kerckhoffs’ maxim: Only a cryptanalyst can judge the secu- rityofacryptosystem. Atheoreticalcourseoncryptographicmethodsalone seems to me to be bloodless. But a course on cryptanalysis is problematic: Either it is not conclusive enough, in which case it is useless, or it is conclusive,buttouchesasensitivearea. Thereislittleclearanceinbetween. Ihave triedtocoveratleastalltheessentialfactsthatareintheopenliteratureor can be deduced from it. No censorship took place. Certaindifficultiesarecausedbythefactthatgovernmentalrestrictionsdur- ing and after World War II, such as the ‘need to know’ rule and other gim- micks,misledevenpeoplewhohadbeenclosetothecentersofcryptanalysis. Examples include the concept of Banburismus and the concept of a ‘cilli’. ThewordBanburismus—thenamewascoinedinBritain—wasmentionedin 1985 by Deavours and Kruh in their book, but the method was only vaguely described. Likewise, the description Kahn gave in 1991 in his book is rather incomplete.On the other hand, in Kozaczuk’s book of 1979 (English edition of 1984),Rejewski gave a description of Ro´z˙ycki’s‘clock method’, which turned out to be the same—but most of the readers could not know of this connection. Then, in 1993, while giving a few more details on the method, Good (in ‘Codebreakers’) confirmed that “Banburism was an elaboration of...the clock method...[of]...Ro´z˙ycki”. He also wrote that this elaboration was ‘invented at least mainly by Turing’, and referred to a sequential Bayesianprocessasthe“methodofscoring”. Forlackofdeclassifiedconcrete examples, the expositioninSect.19.4.2 of the present book, based on the recently published postwar notes of Alexander and of Mahon and articles by ErskineandbyNoskwithintherecentbookActionThisDay, cannotyetbe a fully satisfactory one. And as to cillies, even Gordon Welchman admitted that he had misinterpreted the origin of the word, thinking of ‘silly’. Other publicationsgaveotherspeculations, seeSect.19.7, fn.29. RalphErskine, in Action This Day, based on the recently declassified ‘Cryptanalytic Report on the Yellow Machine’, 71-4 (NACP HCC Box1009, Nr.3175), gives the following summary of the method: ‘Discovered by Dilly Knox in late January 1940, cillies reduced enormously the work involved in using the Zygalski sheets, and after 1 May, when the Zygalskisheetsbecameuseless,theybecameavitalpartofbreakingEnigma by hand during most of 1940. They were still valuable in 1943. Cillies resulted from a combination of two different mistakes in a multi-part message by some Enigma operators. The first was their practice of leaving therotorsuntouchedwhentheyreachedtheendofsomepartofthemessage. Sincethelettercountofeachmessagepartwasincludedinthepreamble,the messagekeyoftheprecedingpartcouldbecalculatedwithinfinelimits. The seconderrorwastheuseofnon-randommessagekeys—stereotypedkeyboard touches and 3-letter-acronyms. In combination, and in conjunction with the differentturnoverpointsofrotorsItoV,theyallowedonetodeterminewhich rotors could, and which could not, be in any given position in the machine.’ Although Banburismus and cillies were highly important in the war, it is hard to understand why Derek Taunt in 1993 was prevented by the British censor from telling the true story about cillies. Possibly, the same happened to Jack Good about Banburismus. *** My intellectual delight in cryptology found an application in the collection ‘Informatik’ of the Deutsches Museum in Munich which I built up in 1984 –1988, where there is a section on cryptological devices and machines. My thanksgototheDeutschesMuseumforprovidingcolorplatesofsomeofthe pieces on exhibit there. And thanks go to my former students and co-workers in Munich, Manfred Broy,HerbertEhler,andAntonGeroldforcontinuingsupportovertheyears, moreover to Hugh Casement for linguistic titbits, and to my late brother- in-law Alston S.Householder for enlightenment on my English. Karl Stein andOttoLeiberichgavemedetailsontheENIGMAstory,andIhadfruitful discussionsandexchangesofletterswithRalphErskine,HeinzUlbricht,Tony Sale,FrodeWeierud,Kjell-OveWidman,OttoJ.Horak,GilbertBloch,Arne Fransén, and Fritz-Rudolf Gu¨ntsch. Great help was given to me by Kirk H.Kirchhofer from Crypto AG, Zug (Switzerland). Hildegard Bauer-Vogg suppliedtranslationsofdifficultLatintexts,MartinBauer,UlrichBauerand Bernhard Bauer made calculations and drawings. Thanks go to all of them. The English version was greatly improved by J.Andrew Ross, with whom working was a pleasure. In particular, my sincere thanks go to David Kahn who encouraged me (“The book is an excellent one and deserves the widest circulation”)andmadequiteanumberofproposalsforimprovementsofthe text. For the present edition, additional material that has been made public recently has been included, among others on Bletchley Park, the British at- tackonTunny,ColossusandMaxNewman’spioneeringwork. Moreover,my particular thanks go to Ralph Erskine who indefatigably provided me with a lot of additional information and checked some of the dates and wordings. In this respect, my thanks also go to Jack Copeland, Heinz Ulbricht, and Augusto Buonafalce. Finally, I have to thank once more Hans Wo¨ssner for a well functioning cooperation of long standing, and the new copy editor Ronan Nugent for very careful work. The publisher is to be thanked for the finepresentationofthebook. AndIshallbegratefultoreaderswhoarekind enough to let me know of errors and omissions. Grafrath, Spring 2006 F.L.Bauer Contents Part I: Cryptography—The People ................................. 1 1 Introductory Synopsis ...........................................9 1.1 Cryptography and Steganography...................................9 1.2 Semagrams....................................................... 10 1.3 Open Code: Masking............................................. 13 1.4 Cues.............................................................. 17 1.5 Open Code: Veiling by Nulls...................................... 19 1.6 Open Code: Veiling by Grilles .................................... 23 1.7 Classification of Cryptographic Methods.......................... 24 2 Aims and Methods of Cryptography......................... 26 2.1 The Nature of Cryptography...................................... 26 2.2 Encryption....................................................... 32 2.3 Cryptosystems.................................................... 34 2.4 Polyphony........................................................ 36 2.5 Character Sets.................................................... 39 2.6 Keys............................................................. 41 3 Encryption Steps: Simple Substitution...................... 44 3.1 Case V(1) −−−(cid:1)W (Unipartite Simple Substitutions) ............. 44 3.2 Special Case V ≺−−−−(cid:1)V (Permutations).......................... 46 3.3 Case V(1)−−−(cid:1)Wm (Multipartite Simple Substitutions)........... 53 3.4 The General Case V(1) −−−(cid:1)W(m), Straddling ................... 55 4 Encryption Steps: Polygraphic Substitution and Coding . 58 4.1 Case V2 −−−(cid:1) W(m) (Digraphic Substitutions) .................. 58 4.2 Special Cases of Playfair and Delastelle: Tomographic Methods.... 64 4.3 Case V3 −−−(cid:1) W(m) (Trigraphic Substitutions) .................. 68 4.4 The General Case V(n) −−−(cid:1)W(m) : Codes ...................... 68 5 Encryption Steps: Linear Substitution........................80 5.1 Self-reciprocal Linear Substitutions................................ 82 5.2 Homogeneous Linear Substitutions................................ 82 5.3 Binary Linear Substitutions....................................... 86 5.4 General Linear Substitutions...................................... 86 5.5 Decomposed Linear Substitutions................................. 87