World War I and the Invention of American Intelligence, 1878

348 Pages·2017·12.74 MB·English

World War I and the Invention of
American Intelligence, 1878 - 1918
by
Mark Stout
Submitted in accordance with the requirement for the degree of Doctor of Philosophy.
The University of Leeds
School of History
Leeds, United Kingdom
June, 2010
The candidate confirms that the work submitted is his own and that appropriate credit
has been given where reference has been made to the work of others.
This copy has been supplied on the understanding that it is copyright material and that
no quotation from the thesis may be published without proper acknowledgement.
Acknowledgements
PhD. dissertations often blight the lives of their authors. Such was not the
case here. Many people deserve credit for this fact. First among them is Professor
David Alvarez for giving me the original idea. I would never have thought of it.
Thank you. Secondly, of course, thanks to my adviser Professor John Gooch. It was
an honor and a privilege to work under his tutelage. It has also been delightful to
study at Leeds where I lived as an 8 year old while my father was a visiting member
of the faculty. Thanks are also due to Dr. Wick Murray for repeatedly telling me to
hurry up and for providing various other forms of help.
Thank you also to Mark Whisler for pointing me toward important resources
when I was just starting on this project. At various times Michael Warner, John Fox,
and John Schindler also did the same. I further owe thanks to John Schindler for
introducing me to Professor John Ferris. Schindler, Ferris and I, in various
combinations, had several stimulating and useful discussions of intelligence, typically
over beer. Thanks are due to Louis Sadler for helping me sort out some obscure
points pertaining to the 1916 Punitive Expedition. In a variety of other ways
Elizabeth Anne Nathan, Paul Manning, Jonathan Winkler, Shawna Cuan, and Jessica
Huckabey provided thought-provoking discussions, suggestions for further reading, or
encouragement and moral support.
Kevin Woods, Jim Lacey, Sharon Moore, and Alec Wahlman were wonderful
study partners, as was Maegen Gandy in the critical final phase.
Thanks are also due to the Woodrow Wilson International Center for Scholars
and in particularly Christian Ostermann and Mircea Munteanu of the History and
Public Policy Project there for naming me a Public Policy Fellow for the last six
11
months of2009. The Wilson Center was the perfect quiet refuge in which to push this
work over the finish line.
Innumerable archivists and librarians helped in innumerable ways, which
should come as no surprise to any historian reading this work. Among them are were
Kathy Buker of the Combined Arms Research Library at Ft. Leavenworth, Evelyn
Cherpak of the library at the Naval War College, David Keough of the US Army
Military History Institute at Carlisle, Pennsylvania, and several archivists at the
National Archives in College Park, Maryland.
I would also very much like to thank Karl Lowe for his encouragement and
support throughout this process. I have never worked for a finer man.
At the beginning of this preface I said that the writing of this dissertation did
not blight my life. Sadly, I fear that it did blight the life of my wife, Pam.
Accordingly, I dedicate this work to her. Perhaps it will be useful as fire-starting
material on a cold winter evening.
11l
Abstract
Intelligence changes as the nature of war changes. From the late 1870s, the
United States military, as part of a broader reform process, began learning about
intelligence in part from experience but more importantly by observing the practices
of the great powers of Europe. The period of American involvement in World War I
saw a rapid acceleration of this development, with the United States continuing to
learn from the United Kingdom and France. The war also saw intelligence spreading
into fields that it had seldom if ever entered in the American experience. During the
nineteen months of American belligerency American intelligence agencies, notably
the War Department's Military Intelligence Division and the Navy Department's
Office of Naval Intelligence expanded greatly. In addition, the services started to
adopt high technology tools such as aerial photography and signals intelligence.
These new tools required the admission into the military departments and services of
esoteric specialists who did not fit previous military stereotypes. The war also
occasioned a vast expansion of domestic surveillance and intelligence, a result of the
idea that the World War was a struggle not only of militaries but of entire societies.
Espionage, too, grew in extent and sophistication and the moral stigma associated
with it began to erode. Overseas, the American Expeditionary Force (AEF) in France
grew its own large intelligence staff. All of these measures allowed General John J.
Pershing, the AEF's commander, as well has other American leaders to be better
informed than they had ever been during previous wars.
IV
Table of Contents
Acknowledgements ........................................................................................................ i
Abstract. ........................................................................................................................ iii
Introduction .................................................................................................................. vi
Note on Agencies and Organizations ......................................................................... xxi
Chapter 1 To the Eve ofWar. ........................................................................................ 1
Chapter 2 Technology Reinvents Intelligence ............................................................ 57
Chapter 3 A New Kind of War: Intelligence at Home .............................................. 115
Chapter 4 Human Intelligence: From Scouting to Stealing Secrets .......................... 161
Chapter 5 Intelligence Support for Commanders .................................................... 2044
Conclusion ............................................................................................................. 26969
Abbreviations .......................................................................................................... 2999
List of Sources and Bibliography .............................................................................. 301
v
LIST OF FIGURES
Figure 1 MID organization ........................................................................................ xxii
Figure 2 Select organizations of MID's Positive Branch ......................................... xxiii
Figure 3 Selected organization of MID's Negative Branch ......................................... vi
Figure 4 Selected AEF G-2 organizations as of November 1918 ................................ vi
Figure 5 Organization of ONI as of May 1918 ............................................................ vi
Figure 6 ONI Section A as of May 1918 ...................................................................... vi
Figure 3 Selected organization of MID's Negative Branch ......................................... vi
VI
Introduction
'To each epoch-its own wars', Lenin wrote in the margin of his copy of
Clausewitz.I In this, if little else, he was correct. It is now axiomatic that new
economic or societal structures, cultural changes, and new technologies all lead to
new forms of war. Less attended to, however, is the fact that new forms of war lead
to new forms of intelligence. Though World War I had some similarities to wars that
had preceded it, notably the American Civil War and the Russo-Japanese War, the
feeling at the time was that it was an utterly new type of war and indeed there is much
justification for that feeling. Scholars have already pointed out the role of the Great
War in reshaping the intelligence practices of the belligerents. One scholar-
practitioner has observed that 'total war needed total intelligence; foreign military
power had corne to depend on factors of industrial capacity, demography and morale
which fell outside the analysis of normal military and naval intelligence,.2 By the
same token, as another scholar has observed, those countries which played only
marginal roles in the war lacked the motivation and the insight necessary to reform
3
their intelligence services. However, hitherto virtually all the scholarly attention
given to this subject has involved assessment of the intelligence services of
belligerents other than the United States.
This dissertation, then, will discuss how the United States expanded and
shaped its intelligence organizations and ideas to fit the new circumstances of World
War 1. It will demonstrate that World War I saw America's invention of modem
intelligence. Of course, 'invention' does not mean that the Americans were the first to
1 Azar Gat, A History of Military Thought: From the Enlightenment to the Cold War, (Oxford,
2001), p. 504.
Michael Herman, Intelligence Power in War and Peace, (Cambridge, 1996), p. 25.
2
3 Ken Kotani, Japanese Intelligence in World War II, (New York, 2009), pp. 8, 99.
Vll
invent intelligence. (The old joke, of course, is that intelligence is the second oldest
professionl As newcomers to this activity, the Americans both could and did profit
from the experience of others. Much of this work will therefore describe how and
what the Americans learned from the Germans, the French, and the British. However,
its central purpose is to demonstrate that the United States invented modem
intelligence for itself, to analyse how it did so, and to show what Americans thought
their invention was. The primary contention of this dissertation, then, will be that the
World War I period and the years immediately preceding it have a strong and hitherto
unrecognized claim to having given birth to modem American intelligence.
Merely by looking at the amount of history published about the various topics
it is easy to discern the emphasis that intelligence historians have placed on World
War II and the early Cold War at the expense of earlier periods. Countless volumes
about American intelligence during World War II and the early Cold War fill1ibrary
shelves while one has to search hard for anything written on intelligence during the
Great War. The dichotomy grows even sharper when one visits an American
s
bookstore. A recent bibliographical work on World War I discusses work done on
the intelligence activities of most of the major combatants but almost entirely neglects
American intelligence, mentioning only a journal article about the American
More seriously, intelligence is, indced, an ancient endeavour. Rosemary Sheldon has made
4
her career as a historian of intelligence in the classical world. See, for instance, her
Intelligence Activities in Ancient Rome: Trust in the Gods, But VerifY, (London, 2005) and
Operation Messiah: St. Paul, Roman Intelligence, and the Birth a/Christianity, (Portland,
2008). Ralph D. Sawyer's, The Tao a/Spycraft: Intelligence Theory and Practice in
Traditional China, (Boulder, 1998), is an encyclopaedic consideration of the topic in ancient
China. Primary source works from the ancient world that contain serious discussions of
intelligence include Sun Tzu's Art a/War and Kautilya's Arthashastra, a text on politics and
statecraft in India that is often compared to Machiavelli's The Prince.
S J. Ransom Clark, a former senior CIA official turncd academic, maintains a remarkably
thorough and discerning bibliography of intelligence literature:
http://intellit.muskingum.edulmaintoc.html. The World War II section of the bibliography is
at http://intellit.muskingum.edulwwii_folder/wwiitoc.html. The majority of the rest of the
site is devoted to Cold War intelligence.
viii
Expeditionary Force's propaganda operations and another on American collection
operations against Japan from 1915 to 1935.6 The lion's share of what has been
written about the World War I period deals with the domestic surveillance operations
conducted by the Army and Navy and the use of the intelligence agencies to oppress
out groups.7 The standard published history of War Department intelligence to 1941
is an official history and is indispensable for its organizational information but largely
devoid of interpretive content. 8 In the 1970s, Marc Powe wrote a fine master's thesis,
later published, on the War Department's intelligence structures from 1885-1918.9
Jeffrey Dorwart has written an outstanding thin volume about the ONI covering its
establishment through World War I and emphasizing its reformist pedigree. His
second volume covered the period 1919 to 1945 and emphasized ONI's involvement
lo
in domestic intelligence. The only other general history of ONI covering the period
in question that has come out in the nearly thirty years since Dorwart's books were
published is a bland official history. I I
With regard to particular collection disciplines or operations, Terrence
Finnegan recently published an exemplary book comparing the aerial reconnaissance
6 Robin Higham with Dennis E. Showalter, cds., Researching World War I: A Handbook,
(Westport, 2003).
7 For examples, see Joan M. Jensen, Army Surveillance in America, 1775-1980, (New Haven,
1991) and Roy Talbert, Jr., Negative Intelligence: The Army and the American Left, 1917-
1941, (Jackson, 1991). Joseph W. Bendersky, The "Jewish Threat": Anti-Semitic Politics of
the U.S. Army, (New York, 2000), especially Chapters 1 and 2. See also Wray R. Johnson,
'Black American Radicalism and the First World War: The Secret Files of the Military
Intelligence Division', Armed Forces & Society, 26: 1, (Fall, 1999), pp. 27-54. Robert C.
Cottrell, 'Roger Nash Baldwin, the National Civil Liberties Bureau, and Intelligence during
World War 1', Historian, 60: 1, (Fall, 1997), pp. 87-106.
8 Bruce W. Bidwell, History of the Military Intelligence Division, Department oft he Army
General Staff: 1775-1941, (Frederick, 1986). There is some useful material in the similarly
non-interpretive John Patrick Finnegan, Military Intelligence, (Washington, 1998).
9 Marc B. Powe, The Emergence of the War Department Intelligence Agency, 1885-1918,
(Manhatten, 1975).
10 Jeffery M. Dorwart, The Office ofN aval Intelligence: The Birth ofA merica's First
Intelligence Agency, 1865-1918, (Annapolis, 1979) and ConjlictofDuty: The U.S. Navy's
Intelligence Dilemma, 1919-1945, (Annapolis, 1983).
us
II Wyman H. Packard, A Century of Naval Intelligence, (Washington, 1996).
IX
12
practices of the British, French and Americans during the war. Some fifty years ago
Sam Frank wrote a fine dissertation on American aerial 'observation' during the
war.13 David Kahn's massive The Codebreakers devotes parts of two chapters to
American signals intelligence (Sigint) during the war.I4 A few other scholars, most of
them officials of the National Security Agency, have written articles about tactical
intelligence operations, particularly in the realm of Sigint, during World War LIS
There has been significant coverage of the State Department's human intelligence
activities, but only one book, Rhodri Jeffreys Jones' American Espionage: From
Secret Service to CIA, takes a Washington-centric approach, and it is marred
throughout by an explicit tendency to portray intelligence activities as intrinsically
sinister.16 The rest, of varying quality, concentrates on operations in Russia or those
17
mounted by particular individuals. Charles Harris and Louis Sadler have written an
12 Terrance J. Finnegan, Shooting the Front: Allied Aerial Reconnaissance and Photographic
Interpretation on the Western Front-World War I, (Washington, 2006).
13 Sam Hager Frank, 'American Air Service Observation in World War I' , (unpublished
doctoral dissertation, University of Florida), 1961.
14 David Kahn, The Codebreakers: The Story ofS ecret Writing, (New Yark, 1996), chapters
11 and 12.
15 All of the following articles from NSA were obtained through the Freedom ofInformation
Act: [name redacted], 'The Origination and Evolution of Radio Traffic Analysis: The World
War I Era', Cryptologic Quarterly, 6:1, (1987), pp. 21-40. William. F. Friedman, 'A Brief
History of U.S. Cryptologic Operations 1917-1920', Cryptologic Spectrum, 6:2, (Spring,
1976), pp. 9-15. [redacted], 'COMINT and COMSEC: The Tactics of 1914-1918',
Cryptologic Spectrum, Summer, 1972, pp. 5-9. [redacted] 'COMINT and COM SEC: The
Tactics of 1914-1918-Part II', Cryptologic Spectrum, Fall, 1972, pp. 8-11. [redacted], 'The
Many Lives of Herbert O. Yardley', Cryptologic Spectrum, 11:4 (Fall, 1981), pp. 5-29. For
an article on this topic (apparently) not by an NSA veteran, see William A. Morgan, 'Invasion
on the Ether: Radio Intelligence at the Battle of St. Mihiel, September 1918', Military Affairs,
Vol. 51, No.2 (Apr., 1987), pp. 57-61.
16 Rhodri Jeffreys-Jones, American Espionage: From Secret Service to CIA, (New York,
1977).
17 Richard B. Spence, 'The Tragic Fate of Kalamatiano: America's Man in Moscow',
International Journal ofI ntelligence and Counterintelligence, 12:3, (1999), 346-374. Klaus
Schwabe, 'U.S. Secret War Diplomacy, Intelligence, and the Coming of the German
Revolution in 1918: The Role of Vice Consul James McNally', Diplomatic History, 16:2
(1992), 175-200. John F. Chalkley, Zach Lamar Cobb: EI Paso Collector of Customs and
Intelligence During the Mexican Revolution, J 9 J3 -J 9 J8 , (El Paso, 1998). David A. Langbart,
'''Spare No Expense": The Department of State and the Search for Information about
Bolshevik Russia, November 1917 -September 1918', Intelligence and National Security, 4:2,