Table Of ContentAn Introduction to p O R M AI
LANGUAGES
A ND
MACHINE
C O M P U T A T I ON
This page is intentionally left blank
An Introduction to F O R M AL
LANGUAGES
A ND
MACHINE
C O M P U T A T I ON
Song Y Yan
University of York, UK
World Scientific
Singapore • New Jersey • London • Hong Kong
Published by
World Scientific Publishing Co. Pte. Ltd.
P O Box 128, Farrer Road, Singapore 912805
USA office: Suite IB, 1060 Main Street, River Edge, NJ 07661
UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE
Library of Congress Cataloging-in-Publication Data
Yan, Song Y.
An introduction to formal languages and machine computation / S. Y. Yan.
p. cm.
Includes bibliographical references and index.
ISBN 9810221673 -- ISBN 9810234228 (pbk)
1. Formal languages. 2. Machine theory. I. Title.
QA267.3.Y36 1998
004'.0r51--dc21 97-44021
CIP
British Library Cataloguing-in-Pu blication Data
A catalogue record for this book is available from the British Library.
Copyright © 1998 by World Scientific Publishing Co. Pte. Ltd.
All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means,
electronic or mechanical, including photocopying, recording or any information storage and retrieval
system now known or to be invented, without written permission from the Publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright
Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to
photocopy is not required from the publisher.
This book is printed on acid-free paper.
Printed in Singapore by UtoPrint
Preface
Computer Science: The study of the use and sometimes construction of digital com
puters. It is a fashionable, interesting, difficult, and perhaps useful activity.
- CHRISTOPHER STRACHEY (1916-1975)
This book provides a concise and a modern introduction to Formal Languages
and Machine Computation, a group of disparate topics in the Theory of Com
putation, including formal languages, automata theory, Turing machines, com-
putability, complexity, number-theoretic computation and cryptography.
The theory of computation is essentially a mathematical subject. Indeed,
many theoretical computer scientists are mathematicians, and some of them
are even pure mathematicians. On reviewing the history of computer science,
one can easily see that the key figures in the development of modern computer
science, such as John von Neumann, Alan Turing, Alonzo Church, Stephen
Kleene, Donald Knuth, Stephen Cook, to name a few, are mathematicians.
However, as pointed out by Prof. Richard Hamlet at the University of Mary
land in his book Introduction to Computation Theory, 1974: "The disappoint
ing fact is that while computer science welcomes some mathematical experts,
and recommends that students of the subject have a dose of theory as 'pre-
ventative mathematics', not very many practical computer scientists attempt
to understand and apply the deep results of theory". The reason for this, we
think, is partly because mathematics, particularly certain branches of mathe
matics such as number theory and topology, are very hard to understand for
most practical computer scientists. But the problem is that in many areas of
computer science, mathematics is a very useful and sometimes even indispens
able tool. In fact, although computer science is a very practical subject, it
has its root in mathematics; for example, logic programming has its root in
first-order logic, compilers in automata theory, programming languages in for
mal languages, computer architecture in Boolean algebra, algorithm analysis
in combinatorial mathematics, and so on. In this book we shall try to bridge
v
vi Preface
the gap between mathematics and computer science by presenting some basic
mathematical concepts and results which are useful in computer science, but are
not too mathematically involved. In fact, the book was written essentially for
students in "practical" computer science, rather than "theoretical" computer
science, or mathematics. To achieve such a goal, we do not follow the tradi
tional mathematics textbook style: "Definition-Theorem-Proof", rather, we use
the more practical style: "Definition-Theorem/Algorithm-Example/Exercise",
because we believe that most practical computer scientists, systems designers
or computer programmers are generally more interested in the applicability and
usefulness of the theorems in computer science/mathematics, rather than how
to prove the theorems. However, for those who really want to know the nature
of the theorems, it is a good exercise for them to try to give all the missing
proofs.
The book is organized as follows. Chapter 1 provides an introduction to
topics in computation-related mathematics, including mathematical logic, set
theory, functions and relations, graph theory, number theory, and abstract alge
bra, which provide a necessary mathematical foundation for the rest of the book.
Although this chapter is rather lengthy in contrast to other similar books on the
market, it proves to be an invaluable source of information because practical
computer scientists usually lack sufficient mathematical background to under
stand the results in the theory of computation.
Chapter 2 is about automata and formal languages. Finite-state automata
(FA) are the simplest possible model for computation. In fact, the human
brain is a type of finite-state automaton with some 1025 states. We shall first
study the basic concepts and results of finite automata and their correspond
ing regular languages LREG- There are actually two different types of finite
automata: deterministic finite automata (DFA) and non-deterministic finite
automata (NFA). We shall show that these two types of automata are equiva
lent, because they can simulate each other, and more importantly, they accept
exactly the same languages. Then we shall study push-down automata (PDA)
and their corresponding languages, context-free languages LCFL- Unlike DFA
and NFA, non-deterministic push-down automata (NPDA) can accept a larger
class of languages than deterministic push-down automata (DPDA), i.e., DPDA
accept only deterministic context-free languages LDCFL, whereas NPDA accept
non-deterministic context-free languages LNCFL- Since LDCFL C -^NCFL, NPDA
have more power than DPDA. Finally, we shall study Turing machines and
their corresponding languages, called recursively enumerable languages L .
RE
We shall show that LRE is acceptable but not decidable by Turing machines.
Preface vii
We shall also show that there are languages, called recursive languages Z/REC,
that are recursive, but not enumerable, i.e., LREC C LRE- This class of recursive
languages LREC is decidable by Turing machines.
The machines we study in Chapter 2 act as language accepters. This does
not mean that automata, or more generally Turing machines, can only be used
to accept languages. Automata, particularly Turing machines, the highest-level
automata, are general-purpose computing machines for general computation.
So in Chapter 3, we shall study the computability/decidability and computa
tional complexity within the theoretical framework of Turing machines. More
specifically, we shall study what Turing machines (or general computers) can do
and what Turing machines cannot do, what problems are feasible by a Turing
machine (or a general computer) and what problems are infeasible by a Turing
machine. We shall also provide a brief account of the asymptotic behaviours of
some common complexity functions and the analysis of algorithms.
Number theory has played an important role in the development of modern
computer science and engineering, noting the titles of the seminal papers by
Alan Turing "On Computable Numbers, with an Application to the Entschei-
dungsproblem" (1936) and by Alonzo Church "An Unsolved Problem of Elemen
tary Number Theory" (1936). So in Chapter 4, we shall study some modern
mathematical methods for large number-theoretic computations, particularly
those methods for primality testing and integer factorization of large numbers.
We shall also discuss some applications of number-theoretic computations in
computer science, including public-key cryptography, database security, ran
dom number generation, and high-speed computation.
Turing machines were proposed by Alan Turing in 1936. Since then, many
new computation models, particularly parallel computation models (e.g., neu
ral networks, cellular automata, quantum computation and molecular biolog
ical computation) have been proposed. In the last chapter of this book, we
shall present a brief introduction to the two recently developed new models of
computation, namely quantum computation and molecular biological compu
tation. We shall show that from a computability point of view, both quantum
and molecular biological models will have no more computational power than
Turing machines. However, from a complexity point of view, both quantum
and molecular biological computers do offer some more computational power.
For example, the traveling salesman problem (TSP) and the directed Hamil-
tonian path problem (HPP) have been proven to be Af'P-complete on Turing
machines, but they can be solved in polynomial biological steps by biological
computers; whereas integer factorization and discrete logarithms are believed
viii Preface
to be intractable by Turing machines, but they can be solved in polynomial
time on a quantum computer. Of course, the key issue concerning quantum
and molecular biological models is how to actually build a truly quantum or
biological computer! At present, no quantum and molecular biological comput
ers are available on the market, but there is no reason why we cannot build
such new generations of computers. Compared with many other books in the
theory of computation on the market, this book contains substantially more
materials in number-theoretic computations and their applications in computer
science and secure communications. As pointed out by Prof. Ian Stewart (Na
ture, Vol. 325, 15 January 1987, page 199): "computer scientists working on
algorithms for factorization would be well advised to brush up on their num
ber theory", we hope these materials will not only help computer scientists to
brush up on their number theory, but also encourage them to apply results in
number theory to computer science in such diverse areas as computational com
plexity, public-key cryptography, network and information security, computer
arithmetic and architectures, graphic design, image and signal processing, and
digital communications.
Acknowledgements: This book grew out from my lecture notes for the fol
lowing three courses: Mathematics for Computing, Formal Languages and Ma
chines, and Algorithms and Complexity, taught at La Trobe University, Aus
tralia (1990-1992), and The Buckinghamshire College of Brunei University,
England (1993-1995). It was completely rewritten while I was in the Depart
ments of Mathematics and Computer Science at the University of York, Eng
land, over the period of 1993 and 1997. No book is created in a vacuum, and
this one is no exception; I could not have written this book without the help
of many friends, colleagues, students and my family. I would like to take this
opportunity to thank James Austin, Thram Dillon, William Freeman, Terence
Jackson, Milton Munroe, Ian Robinson and John Zeleznikow for their encour
agement and help during the writing of this book. I would also like to thank
the editors at World Scientific; Tony Moore at the London Office and partic
ularly Stephen Patt and Joy Marie Tan at the Singapore Office for their help
and encouragement. Special thanks must be given to William Bloodworth,
Gavin Doherty, William Freeman, Milton Munroe, and Robert Pargeter, Peter
Pleasants who read the various earlier versions of the book and suggested many
improvements. Finally, I would like to thank all the authors cited in this book;
they are the giants on whose shoulders I can stand on.
YORK, ENGLAND
DECEMBER 1997
Contents
Preface v
Notations xiii
1 Computation-Related Mathematics 1
1.1 Logics and Proofs 1
1.1.1 Propositional Logic 2
1.1.2 Predicate Logic 7
1.1.3 Programming in Logic 11
1.1.4 Proof Techniques 20
1.2 Sets, Functions and Graphs 27
1.2.1 Sets and Operations on Sets 27
1.2.2 Functions and Relations 33
1.2.3 Directed and Undirected Graphs 42
1.3 Divisibility, Continued Fractions and Congruences 45
1.3.1 Divisibility 46
1.3.2 Continued Fractions 54
1.3.3 Diophantine Equations 59
1.3.4 Congruences 64
1.4 Groups, Rings and Fields 79
1.4.1 Binary Operations 79
1.4.2 Groups 81
1.4.3 Rings and Fields 85
1.4.4 Elliptic Curves over a Ring/Field 89
2 Formal Languages and Automata 95
2.1 Languages, Grammars and Automata 95
2.2 Finite Automata and Regular Languages 100
ix
x CCoonntteennttss
X
2.2.1 Deterministic Finite Automata (DFA) 101
2.2.2 Nondeterministic Finite Automata (NFA) 106
2.2.3 Regular Languages (Z/REG) 108
2.2.4 Applications of Finite Automata 115
2.2.5 Variants of Finite Automata 118
2.3 Push-Down Automata and Context-Free Languages 128
2.3.1 Push-Down Automata (PDA) 129
2.3.2 Context-Free Languages (LCF) 130
2.3.3 Applications of Context-Free Languages 134
2.4 Turing Machines and Recursively Enumerable Languages . . .. 135
2.4.1 Standard Turing Machines 136
2.4.2 Turing Machines as Language Accepters 137
2.4.3 Variants of Turing Machines 146
2.4.4 Recursively Enumerable Languages LRE 152
2.4.5 Context-Sensitive Languages Lcs 153
2.4.6 Hierarchy of Machines and Grammars/Languages . . .. 155
3 Turing Computability and Complexity 163
3.1 Computability and Noncomputability 163
3.1.1 Computation with Turing Machines 164
3.1.2 Computable Functions 169
3.1.3 Noncomputable Functions 175
3.1.4 Horn Clause Computability 179
3.2 Decidability and Undecidability 182
3.2.1 Decision Problems 182
3.2.2 Decidable Problems 185
3.2.3 Decidable Problems in Formal Languages 187
3.2.4 Undecidable Problems 188
3.2.5 Semi-decidable and Hilbert's Tenth Problem 193
3.2.6 Undecidability in Formal Languages 197
3.3 Computational Complexity 198
3.3.1 Time and Space Complexity Classes 199
3.3.2 Complexity Classification of Formal Languages 201
3.3.3 Polynomial-Bounded Complexity Classes 203
3.3.4 Parallel and Probabilistic Complexity Classes 208
3.3.5 JvT-Complete Problems 215
3.4 Design and Analysis of Algorithms 218
3.4.1 Asymptotic Complexity 219
