Java 2 Network Security

702 Pages·1999·5.589 MB·English

by  Marco Pistoia, Duane F. Reller, Deepak Gupta, Milind Nagnur, Ashok Ramani

Java 2 Network Security
Marco Pistoia, Duane F. Reller
Deepak Gupta, Milind Nagnur, Ashok K. Ramani
International Technical Support Organization
http://www.redbooks.ibm.com
SG24-2109-01

SG24-2109-01
International Technical Support Organization
Java 2 Network Security
Marco Pistoia, Duane F. Reller
Deepak Gupta, Milind Nagnur, Ashok K. Ramani
Foreward by Li Gong
Distinguished Engineer and Chief Java Security Architect
Sun Microsystems, Inc.
June 1999
Take Note!
Before using this information and the product it supports, be sure to read the general information in
Appendix F, “Special Notices” on page 659.
Second Edition (June 1999)
This edition applies to Java 2 SDK, Standard Edition, V 1.2.
Comments may be addressed to:
IBM Corporation, International Technical Support Organization
Dept. HZ8 Building 678
P.O. Box 12195
Research Triangle Park, NC 27709-2195
When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the
information in any way it believes appropriate without incurring any obligation to you.
© Copyright International Business Machines Corporation 1997 1999. All rights reserved.
Note to U.S Government Users – Documentation related to restricted rights – Use, duplication or disclosure is
subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.
Foreword
As the person who led the JavaSoft team that developed the Java security
technology discussed in this book, it is extremely gratifying to see people
spend their precious time writing about our technology and products. Every
engineer’s dream is to have his or her technology deployed and used by
thousands of others, and this book is a great help to Java developers who
write security-aware applications.
Security is a difficult subject to write about. On the one hand, security is in
people’s daily consciousness so that it appears easy to get across (to the
reader) some of the basic concepts. On the other hand, security applied to
computer and networking is often subtle and unexpected. Security also is
pervasive in that it touches all aspects of the computing technology, including
hardware, software, operating system, software libraries, communication
software, networking infrastructure, application software, user interface, and
management software. In order to understand security in any situation, one
has to understand the entire system under consideration as well as each
individual component so that one can identity their strengths and weaknesses
and design the appropriate solutions.
Java security is one of the more recent additions to the family of security
technologies. Ever since Sun Microsystems announced Java technology in
the spring of 1995, there has been strong and growing interest (in industry,
research laboratories, and academia) around the security of the Java platform
as well as new security issues raised by the deployment of Java technology.
Such close attention being paid to security is almost unprecedented in that
new computing technologies normally ignore security considerations when
they emerge initially. Most of them remain unsecured forever. In the few cases
where efforts are made to secure them later, the efforts are typically not very
successful because retrofitting security is usually very difficult, if possible at
all, and often causes backward compatibility problems.
Therefore, it is extremely fortunate that the Java technology had security as a
primary design goal from the very beginning. (Hats off to the original Java
development team. I joined JavaSoft only in 1996.) Although the initial
security model was very simplistic, it enabled later improvements in the
security architecture.
The Java language is a general-purpose object-oriented programming
language and is specifically designed to be platform independent so that
application developers can write a program once and then run it securely
© Copyright IBM Corp. 1999 3
everywhere on the Internet. To achieve this platform independence, a Java
program is compiled to a bytecode instruction set and binary format defined
in the Java Virtual Machine Specification. The Java platform consists of the
Java language and its associated tools (such as compilers), together with the
Java Virtual Machine (JVM) and its associated libraries that define a rich set
of application programming interfaces (APIs).
Security for the Java platform has multiple layers. First of all, the Java
language is strongly typed and does not include any unsafe constructs, such
as array accesses without index checking, because such unsafe constructs
may result in unspecified and unpredictable program behavior that can lead to
security compromises. Type safety is checked both at the time a piece of
bytecode is loaded into the JVM and throughout the lifetime of the bytecode
(that is, during run time) until it is no longer used and garbage collected.
Second, mechanisms (for example, class loaders) are in place to ensure a
sufficient degree of separation between multiple Java programs so that they
do not interfere with each other in undesirable ways.
Third, access to crucial system resources is mediated by the JVM. A security
manager is installed to deny all requests for unauthorized access. The access
control model, in the initial release of the Java Development Kit (JDK 1.0),
was to grant full access to local code (that is, trust such code and let it do
anything it wants) and to grant very restricted access to code loaded over the
network because such code (often referred to as applets) may not be trusted.
JDK 1.1 introduced a notion of trusted applets and granted full access to
these applets. The latest release, JDK 1.2 (also called Java 2), incorporates a
new security architecture that supports policy-driven, fine-grained, flexible,
and extensible access control. (For design rationales of this architecture, as
well as difficulties and subtleties we encountered during JDK 1.2
development, please refer to my book Inside Java 2 Platform Security.)
On top of type safety and access control, there are the Java Cryptography
Architecture (implemented in JDK 1.2 and in the Java Cryptography
Extension 1.2), support for secure communication (the Java Secure Socket
Extension), and a framework for user-based authentication and access
control (the Java Authentication and Authorization Service). These
technologies are at various stages in the development and release cycle.
Finally, applications can provide their own specific security features and can
customize security features that are built into the Java platform.
Our colleagues at IBM, among other industrial partners, have been closely
involved with the recent development of Java security technology. They have
supported our efforts in many ways, and have provided excellent technical
suggestions. This latest book from IBM is a comprehensive guidebook that
4 Java 2 Network Security
provides the programmer/reader with well-organized details of the Java
security APIs and their usage. The book is also broad in its coverage of the
wider security context and related issues.
I am very excited to see such a good book being published on Java security. It
will contribute greatly toward making the Java platform the most popular
deployment environment for secure computing.
Li Gong
Distinguished Engineer and Chief Java Security Architect
Sun Microsystems
Cupertino, California
May 1999
5
6 Java 2 Network Security
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
The Team That Wrote This Redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Comments Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xix
Part 1. Introduction to Java and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Chapter 1. An Overview of Java and Security . . . . . . . . . . . . . . . . . . . .3
1.1 Java Is Not Just a Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
1.2 What Java Does. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
1.3 Java Is Not an Island: Java as a Part of Security . . . . . . . . . . . . . . . . .5
1.3.1 Safety and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
1.3.2 Java as an Aid to Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
1.3.3 Java as a Threat to Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
1.3.4 Writing Secure Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
1.3.5 Staying One Jump Ahead. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
1.3.6 The Vigilant Web Site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
1.4 Understanding Java 2 Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
1.4.1 An Example of Applet Security in Java 2 . . . . . . . . . . . . . . . . . .14
1.4.2 An Example of Application Security in Java 2. . . . . . . . . . . . . . .26
1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Chapter 2. Attack and Defense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
2.1 Components of Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
2.1.1 The Development Environment. . . . . . . . . . . . . . . . . . . . . . . . . .36
2.1.2 The Execution Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
2.1.3 Interfaces and Architectures. . . . . . . . . . . . . . . . . . . . . . . . . . . .50
2.2 Java 2 and Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
2.2.1 Cryptographic Tools in Brief . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
2.2.2 Java Cryptography Architecture . . . . . . . . . . . . . . . . . . . . . . . . .56
2.2.3 United States Export Rules for Encryption . . . . . . . . . . . . . . . . .57
2.2.4 Signed Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
2.2.5 The Other Side of the Coin – Access Control . . . . . . . . . . . . . . .59
2.3 Attacking the World of Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59
2.3.1 Perils in the Life of Remote Code. . . . . . . . . . . . . . . . . . . . . . . .59
2.3.2 Vulnerabilities in Java Applications. . . . . . . . . . . . . . . . . . . . . . .66
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68
Chapter 3. The New Java Security Model . . . . . . . . . . . . . . . . . . . . . . .69
3.1 The Need for Java Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .69
© Copyright IBM Corp. 1997 1999 vii
3.2 Evolution of the Java Security Model . . . . . . . . . . . . . . . . . . . . . . . . .70
3.2.1 The JDK 1.0 Sandbox Security Model . . . . . . . . . . . . . . . . . . . .70
3.2.2 The Concept of Trusted Code in JDK 1.1 . . . . . . . . . . . . . . . . . .72
3.2.3 The Fine-Grained Access Control of Java 2 . . . . . . . . . . . . . . . .74
3.2.4 A Comparison of the Three Java Security Models . . . . . . . . . . .78
3.3 Java 2 Protection Domain and Permissions Model. . . . . . . . . . . . . . .80
3.4 New Class Search Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83
3.4.1 Boot Class Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .84
3.4.2 Extensions Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86
3.4.3 Application Class Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
3.4.4 Class Search Paths in Summary . . . . . . . . . . . . . . . . . . . . . . . .89
3.5 Java 2 Class Loading Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . .89
3.5.1 Run-Time Access Controls. . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
3.6 The Policy File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
3.6.1 The Default System-Wide Policy File . . . . . . . . . . . . . . . . . . . . .96
3.7 Security Manager vs Access Controller . . . . . . . . . . . . . . . . . . . . . . .98
3.8 Security Management with Java 2 . . . . . . . . . . . . . . . . . . . . . . . . . . .98
3.8.1 Applying a Security Manager to Applets and Applications. . . . . .99
3.8.2 Applying a User-Defined Security Policy. . . . . . . . . . . . . . . . . . .99
3.8.3 Java Security Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100
3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .106
Part 2. Under the Hood. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107
Chapter 4. The Java Virtual Machine. . . . . . . . . . . . . . . . . . . . . . . . . .109
4.1 The Java Virtual Machine, Close Up. . . . . . . . . . . . . . . . . . . . . . . . .109
4.1.1 The Class Loader. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110
4.1.2 The Class File Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112
4.1.3 The Heap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112
4.1.4 The Class Area. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112
4.1.5 The Native Method Loader. . . . . . . . . . . . . . . . . . . . . . . . . . . .113
4.1.6 The Security Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113
4.1.7 The Execution Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113
4.1.8 Just-in-Time Compilers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113
4.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .115
Chapter 5. Class Files in Java 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117
5.1 The Traditional Development Life Cycle. . . . . . . . . . . . . . . . . . . . . .117
5.2 The Java Development Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . . .119
5.3 The Java 2 Class File Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .124
5.3.1 Decompilation Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126
5.4 The Constant Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .129
5.4.1 Beating the Decompilation Threat. . . . . . . . . . . . . . . . . . . . . . .134
viii Java 2 Network Security