A comprehensive guide to virtual private networks. Volume III, Cross-platform key and policy management

690 Pages·1999·8.631 MB·English

by  International Business Machines Corporation

A Comprehensive Guide to
Virtual Private Networks, Volume III:
Cross-Platform Key and Policy Management
Martin W. Murhammer, Orcun Atakan, Zikrun Badri, Beomjun Cho
Hyun Jeong Lee, Alexander Schmid
International Technical Support Organization
http://www.redbooks.ibm.com
Draft Document for Review July 12, 1999 11:13 am SG24-5309-00

International Technical Support Organization SG24-5309-00
A Comprehensive Guide to
Virtual Private Networks, Volume III:
Cross-Platform Key and Policy Management
October 1999
Draft Document for Review October 8, 1999 2:47 pm
5309edno.fm Draft Document for Review October 8, 1999 2:47 pm
Take Note!
Before using this information and the product it supports, be sure to read the general information in Appendix A,
“Special Notices” on page 659.
First Edition (October 1999)
This edition applies to the VPN components of the following IBM products:
• AIX V4.3.2 and V4.3.3
• OS/400 V4R4
• Comunicatoins Server and Security Server for OS/390 V2R8
• Nways 2210, 2212 and 2216 routers using MRS/AIS/MAS V3.3
This edition also applies to the VPN components of selected non-IBM products.
Note
This book is based on a pre-GA version of a product and may not apply when the product becomes generally
available. We recommend that you consult the product documentation or follow-on versions of this redbook for
more current information.
Comments may be addressed to:
IBM Corporation, International Technical Support Organization
Dept. HZ8 Building 678
P.O. Box 12195
Research Triangle Park, NC 27709-2195
When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any
way it believes appropriate without incurring any obligation to you.
© Copyright International Business Machines Corporation 1999. All rights reserved
Note to U.S Government Users - Documentation related to restricted rights - Use, duplication or disclosure is subject to restrictions
set forth in GSA ADP Schedule Contract with IBM Corp.
Draft Document for Review October 15, 1999 12:01 pm 5309TOC.fm
Contents
Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxiii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii
0.1 How this Book is Organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii
The Team That Wrote This Redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxviii
Comments Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix
Part 1. VPN Overview and Technology Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Chapter 1. Virtual Private Network (VPN) Introduction. . . . . . . . . . . . . . . .33
1.1 What is a VPN? A quick review. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
1.2 VPN benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
1.3 VPN requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
1.3.1 Security considerations for VPNs . . . . . . . . . . . . . . . . . . . . . . . . . . .35
1.3.2 Performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
1.3.3 Management considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42
1.3.4 General purpose encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
1.4 A basic pproach to VPN design and implementation. . . . . . . . . . . . . . . . .44
1.5 Common VPN scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
1.5.1 Branch Office Interconnections . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
1.5.2 Business partner/supplier networks. . . . . . . . . . . . . . . . . . . . . . . . . .47
1.5.3 Remote Access Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
1.6 VPN technologies and security policies. . . . . . . . . . . . . . . . . . . . . . . . . . .49
1.6.1 The need for a security policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50
1.6.2 Network security policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51
1.6.3 VPN security policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52
Chapter 2. Layer 2 VPN Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
2.1 Layer 2 Tunneling Protocol (L2TP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
2.1.1 Overview and standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
2.1.2 L2TP flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
2.1.3 Compulsory and voluntary tunnel modes. . . . . . . . . . . . . . . . . . . . . .56
2.1.4 Securing the tunnels with IPSec . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
2.1.5 Multiprotocol support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
2.2 Point-to-Point Tunneling Protocol (PPTP). . . . . . . . . . . . . . . . . . . . . . . . .60
2.3 Layer 2 Forwarding (L2F) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61
2.4 Comparing remote access tunneling protocols . . . . . . . . . . . . . . . . . . . . .62
2.5 Layer 2 tunneling authentication and encryption . . . . . . . . . . . . . . . . . . . .63
2.5.1 Authentication options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
2.5.2 Encryption options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65
Chapter 3. Layer 3 VPN Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
3.1 IP Security Architecture (IPSec) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
3.1.1 Overview and standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
3.1.2 Security Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68
3.1.3 IP Authentication Header (AH) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .69
3.1.4 Encapsulating Security Payload (ESP) . . . . . . . . . . . . . . . . . . . . . . .70
3.1.5 Tunnel and transport mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71
3.1.6 SA combinations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72
© Copyright IBM Corp. 1999 iii
5309TOC.fm Draft Document for Review October 15, 1999 12:01 pm
3.2 Coming to terms with the Internet Key Exchange (IKE) protocol . . . . . . . 75
3.2.1 Overview and standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.2.2 Key management requirements for IPSec . . . . . . . . . . . . . . . . . . . . 76
3.2.3 IKE Phase 1 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2.4 IKE Phase 2 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2.5 ISAKMP Message Structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.2.6 General Phase 1 process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.2.7 General Phase 2 process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.8 Summary of successful IKE negotiation. . . . . . . . . . . . . . . . . . . . . . 95
3.2.9 Optional IKE Exchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3 IPSec/IKE system processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.3.1 Outbound IPSec processing for host systems . . . . . . . . . . . . . . . . . 98
3.3.2 Inbound processing for host systems. . . . . . . . . . . . . . . . . . . . . . . . 99
3.3.3 Outbound processing for gateway systems . . . . . . . . . . . . . . . . . . . 99
3.3.4 Inbound processing for gateway systems. . . . . . . . . . . . . . . . . . . . 100
Chapter 4. Certificates and Public Key Infrastructures. . . . . . . . . . . . . . 103
4.1 Public Key Cryptography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.2 Digital Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.3 Registration Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.4 Multiple Certificate Authorities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.4.1 Single Root CA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.4.2 Hierarchial Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.4.3 Peer Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.5 PKI Requirements for IKE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Chapter 5. Security Technologies Complementing VPNs. . . . . . . . . . . . 115
5.1 Authentication for Remote Access Dial-In Users . . . . . . . . . . . . . . . . . . 115
5.1.1 RADIUS Operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.1.2 Using RADIUS with Layer 2 Tunnels . . . . . . . . . . . . . . . . . . . . . . . 118
5.2 Network Address Translation (NAT). . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.3 SOCKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.4 Secure Sockets Layer (SSL) and Transport Layer Security (TLS) . . . . . 122
5.5 Comparing IPSec to SSL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Chapter 6. Directory Assisted Policy Management. . . . . . . . . . . . . . . . . 127
6.1 The Benefits of Directory Assisted Policy Management . . . . . . . . . . . . . 127
6.2 Directory Client and Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.2.1 LDAP Schema. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.3 Directory Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.4 Policy Deployment using LDAP for IBM 221x Router. . . . . . . . . . . . . . . 128
6.4.1 LDAP server configuration on AIX . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.4.2 LDAP Client Configuration on the NWays 221x-Routers . . . . . . . . 133
6.5 Secure transmission of LDAP traffic using tunnel. . . . . . . . . . . . . . . . . . 136
Chapter 7. Internet VPN Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.1 Management Area. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2 Management requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.3 Design Consideration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.4 Management object for Internet VPN. . . . . . . . . . . . . . . . . . . . . . . . . . . 145
7.5 Integration to other management tool . . . . . . . . . . . . . . . . . . . . . . . . . . 146
7.6 Network management system for IBM 221x router. . . . . . . . . . . . . . . . . 146
iv A Comprehensive Guide to Virtual Private Networks, Volume III
Draft Document for Review October 15, 1999 12:01 pm 5309TOC.fm
Part 2. IBM VPN Platforms with IKE Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149
Chapter 8. Introduction to IBM VPN solutions . . . . . . . . . . . . . . . . . . . . .151
8.1 IBM VPN platforms - IPSec and IKE feature summary . . . . . . . . . . . . . .151
8.2 IBM VPN platforms - layer 2 tunneling feature summary . . . . . . . . . . . . .153
8.3 IBM VPN platforms - interoperability matrix for IKE. . . . . . . . . . . . . . . . .154
8.4 IBM VPN platforms supporting IPSec but not IKE . . . . . . . . . . . . . . . . . .154
8.5 IBM VPN platforms- interoperability matrix for IPSec without IKE . . . . . .155
8.6 IBM and OEM VPN platforms - interoperability matrix. . . . . . . . . . . . . . .156
Chapter 9. AIX V4.3.2 and V4.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
9.1 AIX V4.3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
9.1.1 IPSec and Internet Key Exchange (IKE) VPN Features. . . . . . . . . .159
9.1.2 VPN Feature Installation on AIX V4.3.2. . . . . . . . . . . . . . . . . . . . . .160
9.1.3 AIX V4.3.2 IP Security: IKE tunnel basic set up. . . . . . . . . . . . . . . .161
9.1.4 AIX V4.3.2 IP Security IKE Advanced Setup . . . . . . . . . . . . . . . . . .171
9.1.5 Use Tunnel Lifetime and Lifesize. . . . . . . . . . . . . . . . . . . . . . . . . . .179
9.1.6 Packet Filtering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .180
9.1.7 Manual Tunnel Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .182
9.2 AIX V4.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .184
9.2.1 VPN Features and Improvements in AIX V4.3.3 . . . . . . . . . . . . . . .184
9.2.2 AIX V4.3.3 VPN Feature Installation . . . . . . . . . . . . . . . . . . . . . . . .185
9.2.3 IP Security IKE Tunnel Basic Setup Using the Configuration Wizard187
9.2.4 IP Security IKE Tunnel Advanced Setup . . . . . . . . . . . . . . . . . . . . .190
9.2.5 Manual tunnel configuration using the WebSM . . . . . . . . . . . . . . . .197
9.2.6 Filtering Capability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .200
9.3 Creating a VPN host-to-host connection . . . . . . . . . . . . . . . . . . . . . . . . .203
Chapter 10. OS/400 V4R4 Native VPN Support . . . . . . . . . . . . . . . . . . . . .211
10.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211
10.2 VPN software prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211
10.3 AS/400 VPN components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .212
10.3.1 AS/400 Operations Navigator . . . . . . . . . . . . . . . . . . . . . . . . . . . .212
10.3.2 New Connection Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213
10.3.3 VPN server jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213
10.3.4 VPN policy database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213
10.3.5 IP packet filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .214
10.4 Basic planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215
10.5 VPN configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .221
10.5.1 AS/400 Operations Navigator . . . . . . . . . . . . . . . . . . . . . . . . . . . .221
10.5.2 Using the New Connection Wizard . . . . . . . . . . . . . . . . . . . . . . . .223
10.5.3 Changing the New Connection Wizard default values . . . . . . . . . .226
10.5.4 Objects created by the wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . .226
10.5.5 Configuring IP filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .227
10.5.6 Object relationships. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228
10.6 VPN management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .229
10.6.1 IP packet security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .229
10.6.2 VPN server jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .231
10.6.3 Starting VPN connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .234
10.7 Backup and recovery considerations. . . . . . . . . . . . . . . . . . . . . . . . . . .237
10.7.1 Creating a VPN Host-to-Host Connection . . . . . . . . . . . . . . . . . . .237
10.7.2 Configuring IP Packet Security . . . . . . . . . . . . . . . . . . . . . . . . . . .245
10.7.3 Starting the VPN connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . .252
v
5309TOC.fm Draft Document for Review October 15, 1999 12:01 pm
10.7.4 Relationship between the wizard and the configuration objects . . 257
Chapter 11. Communications Server V2R8 for OS/390 . . . . . . . . . . . . . . 261
11.1 Firewall Technologies for OS/390 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
11.2 Installation and Customization of VPN IKE feature . . . . . . . . . . . . . . . 262
11.2.1 OS/390 SecureWay CS IP services customization. . . . . . . . . . . . 262
11.2.2 Unix System Services customization . . . . . . . . . . . . . . . . . . . . . . 265
11.2.3 OS/390 Security Server and cryptographic services customization266
11.2.4 OS/390 Firewall USS customization and starting. . . . . . . . . . . . . 279
11.3 Dynamic tunnel scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
11.3.1 Creating a dynamic VPN connection using the GUI panels . . . . . 302
11.3.2 Creating a dynamic VPN using the shell commands . . . . . . . . . . 323
Chapter 12. Nways Routers Using MRS/AIS/MAS V3.3 . . . . . . . . . . . . . . 329
12.1 Policy Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
12.2 Configuring IPSec on an Nways Router. . . . . . . . . . . . . . . . . . . . . . . . 331
12.2.1 Configuring Manual IPSec Tunnels . . . . . . . . . . . . . . . . . . . . . . . 334
12.2.2 Configuring IKE with Pre-shared Keys. . . . . . . . . . . . . . . . . . . . . 344
12.2.3 IKE with PKI Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
Part 3. VPN Scenarios Using IBM VPN Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
Chapter 13. Building Branch Office VPNs . . . . . . . . . . . . . . . . . . . . . . . . 385
13.1 Design Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
13.1.1 Authenticating Backbone Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . 385
13.1.2 Data Confidentiality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
13.1.3 Addressing Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
13.1.4 Routing Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
13.1.5 Summary: Branch Office Connection. . . . . . . . . . . . . . . . . . . . . . 388
13.2 Central Site - Small Enterprise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
13.2.1 Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
13.2.2 Gateway-to-Gateway Tunnel with IPSec between IBM Routers . . 390
13.2.3 Scenario Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
13.2.4 Implementation Tasks - Summary . . . . . . . . . . . . . . . . . . . . . . . . 391
13.2.5 Completing the IBM 2216 Router Planning Worksheet. . . . . . . . . 392
13.2.6 Configuring the VPN in the IBM 2216 Routers. . . . . . . . . . . . . . . 396
13.2.7 Connection Verification and Testing. . . . . . . . . . . . . . . . . . . . . . . 399
13.3 Central Site - Medium Enterprise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
13.3.1 Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
13.3.2 Gateway-to-Gateway Tunnel with IPSec between IBM AIX Systems .
400
13.3.3 Scenario Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
13.3.4 Implementation Tasks - Summary . . . . . . . . . . . . . . . . . . . . . . . . 402
13.3.5 Completing the AIX Planning Worksheet . . . . . . . . . . . . . . . . . . . 402
13.3.6 Configuring the Central Site Gateway . . . . . . . . . . . . . . . . . . . . . 404
13.3.7 Configuring the Branch Office Gateway. . . . . . . . . . . . . . . . . . . . 405
13.3.8 Connection Verification and Testing. . . . . . . . . . . . . . . . . . . . . . . 405
13.4 Central and Regional Sites - Large Enterprise. . . . . . . . . . . . . . . . . . . 406
13.4.1 Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
13.4.2 IBM AS/400 to IBM 2210 Gateway-to-Gateway tunnel with IPSec 407
13.4.3 Scenario Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
13.4.4 Implementation tasks - Summary. . . . . . . . . . . . . . . . . . . . . . . . . 409
13.4.5 Completing the 2210 router planning worksheet. . . . . . . . . . . . . . 410
vi A Comprehensive Guide to Virtual Private Networks, Volume III
Draft Document for Review October 15, 1999 12:01 pm 5309TOC.fm
13.4.6 Completing the AS/400 system planning worksheet . . . . . . . . . . .415
13.4.7 VPN configuration cross reference table - OS/400 to 2210 router .418
13.4.8 Configuring the VPN in the 2210 router. . . . . . . . . . . . . . . . . . . . .419
13.4.9 Configuring the VPN on the AS/400 system (RALYAS4A . . . . . . .421
13.4.10 Configuring IP filtering on the AS/400 system (RALYAS4A) . . . .423
13.4.11 Starting IP filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .424
13.4.12 Starting the VPN connection . . . . . . . . . . . . . . . . . . . . . . . . . . . .424
13.4.13 Verification tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .427
Chapter 14. Building Business Partner / Supplier VPNs . . . . . . . . . . . . .429
14.1 Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .429
14.1.1 Authenticating and Encrypting Supplier Traffic . . . . . . . . . . . . . . .430
14.1.2 Addressing Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .432
14.1.3 Packet Filtering and Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . .432
14.1.4 Summary: Inter-Company Interconnection. . . . . . . . . . . . . . . . . . .433
14.2 Nested Tunnel Configurations With IKE . . . . . . . . . . . . . . . . . . . . . . . .433
14.2.1 IBM Router configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .434
14.3 End-to-End Tunnels with IPSec. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .443
14.3.1 Scenario characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .443
14.3.2 Implementation Tasks - Summary. . . . . . . . . . . . . . . . . . . . . . . . .444
14.3.3 Completing the AIX server planning worksheet . . . . . . . . . . . . . . .444
14.3.4 Completing the AS/400 system planning worksheet . . . . . . . . . . .446
14.3.5 Configuring a host to host VPN in the AIX server . . . . . . . . . . . . .448
14.3.6 Configuring a host to host VPN in the AS/400 system. . . . . . . . . .450
14.3.7 Matching the AIX server VPN configuration. . . . . . . . . . . . . . . . . .452
14.3.8 Configuring IP filters on the AS/400 system (RALYAS4C). . . . . . .454
14.3.9 Starting the VPN Connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . .458
14.3.10 Verification Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .460
Chapter 15. Building Remote Access VPNs . . . . . . . . . . . . . . . . . . . . . . .461
15.1 Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .461
15.1.1 Data Confidentiality and Authentication. . . . . . . . . . . . . . . . . . . . .462
15.1.2 Addressing and Routing Issues. . . . . . . . . . . . . . . . . . . . . . . . . . .462
15.1.3 Multiprotocol Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .462
15.1.4 Summary: Remote Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .463
15.2 Remote Access With IPSec. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .463
15.2.1 Description of the Scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .464
15.2.2 Configuration of the ISP Router. . . . . . . . . . . . . . . . . . . . . . . . . . .465
15.2.3 Configuration of the VPN Gateway (Center 2216 Router) . . . . . . .468
15.2.4 Configure IPSec Action and Proposal . . . . . . . . . . . . . . . . . . . . . .471
15.2.5 Configure ISAKMP Action and Proposal . . . . . . . . . . . . . . . . . . . .473
15.2.6 Configuration of the IRE SafeNet VPN Client . . . . . . . . . . . . . . . .475
15.2.7 Testing and Verifying the Connection . . . . . . . . . . . . . . . . . . . . . .477
15.3 End-to-End Connections Using L2TP and IPSec. . . . . . . . . . . . . . . . . .479
15.4 Dial-on-Demand via ISP Using L2TP. . . . . . . . . . . . . . . . . . . . . . . . . . .479
Chapter 16. VPN Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .481
16.1 Log Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .481
16.2 Alerting and Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .481
16.3 Traces, Dumps and Traffic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . .481
16.3.1 Traces and Dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .481
16.3.2 Traffic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .482
16.4 Interfaces to Systems Management Tools. . . . . . . . . . . . . . . . . . . . . . .492
16.5 Ethical Hacking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .492
vii
5309TOC.fm Draft Document for Review October 15, 1999 12:01 pm
16.6 Troubleshooting for AIX 4.3.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
16.6.1 IP Security log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
16.6.2 ISAKMPD log file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
16.7 Troubleshooting for OS/400. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497
16.7.1 Available methods troubleshooting Virtual Private Networks . . . . 497
16.7.2 General guideline for VPN troubleshooting . . . . . . . . . . . . . . . . . 498
16.7.3 Using and customizing the Active Connections window. . . . . . . 499
16.7.4 Using the QIPFILTER Journal . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
16.7.5 Using the QVPN journal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503
16.7.6 The Trace TCP/IP Application (TRCTCPAPP) command. . . . . . . 505
16.7.7 Using joblogs for problem determination . . . . . . . . . . . . . . . . . . . 507
16.7.8 Using the AS/400 communications trace . . . . . . . . . . . . . . . . . . . 508
16.8 Troubleshooting for OS/390. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508
16.8.1 Using the Firewall Log to Check the Tunnel. . . . . . . . . . . . . . . . . 508
16.9 Troubleshooting for IBM 221x Router . . . . . . . . . . . . . . . . . . . . . . . . . 509
16.9.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
16.9.2 Order of Commands While Troubleshooting. . . . . . . . . . . . . . . . . 510
16.9.3 Useful Commands for Policy and IPSec. . . . . . . . . . . . . . . . . . . . 510
16.9.4 Useful Commands for IKE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
16.9.5 Useful Commands for layer 2 VPNs. . . . . . . . . . . . . . . . . . . . . . . 516
16.9.6 Authentication commands and RADIUS. . . . . . . . . . . . . . . . . . 520
16.9.7 Useful Commands for LDAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522
16.9.8 Using ELS Subsystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
16.9.9 Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
Part 4. OEM VPN Platforms and Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
Chapter 17. Interoperability with Cisco Routers . . . . . . . . . . . . . . . . . . . 527
17.1 Cisco IOS VPN Capabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
17.2 Configuring Cisco IOS for IPSec and IKE . . . . . . . . . . . . . . . . . . . . . . 528
17.2.1 IKE Configuration using pre-shared key authentication . . . . . . . . 528
17.2.2 IKE Configuration using RSA signature authentication. . . . . . . . . 532
17.2.3 IPSec Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
17.2.4 Connection Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
17.3 IBM 2216 to Cisco 2612, Gateway-to-Gateway . . . . . . . . . . . . . . . . . . 538
17.3.1 Scenario characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
17.3.2 Implementation Tasks - Summary . . . . . . . . . . . . . . . . . . . . . . . . 539
17.3.3 Completing the IBM 2216 Router Planning Worksheet. . . . . . . . . 539
17.3.4 Configuring the VPN in the IBM 2216 router . . . . . . . . . . . . . . . . 544
17.3.5 Completing the Cisco Router Planning Worksheet. . . . . . . . . . . . 546
17.3.6 Configuring the VPN in the Cisco router. . . . . . . . . . . . . . . . . . . . 548
17.3.7 Connection Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549
17.3.8 Verification tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
17.4 IBM AS/400 to Cisco 2612, Gateway-to-Gateway . . . . . . . . . . . . . . . . 551
17.4.1 Scenario Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
17.4.2 Implementation Tasks - Summary . . . . . . . . . . . . . . . . . . . . . . . . 553
17.4.3 Completing the Cisco Router Planning Worksheet. . . . . . . . . . . . 554
17.4.4 Completing the AS/400 System Planning Worksheet. . . . . . . . . . 556
17.4.5 Configuring the VPN in the Cisco router. . . . . . . . . . . . . . . . . . . . 558
17.4.6 Configuring the VPN on the AS/400 system (RALYAS4A) . . . . . . 562
17.4.7 Matching the Cisco router VPN configuration. . . . . . . . . . . . . . . . 563
17.4.8 Configuring IP filtering on the AS/400 system (RALYAS4A). . . . . 564
17.4.9 Starting IP filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565
viii A Comprehensive Guide to Virtual Private Networks, Volume III