Oracle 9i Real Application Clusters. Concepts

248 Pages·2001·1.556 MB·English

by  Rawles J. W.

Oracle9iReal Application Clusters
Concepts
Release 1 (9.0.1)
July 2001
Part No. A89867-02
Oracle9i Real Application Clusters Concepts, Release 1 (9.0.1)
Part No. A89867-02
Copyright © 1996, 2001, Oracle Corporation. All rights reserved.
Primary Author: James W. Rawles.
Primary Contributors: David Austin, Mark Bauer, Mitch Flatland, Merrill Holt, and Michael Zoll.
Contributors: Lance Ashdown, Bill Bridge, Wilson Chan, Sashikanth Chandraserkaran, Sandra Cheever,
Carol Colrain, Jonathan Creighton, Sohan DeMel, Connie Dialeris, Xing Du, Mitch Flatland, Deepak
Gupta, Mike Hartstein, Eugene Ho, Wei Hu, Ashok Joshi, Bill Kehoe, Susan Kim, Jonathan Klein, Raj
Kumar, Tirthankar Lahiri, Bill Lee, Sajjad Masud, Neil Macnaughton, Rita Moran, Kotaro Ono, Kant
Patel,StefanPommerenk,MarkPorter,DarrylPresley,BrianQuigley,RebeccaReitmeyer,AnnRhee,Joao
Rimoli,PatRitto,RogerSanders,DanielSemler,VinaySrihari,AlokSrivastava,BobThome,AlexTsuker-
man, Tak Wang, Shari Yamaguchi, and Tolga Yurek.
Graphic Designer: Valarie Moore.
The Programs (which include both the software and documentation) contain proprietary information of
Oracle Corporation; they are provided under a license agreement containing restrictions on use and
disclosure and are also protected by copyright, patent, and other intellectual and industrial property
laws. Reverse engineering, disassembly, or decompilation of the Programs is prohibited.
Theinformationcontainedinthisdocumentissubjecttochangewithoutnotice.Ifyoufindanyproblems
in the documentation, please report them to us in writing. Oracle Corporation does not warrant that this
document is error free. Except as may be expressly permitted in your license agreement for these
Programs, no part of these Programs may be reproduced or transmitted in any form or by any means,
electronic or mechanical, for any purpose, without the express written permission of Oracle Corporation.
If the Programs are delivered to the U.S. Government or anyone licensing or using the programs on
behalf of the U.S. Government, the following notice is applicable:
Restricted Rights Notice Programs delivered subject to the DOD FAR Supplement are "commercial
computer software" and use, duplication, and disclosure of the Programs, including documentation,
shall be subject to the licensing restrictions set forth in the applicable Oracle license agreement.
Otherwise, Programs delivered subject to the Federal Acquisition Regulations are "restricted computer
software" and use, duplication, and disclosure of the Programs shall be subject to the restrictions in FAR
52.227-19, Commercial Computer Software - Restricted Rights (June, 1987). Oracle Corporation, 500
Oracle Parkway, Redwood City, CA 94065.
The Programs are not intended for use in any nuclear, aviation, mass transit, medical, or other inherently
dangerous applications. It shall be the licensee's responsibility to take all appropriate fail-safe, backup,
redundancy, and other measures to ensure the safe use of such applications if the Programs are used for
such purposes, and Oracle Corporation disclaims liability for any damages caused by such use of the
Programs.
Oracle is a registered trademark, and SQL*Loader, Secure Network Services, SQL*Plus, Real Application
Clusters, Oracle Call Interface, Oracle9i, Oracle8i, Oracle8, Oracle Parallel Server, Oracle Forms, Oracle
TRACE, Oracle Expert, Oracle Enterprise Manager, Oracle Server Manager, Oracle Net, Net8, PL/SQL,
and Pro*C are trademarks or registered trademarks of Oracle Corporation. Other names may be trade-
marks of their respective owners.
Contents
Send Us Your Comments.................................................................................................................. xv
Preface......................................................................................................................................................... xvii
Audience............................................................................................................................................. xviii
Organization....................................................................................................................................... xviii
Related Documentation....................................................................................................................... xx
Conventions......................................................................................................................................... xxii
Documentation Accessibility........................................................................................................... xxiv
What’s New in Oracle Cluster Software?............................................................................... xxv
Release 1 (9.0.1) Real Application Clusters Features..................................................................... xxv
New Terminology........................................................................................................................ xxvi
Installation and Configuration................................................................................................. xxvii
Cache Fusion and Resource Management................................................................................ xxx
SRVCTL Utility............................................................................................................................ xxxi
Storage.......................................................................................................................................... xxxii
Oracle Enterprise Manager....................................................................................................... xxxii
Listener Load Balancing and Failover for Dedicated Servers.............................................. xxxii
Diagnostic Features.................................................................................................................... xxxii
Discovery.................................................................................................................................... xxxiii
High Availability Features....................................................................................................... xxxiii
Shutdown Transactional Local Command............................................................................. xxxv
Quiesce Database Feature.......................................................................................................... xxxv
New SQL Scripts......................................................................................................................... xxxv
iii
TRACE_ENABLED Parameter................................................................................................. xxxv
New Parameters......................................................................................................................... xxxvi
Obsolete Parameters.................................................................................................................. xxxvi
New Views................................................................................................................................. xxxvii
Previous Cluster Software Product Features............................................................................. xxxviii
Oracle8i Release 3 (8.1.7) New Features for Oracle Parallel Server.................................. xxxviii
Oracle8i Release 2 (8.1.6) New Features for Oracle Parallel Server.................................... xxxix
Part I Cluster Database Processing Fundamentals
1 Introduction to Real Application Clusters
Prerequisite Knowledge.................................................................................................................... 1-2
Real Applications Clusters Terminology....................................................................................... 1-2
What is Real Applications Clusters?............................................................................................... 1-2
Benefits of Real Applications Clusters........................................................................................... 1-3
Scalability....................................................................................................................................... 1-3
High Availability.......................................................................................................................... 1-4
Transparency................................................................................................................................. 1-4
2 Real Application Clusters Architecture
Overview of Clustered Systems Components............................................................................... 2-2
Real Application Clusters Software Components................................................................... 2-2
Overview of Components for Clustered Systems.................................................................... 2-2
Cluster Manager.................................................................................................................................. 2-3
Failure Detection........................................................................................................................... 2-3
The Node Monitor........................................................................................................................ 2-4
Diagnosability Daemon............................................................................................................... 2-4
The Global Cache Service and Global Enqueue Service............................................................ 2-5
Transparency in the Global Cache Service and Global Enqueue Service............................. 2-5
Distributed Architecture in the Global Cache Service and Global Enqueue Service.......... 2-5
Fault Tolerance in the Global Cache Service and Global Enqueue Service.......................... 2-6
Resource Mastering in the Global Cache Service and Global Enqueue Service.................. 2-6
Resource Affinity in the Global Cache Service and Global Enqueue Service...................... 2-6
Example of Global Cache Service Processing........................................................................... 2-7
iv
Interaction of the Global Cache Service and Global Enqueue Service with the Cluster
Manager 2-7
Cluster Interconnect and Interprocess Communication (Node-to-Node)............................... 2-7
Disk Subsystems................................................................................................................................. 2-8
3 Cluster Hardware Architecture
Overview of Cluster Hardware Components................................................................................ 3-2
Node Components.............................................................................................................................. 3-2
Memory, Interconnect, and Storage................................................................................................. 3-2
Memory Access............................................................................................................................. 3-2
The High Speed IPC Interconnect................................................................................................... 3-4
Storage Access in Clustered Systems.............................................................................................. 3-5
Uniform Disk Access.................................................................................................................... 3-5
Non-Uniform Disk Access........................................................................................................... 3-7
Clusters: Nodes and the Interconnect............................................................................................. 3-8
Interoperability with Other Systems.............................................................................................. 3-8
Part II Resource Coordination in Real Application Clusters
4 Local Resource Coordination
Overview of Resource Coordination.............................................................................................. 4-2
Resource Coordination Components.............................................................................................. 4-3
Local Concurrency Controls............................................................................................................. 4-3
Latches............................................................................................................................................ 4-3
Local Enqueues............................................................................................................................. 4-4
Row Locks...................................................................................................................................... 4-4
5 Cache Fusion and the Global
Cache Service
Overview of Cache Fusion................................................................................................................ 5-2
Global Cache Service Operations............................................................................................... 5-3
Cache Coherency.......................................................................................................................... 5-3
GCS Resource Modes and Roles..................................................................................................... 5-4
Resource Modes............................................................................................................................ 5-4
v
Resource Roles............................................................................................................................... 5-5
Write Protocol and Past Image Tracking......................................................................................... 5-5
Real Application Clusters Resource Control Mechanisms......................................................... 5-6
Eliminating the Need for Configuring Resources........................................................................ 5-7
Resource Control, Cache-to-Cache Transfer, and Cache Coherency......................................... 5-7
Cache Fusion Resource Assignment and Block Coverage.......................................................... 5-8
Block Access Modes and Buffer States...................................................................................... 5-8
How Buffer States and Block Access Modes Change.............................................................. 5-9
Block Access Modes Can Be Compatible or Incompatible................................................... 5-10
Cache Fusion Scenarios................................................................................................................... 5-10
Requesting a Block for a Read from Another Instance: Scenario........................................ 5-10
Requesting a Changed Block for Modification: Scenario...................................................... 5-12
Writing Blocks to Disk: Scenario.............................................................................................. 5-13
How the GCS Grants and Coordinates Resource Requests..................................................... 5-15
Interrupt and Completion Processing..................................................................................... 5-15
Block Access Requests are Queued.......................................................................................... 5-16
Acquisition Interrupts Communicate Block Access Request Status................................... 5-16
Block Access Requests are Granted and Converted.............................................................. 5-17
Recovery in Real Applications Clusters....................................................................................... 5-19
6 Coordination by the Global Enqueue Service
Cache Synchronization...................................................................................................................... 6-2
Global Enqueue Service Concurrency Control............................................................................. 6-2
Concurrency for Database Global Enqueues................................................................................ 6-3
Global Enqueue Service Processing................................................................................................ 6-3
Transaction Locks......................................................................................................................... 6-4
Table Locks.................................................................................................................................... 6-4
Library Cache Locks..................................................................................................................... 6-5
Dictionary Cache Locks .............................................................................................................. 6-5
Database Mount Lock ................................................................................................................. 6-6
Deadlock Detection in the Global Enqueue Service................................................................... 6-6
vi
Part III Implementing Real Application Clusters
7 Real Application Clusters Components
Instance and Database Components for Real Application Clusters........................................ 7-2
Real Application Clusters Processes.......................................................................................... 7-2
Cache Fusion Processing............................................................................................................. 7-3
Overview of Real Application Clusters Processes................................................................... 7-5
System Change Number Processing............................................................................................... 7-6
Lamport SCN Generation............................................................................................................ 7-6
8 Real Application Clusters Storage Considerations
Storage Issues...................................................................................................................................... 8-2
Datafile Storage in Real Application Clusters.......................................................................... 8-2
Parameter File Storage in Real Application Clusters.............................................................. 8-3
Redo Log File Storage in Real Application Clusters............................................................... 8-4
Rollback Segments in Real Application Clusters..................................................................... 8-5
Space Management and Free List Groups..................................................................................... 8-6
How Oracle Handles Free Space................................................................................................ 8-6
Free Lists and Free List Groups.................................................................................................. 8-7
Free List Group Examples............................................................................................................... 8-10
Basic Free List Group Example................................................................................................. 8-10
Complex Free List Group Example.......................................................................................... 8-12
Controlling Extent Allocation........................................................................................................ 8-12
Automatic Allocation of New Extents..................................................................................... 8-13
9 Scalability in Real Application Clusters
Scalability Features of Real Application Clusters........................................................................ 9-2
Enhanced Throughput: Scale Up............................................................................................... 9-2
Speed-Up and Scale Up: The Goals of Cluster Database Processing.................................... 9-3
Where are Cluster Databases Advantageous?............................................................................... 9-5
Transaction Systems..................................................................................................................... 9-5
Decision Support Systems........................................................................................................... 9-6
Oracle Parallel Execution on Real Application Clusters............................................................ 9-6
Levels of Scalability........................................................................................................................... 9-7
vii
Network Scalability...................................................................................................................... 9-7
Operating System Scalability.................................................................................................... 9-10
Database Management System Scalability.............................................................................. 9-10
Part IV High Availability
10 High Availability Concepts and Best Practices
Understanding High Availability.................................................................................................. 10-2
Measuring Availability.............................................................................................................. 10-2
High Availability Measurements............................................................................................. 10-3
Causes of Downtime.................................................................................................................. 10-3
Planning for High Availability...................................................................................................... 10-6
System Level Planning............................................................................................................... 10-6
Configuring Real Application Clusters for High Availability................................................. 10-7
Cluster Components and High Availability........................................................................... 10-7
Disaster Planning.............................................................................................................................. 10-9
Failure Protection Validation.......................................................................................................... 10-9
Failover and Real Application Clusters...................................................................................... 10-10
Failover Basics........................................................................................................................... 10-10
Client Failover........................................................................................................................... 10-11
Uses of Transparent Application Failover............................................................................ 10-12
Server Failover.......................................................................................................................... 10-16
How Failover Works....................................................................................................................... 10-17
Detecting Failure....................................................................................................................... 10-18
Reorganizing Cluster Membership........................................................................................ 10-18
Performing Database Recovery.............................................................................................. 10-19
High Availability Configurations................................................................................................ 10-22
Default n-node Configuration................................................................................................. 10-22
Basic High Availability Configurations................................................................................. 10-22
Shared High Availability Node Configuration.................................................................... 10-29
Toward Deploying High Availability......................................................................................... 10-30
viii
11 Oracle Real Application Clusters Guard Architecture
Overview of Oracle Real Application Clusters Guard Components..................................... 11-2
Packs............................................................................................................................................. 11-3
PFSCTL Control Utility.............................................................................................................. 11-3
Oracle Real Application Clusters Guard Monitors............................................................... 11-4
Oracle Real Application Clusters Guard Configuration Templates................................... 11-4
PFS Setup Utility......................................................................................................................... 11-4
Concepts of Oracle Real Application Clusters Guard............................................................... 11-5
Instance Roles.............................................................................................................................. 11-5
Preferred Primary and Secondary Nodes............................................................................... 11-5
Home and Foreign Nodes......................................................................................................... 11-6
Architecture of Oracle Real Application Clusters Guard......................................................... 11-6
Packs............................................................................................................................................. 11-8
Monitors....................................................................................................................................... 11-9
Additional Configurations of Oracle Real Application Clusters Guard............................. 11-10
Hub Configuration................................................................................................................... 11-11
Ring Configuration................................................................................................................... 11-13
12 Oracle Real Application Clusters Guard Operation
Overview of Oracle Real Application Clusters Guard Operation.......................................... 12-2
Failure of the Primary Instance...................................................................................................... 12-2
Restoring the Nodes to their Original Roles............................................................................... 12-4
Failure of the Secondary Instance................................................................................................. 12-6
Failure of Both Instances................................................................................................................. 12-7
Part V Reference
A Restrictions
Compatibility Between Shared and Exclusive Mode.................................................................. A-2
Export and Import Utilities......................................................................................................... A-2
Mode Compatibility..................................................................................................................... A-2
Restrictions........................................................................................................................................... A-2
Maximum Number of Blocks Allocated at a Time.................................................................. A-2
Restrictions in Cluster Database Mode..................................................................................... A-3
ix
B Using Multi-Block Lock Assignments (Optional)
When to Use Locks.............................................................................................................................. B-2
How to Use Locks................................................................................................................................ B-2
Lock Granularity................................................................................................................................. B-3
1:1 Locks......................................................................................................................................... B-3
1:n Locks......................................................................................................................................... B-3
Understanding Lock Management.................................................................................................. B-4
Example of Locks Covering Multiple Blocks............................................................................ B-5
Periodicity of Locks...................................................................................................................... B-6
Forced Writing: Signaling the Need to Update........................................................................ B-7
................................................................................................................................................................. B-8
Glossary
Index
x