IBM Spectrum Scale Advanced Admin Guide

396 Pages·2015·3.56 MB·Romanian

IBM Spectrum Scale
Version 4 Release 1.1
Advanced Administration Guide
IBM
SC23-7032-02

IBM Spectrum Scale
Version 4 Release 1.1
Advanced Administration Guide
IBM
SC23-7032-02
Note
Before using this information and the product it supports, read the information in “Notices” on page 353.
This edition applies to version 4 release 1 modification 1 of the following products, and to all subsequent releases
and modifications until otherwise indicated in new editions:
v IBM Spectrum Scale ordered through Passport Advantage®(product number 5725-Q01)
v IBM Spectrum Scale ordered through AAS/eConfig (product number 5641-GPF)
v IBM Spectrum Scale for Linux on z Systems™(product number 5725-S28)
Significant changes or additions to the text and illustrations are indicated by a vertical line (|) to the left of the
change.
IBM welcomes your comments; see the topic “How to send your comments” on page xiv. When you send
information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes
appropriate without incurring any obligation to you.
© Copyright IBM Corporation 2014, 2015.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
Contents
Figures . . . . . . . . . . . . .. vii Chapter 3. Creating and maintaining
snapshots of GPFS file systems. . .. 65
Tables . . . . . . . . . . . . . .. ix Creating your GPFS snapshot . . . . . . .. 65
Listing GPFS snapshots . . . . . . . . .. 66
About this information . . . . . . .. xi Restoring a GPFS file system from a snapshot. .. 67
Using the policy engine to read a snapshot. . .. 68
Prerequisite and related information. . . . .. xiii
Linking to your GPFS snapshots . . . . . .. 68
Conventions used in this information . . . .. xiii
Deleting your GPFS snapshot . . . . . . .. 69
How to send your comments . . . . . . .. xiv
Chapter 4. Creating and managing file
Summary of changes . . . . . . .. xv
clones . . . . . . . . . . . . . .. 71
Chapter 1. Accessing GPFS file systems Creating file clones. . . . . . . . . . .. 71
Listing file clones . . . . . . . . . . .. 72
from other GPFS clusters . . . . . .. 1
Deleting file clones. . . . . . . . . . .. 73
User access to a GPFS file system owned by another
Splitting file clones from clone parents . . . .. 73
GPFS cluster . . . . . . . . . . . . .. 3
File clones and disk space management . . . .. 73
Mounting a file system owned and served by another
File clones and snapshots. . . . . . . . .. 73
GPFS cluster . . . . . . . . . . . . .. 4
File clones and policy files . . . . . . . .. 74
Managing remote access to GPFS file systems . .. 6
Using remote access with public and private IP
Chapter 5. Scale Out Backup and
addresses . . . . . . . . . . . . . .. 7
Restore (SOBAR) . . . . . . . . .. 75
Using multiple security levels . . . . . . .. 9
Changing security keys . . . . . . . . .. 10 Backup procedure with SOBAR. . . . . . .. 75
NIST compliance . . . . . . . . . . .. 11 Restore procedure with SOBAR. . . . . . .. 76
Additional information about GPFS file systems
accessed by nodes that belong to other GPFS Chapter 6. Establishing disaster
clusters. . . . . . . . . . . . . . .. 11 recovery for your GPFS cluster . . .. 79
Synchronous mirroring utilizing GPFS replication . 80
Chapter 2. Information Lifecycle
Setting up a GPFS cluster with synchronous
Management for GPFS . . . . . . .. 13 mirroring utilizing GPFS replication . . . .. 81
Storage pools. . . . . . . . . . . . .. 13 Steps to take after a disaster when using GPFS
Internal GPFS storage pools . . . . . . .. 14 replication. . . . . . . . . . . . .. 83
External storage pools. . . . . . . . .. 19 Synchronous mirroring utilizing IBM TotalStorage
Policies and rules . . . . . . . . . . .. 20 ESS PPRC . . . . . . . . . . . . . .. 88
Policies. . . . . . . . . . . . . .. 20 An active/active GPFS cluster . . . . . .. 89
Policy rules . . . . . . . . . . . .. 21 An active/passive GPFS cluster . . . . .. 93
Semantics of the mmapplypolicy command and Data integrity and the use of PPRC consistency
its policy rules . . . . . . . . . . .. 38 groups . . . . . . . . . . . . . .. 98
Policy rules - examples and tips . . . . .. 42 Asynchronous mirroring utilizing ESS FlashCopy.. 99
Managing policies . . . . . . . . . .. 46 Setting up FlashCopy using file-system-level
Working with external storage pools . . . .. 48 suspension . . . . . . . . . . . .. 103
Backup and restore with storage pools . . .. 54
Filesets. . . . . . . . . . . . . . .. 55 Chapter 7. Implementing a clustered
Fileset namespace . . . . . . . . . .. 56 NFS using GPFS on Linux. . . . .. 105
Filesets and quotas. . . . . . . . . .. 56 NFS monitoring . . . . . . . . . . .. 105
Filesets and storage pools . . . . . . .. 57 NFS failover. . . . . . . . . . . . .. 105
Filesets and global snapshots . . . . . .. 57 NFS locking and load balancing . . . . . .. 105
Fileset-level snapshots. . . . . . . . .. 58 CNFS network setup. . . . . . . . . .. 106
Filesets and backup. . . . . . . . . .. 58 CNFS setup . . . . . . . . . . . . .. 106
Managing filesets . . . . . . . . . .. 59 CNFS administration. . . . . . . . . .. 107
Immutability and appendOnly restrictions . . .. 61
Chapter 8. Implementing Cluster
Export Services . . . . . . . . .. 109
CES features. . . . . . . . . . . . .. 109
© Copyright IBM Corp. 2014, 2015 iii
CES cluster setup . . . . . . . . . .. 109 Changing the request histogram facility request
CES network configuration. . . . . . .. 110 size and latency ranges . . . . . . . .. 171
CES address failover and distribution policies 112 Disabling the request histogram facility . .. 173
CES protocol management . . . . . . .. 112 Enabling the request histogram facility. . .. 174
CES management and administration . . .. 113 Displaying the request histogram facility pattern 174
CES NFS support . . . . . . . . . . .. 113 Resetting the request histogram facility data to
CES SMB support . . . . . . . . . . .. 115 zero . . . . . . . . . . . . . .. 177
CES OBJ support . . . . . . . . . . .. 116 Displaying the request histogram facility
CES monitoring and troubleshooting . . . .. 118 statistics values. . . . . . . . . . .. 178
CES tracing and debug data collection . . . .. 119 Understanding the Remote Procedure Call (RPC)
Migration of CNFS clusters to CES clusters . .. 122 facility. . . . . . . . . . . . . . .. 180
Displaying the aggregation of execution time for
Chapter 9. Managing Object storage 125 Remote Procedure Calls (RPCs) . . . . .. 180
Displaying the Remote Procedure Call (RPC)
Understanding and managing Object services .. 125
execution time according to the size of messages 182
Understanding the mapping of OpenStack
Displaying mmpmon version . . . . . . .. 183
commands to IBM Spectrum Scale administrator
commands . . . . . . . . . . . . .. 126 Example of mmpmon ver request . . . .. 183
Changing Object configuration values . . . .. 127 Example mmpmon scenarios and how to analyze
and interpret their results . . . . . . . .. 183
Changing the Object base configuration to enable
Amazon S3 . . . . . . . . . . . . .. 127 fs_io_s and io_s output - how to aggregate and
analyze the results. . . . . . . . . .. 184
Configuring OpenStack EC2 credentials . . .. 129
Request histogram (rhist) output - how to
Managing OpenStack access control lists via the
aggregate and analyze the results. . . . .. 186
Amazon S3 API . . . . . . . . . . .. 130
Using request sourceand prefix directive once 186
Chapter 10. Performance monitoring 133 Other information about mmpmon output. . .. 192
Counter sizes and counter wrapping . . .. 192
Network performance monitoring . . . . .. 133
Return codes from mmpmon . . . . . .. 192
Performance Monitoring tool overview. . . .. 135
Configuring the Performance Monitoring tool 135
Chapter 12. GPFS SNMP support . .. 195
Restarting the Performance Monitoring tool .. 137
Installing Net-SNMP . . . . . . . . . .. 195
Defining the metrics . . . . . . . . .. 138
Configuring Net-SNMP . . . . . . . . .. 196
Performance monitoring. . . . . . . .. 138
Configuring management applications . . . .. 196
List of queries . . . . . . . . . . .. 139
Installing MIB files on the collector node and
List of metrics . . . . . . . . . . .. 143
management node. . . . . . . . . . .. 197
Chapter 11. Monitoring GPFS I/O Collector node administration. . . . . . .. 197
Starting and stopping the SNMP subagent. . .. 198
performance with the mmpmon
The management and monitoring subagent . .. 198
command . . . . . . . . . . . .. 157
SNMP object IDs . . . . . . . . . .. 199
Overview of mmpmon . . . . . . . . .. 157 MIB objects . . . . . . . . . . . .. 199
Specifying input to the mmpmon command . .. 157 Cluster status information . . . . . . .. 199
Running mmpmon on multiple nodes . . .. 158 Cluster configuration information . . . .. 199
Running mmpmon concurrently from multiple Node status information. . . . . . . .. 200
users on the same node . . . . . . . .. 158 Node configuration information . . . . .. 200
Display I/O statistics per mounted file system .. 159 File system status information. . . . . .. 201
Example of mmpmon fs_io_s request . . .. 159 File system performance information . . .. 202
Display I/O statistics for the entire node . . .. 160 Storage pool information . . . . . . .. 202
Example of mmpmon io_s request . . . .. 161 Disk status information . . . . . . . .. 203
Understanding the node list facility . . . . .. 161 Disk configuration information . . . . .. 203
Add node names to a list of nodes for mmpmon Disk performance information. . . . . .. 204
processing . . . . . . . . . . . .. 162 Net-SNMP traps . . . . . . . . . .. 204
Delete a node list . . . . . . . . . .. 163
Create a new node list . . . . . . . .. 164 Chapter 13. Identity management on
Show the contents of the current node list. .. 164 Windows . . . . . . . . . . . .. 207
Delete node names from a list of nodes for
Auto-generated ID mappings . . . . . . .. 207
mmpmon processing. . . . . . . . .. 165
Installing Windows IMU . . . . . . . .. 207
Node list examples and error handling. . .. 165
Configuring ID mappings in IMU . . . . .. 208
Reset statistics to zero . . . . . . . . .. 168
Example of mmpmon reset request . . . .. 169
Chapter 14. Active file management 211
Understanding the request histogram facility. .. 169
Specifying the size ranges for I/O histograms 169 Active file management architecture. . . . .. 211
Specifying the latency ranges for I/O . . .. 170 Caching modes. . . . . . . . . . . .. 212
iv IBM Spectrum Scale 4.1.1: Advanced Administration Guide
File system caching and synchronization . . .. 214 Protocol and cluster configuration data required for
Asynchronous operations . . . . . . .. 214 disaster recovery . . . . . . . . . . .. 261
Synchronous operations. . . . . . . .. 214 Collecting the object data required for disaster
Update synchronization. . . . . . . .. 214 recovery . . . . . . . . . . . . .. 261
Components of a cluster that is running AFM .. 215 Collecting the SMB data required for disaster
Global namespace. . . . . . . . . . .. 216 recovery . . . . . . . . . . . . .. 264
Cache eviction . . . . . . . . . . . .. 216 Collecting the NFS data required for disaster
Disconnected operation . . . . . . . . .. 217 recovery . . . . . . . . . . . . .. 266
Expiration . . . . . . . . . . . . .. 217 Collecting authentication related data required
Cache states. . . . . . . . . . . . .. 218 for disaster recovery . . . . . . . . .. 269
Failure and recovery . . . . . . . . . .. 220 Collecting the CES data required for disaster
Steps to deal with an IW cache fileset disaster .. 220 recovery . . . . . . . . . . . . .. 272
Prefetching . . . . . . . . . . . . .. 222 Collecting GPFS cluster configuration data
Peer snapshots . . . . . . . . . . . .. 222 required for disaster recovery . . . . . .. 273
Viewing snapshots at home . . . . . . .. 222 Setting up the DR cluster for disaster recovery .. 273
Failover of cache filesets. . . . . . . . .. 223 Setting up independent filesets for DR that
Partial file caching. . . . . . . . . . .. 224 correspond to independent filesets on the
Parallel I/O . . . . . . . . . . . . .. 224 primary cluster. . . . . . . . . . .. 274
Disabling AFM. . . . . . . . . . . .. 225 Converting independent filesets on the primary
AFM-based NFS migration. . . . . . . .. 225 cluster to primary AFM DR filesets . . . .. 275
Home cluster errors . . . . . . . . . .. 228 Converting independent filesets on the DR
Administrative actions . . . . . . . . .. 228 cluster to secondary AFM DR filesets . . .. 276
Dealing with requeued messages. . . . .. 228 Scheduling regular configuration updates . .. 276
Handling inadvertent changes at home for SW 228 Using the sample script for automated DR setup 277
Tuning active file management home Failover in protocols disaster recovery . . . .. 282
communications . . . . . . . . . . .. 229 Saving current DR cluster NFS configuration 283
System requirements and setup of a home and Promoting protected independent filesets to
cache . . . . . . . . . . . . . . .. 232 primary . . . . . . . . . . . . .. 283
Setting up home and cache clusters . . . .. 232 Restoring the object configuration . . . .. 283
Restoring the SMB configuration . . . . .. 286
Chapter 15. AFM-based disaster Removing false SMB exports . . . . . .. 286
recovery . . . . . . . . . . . .. 235 Restoring the NFS configuration . . . . .. 287
Removing false NFS exports . . . . . .. 287
Introduction. . . . . . . . . . . . .. 235
Restoring required NFS AFM DR exports . .. 287
RTO . . . . . . . . . . . . . . .. 236
Using the sample script for automated failover 287
Creation and using an async DR relationship. .. 236
Failback or restore in protocols disaster recovery 293
Conversion of GPFS filesets to primary/secondary 237
Protocol cluster failback. . . . . . . .. 293
Conversion of AFM filesets to primary. . . .. 238
Protocol configuration backup. . . . . .. 294
RPO snapshots. . . . . . . . . . . .. 240
Using the sample script for automated protocol
Failover to secondary. . . . . . . . . .. 241
configuration backup. . . . . . . . .. 294
Failback to old primary . . . . . . . . .. 241
Protocol configuration restore . . . . . .. 295
Failback to new primary. . . . . . . . .. 242
Using the sample script for automated protocol
Change secondary. . . . . . . . . . .. 242
configuration restore . . . . . . . . .. 295
Cache states. . . . . . . . . . . . .. 243
Use cases descriptions . . . . . . . . .. 243
Chapter 17. GPFS File Placement
Optimizer . . . . . . . . . . . .. 299
Chapter 16. Protocols disaster
recovery . . . . . . . . . . . .. 257 Distributing data across a cluster. . . . . .. 302
FPO pool file placement and AFM . . . . .. 303
Protocols disaster recovery limitations and
Restrictions . . . . . . . . . . . . .. 303
prerequisites. . . . . . . . . . . . .. 257
Example setup for protocols disaster recovery .. 257
Chapter 18. Hadoop support for GPFS 305
Sample script for protocols disaster recovery. .. 258
Setting up gateway nodes to ensure cluster
communication during failover . . . . . .. 259 Chapter 19. Encryption . . . . . .. 311
Setting up independent fileset for protocol and Encryption keys . . . . . . . . . . .. 311
cluster configuration information. . . . . .. 260 Encryption policies . . . . . . . . . .. 312
Determining the file system containing the CES Encryption policy rules . . . . . . . . .. 312
shared root . . . . . . . . . . . .. 260 Encryption setup requirements . . . . . .. 317
Creating new independent fileset and Establishing an encryption-enabled environment 319
subdirectories for configuration information .. 261 Secure deletion. . . . . . . . . . . .. 323
Encryption and FIPS compliance . . . . . .. 324
Contents v
Encryption and NIST compliance. . . . . .. 325 Using multiple token servers . . . . . . .. 342
Encryption and backup/restore . . . . . .. 325 Exporting file system definitions between clusters 343
Encryption and snapshots . . . . . . . .. 325 GPFS port usage . . . . . . . . . . .. 343
Securing the IBM Spectrum Scale system using
Chapter 20. Securing protocol data 327 firewall . . . . . . . . . . . . . .. 345
Planning for protocol data security . . . . .. 329 Firewall recommendations for the IBM
Configuring protocol data security . . . . .. 329 Spectrum Scale installation. . . . . . .. 345
Enabling secured connection between the IBM Firewall recommendations for internal
Spectrum Scale system and authentication communication among nodes . . . . . .. 346
server . . . . . . . . . . . . . .. 329 Firewall recommendations for protocol access 346
Securing data transfer . . . . . . . .. 332 Firewall recommendations for Performance
Securing NFS data transfer. . . . . . .. 333 Monitoring tool . . . . . . . . . .. 350
Securing SMB data transfer. . . . . . .. 335
Accessibility features for IBM
Secured object data transfer . . . . . .. 335
Data security limitations. . . . . . . . .. 335 Spectrum Scale . . . . . . . . .. 351
Accessibility features. . . . . . . . . .. 351
Chapter 21. Highly-available write Keyboard navigation. . . . . . . . . .. 351
cache (HAWC) . . . . . . . . . .. 337 IBM and accessibility. . . . . . . . . .. 351
Applications that can benefit from HAWC. . .. 338
Notices . . . . . . . . . . . . .. 353
Restrictions and tuning recommendations for
HAWC . . . . . . . . . . . . . .. 338 Trademarks . . . . . . . . . . . . .. 355
Using HAWC . . . . . . . . . . . .. 339 Terms and conditions for product documentation 355
IBM Online Privacy Statement. . . . . . .. 356
Chapter 22. Miscellaneous advanced
administration topics . . . . . . .. 341 Glossary . . . . . . . . . . . .. 357
Changing IP addresses and host names. . . .. 341
Enabling a cluster for IPv6 . . . . . . . .. 342 Index . . . . . . . . . . . . . .. 363
vi IBM Spectrum Scale 4.1.1: Advanced Administration Guide
Figures
1. Remote mount of a file system using NSD 8. Violation of write ordering without the use of
server access . . . . . . . . . . .. 2 a PPRC consistency group . . . . . .. 99
2. Remote mount of a file system using 9. High-level organization of a
SAN-attached disks . . . . . . . . .. 2 FlashCopy/PPRC recovery environment .. 102
3. Multi-cluster configuration with multiple NSD 10. Node running mmpmon. . . . . . .. 157
servers . . . . . . . . . . . . .. 3 11. Properties window. . . . . . . . .. 208
4. Use of public and private IP addresses in three 12. Global namespace implemented using AFM 216
GPFS clusters . . . . . . . . . . .. 9 13. GPFS Hadoop connector overview . . .. 306
5. Synchronous mirroring utilizing GPFS 14. Hadoop on centralized storage. . . . .. 307
replication . . . . . . . . . . . .. 81 15. Hadoop on Local storage (FPO) . . . .. 307
6. A synchronous active/active PPRC-based 16. Implementation of data in transit security for
mirrored GPFS configuration with a tiebreaker protocol data. . . . . . . . . . .. 327
site . . . . . . . . . . . . . .. 90
7. A synchronous active/passive PPRC-based
GPFS configuration without a tiebreaker site . 94
© Copyright IBM Corp. 2014, 2015 vii
viii IBM Spectrum Scale 4.1.1: Advanced Administration Guide