System Administrator Guide - Greenplum Admins

99 Pages·2013·2.02 MB·English

by  Pivotal Information Development

PRODUCT DOCUMENTATION
Greenplum Database
®
Version 4.3
System Administrator Guide
Rev: A01
© 2013 GoPivotal, Inc.
Copyright © 2013 GoPivotal, Inc. All rights reserved.
GoPivotal, Inc. believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." GOPIVOTAL, INC. ("Pivotal") MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any Pivotal software described in this publication requires an applicable
software license.
All trademarks used herein are the property of Pivotal or their respective owners.
Revised November 2013 (4.3.0.0)
Greenplum Database System Administrator Guide 4.3 – Contents
Greenplum Database System Administrator Guide - 4.3 -
Contents
Preface...............................................................................................1
About This Guide..............................................................................1
About the Greenplum Database Documentation Set.........................1
Document Conventions....................................................................2
Text Conventions........................................................................2
Command Syntax Conventions...................................................3
Getting Support ...............................................................................3
Product information....................................................................3
Technical support .......................................................................4
Chapter 1: About the Greenplum Architecture........................5
About the Greenplum Master............................................................6
About the Greenplum Segments.......................................................6
About the Greenplum Interconnect..................................................6
About Redundancy and Failover in Greenplum Database..................7
About Segment Mirroring............................................................7
About Master Mirroring...............................................................8
About Interconnect Redundancy.................................................8
About Parallel Data Loading.............................................................9
About Management and Monitoring..................................................9
Chapter 2: Starting and Stopping Greenplum ........................11
Overview.........................................................................................11
Starting Greenplum Database.........................................................11
Restarting Greenplum Database................................................11
Uploading Configuration File Changes Only................................12
Starting the Master in Maintenance Mode..................................12
Stopping Greenplum Database........................................................12
Chapter 3: Configuring Your Greenplum System...................14
About Greenplum Master and Local Parameters..............................14
Setting Configuration Parameters....................................................14
Setting a Local Configuration Parameter....................................14
Setting a Master Configuration Parameter.................................15
Viewing Server Configuration Parameter Settings ...........................16
Configuration Parameter Categories................................................16
Connection and Authentication Parameters................................17
System Resource Consumption Parameters...............................17
Query Tuning Parameters..........................................................18
Error Reporting and Logging Parameters...................................20
System Monitoring Parameters..................................................20
Runtime Statistics Collection Parameters...................................21
Automatic Statistics Collection Parameters................................21
Client Connection Default Parameters........................................22
Lock Management Parameters...................................................22
Workload Management Parameters............................................22
External Table Parameters.........................................................23
Append-Optimized Table Parameters.........................................23
iii
Greenplum Database System Administrator Guide 4.3 – Contents
Database and Tablespace/Filespace Parameters........................23
Past PostgreSQL Version Compatibility Parameters....................23
Greenplum Array Configuration Parameters...............................23
Greenplum Master Mirroring Parameters....................................24
Chapter 4: Enabling High Availability Features......................25
Overview of High Availability in Greenplum Database......................25
Overview of Segment Mirroring.................................................25
Overview of Master Mirroring.....................................................26
Overview of Fault Detection and Recovery.................................27
Enabling Mirroring in Greenplum Database......................................27
Enabling Segment Mirroring.......................................................27
Enabling Master Mirroring..........................................................28
Detecting a Failed Segment.............................................................29
Enabling Alerts and Notifications................................................29
Checking for Failed Segments....................................................30
Checking the Log Files...............................................................30
Recovering a Failed Segment..........................................................31
Recovering From Segment Failures............................................32
Recovering a Failed Master..............................................................35
Restoring Master Mirroring After a Recovery..............................35
Chapter 5: Backing Up and Restoring Databases..................37
Backup and Restore Operations.......................................................37
Parallel Backup Support.............................................................37
Non-Parallel Backup Support.....................................................38
Parallel Restores........................................................................38
Non-Parallel Restores ................................................................39
Backing Up a Database...................................................................39
Incremental Backup Support.....................................................40
Using Direct I/O.........................................................................42
Using Data Domain Boost..........................................................43
Using Named Pipes....................................................................46
Backing Up a Database with gp_dump.......................................47
Automating Parallel Backups with gpcrondump..........................48
Restoring From Parallel Backup Files...............................................49
Restoring a Database with gp_restore.......................................50
Restoring a Database Using gpdbrestore...................................51
Restoring to a Different Greenplum System Configuration.........51
Chapter 6: Expanding a Greenplum System............................53
Planning Greenplum System Expansion...........................................53
System Expansion Overview......................................................53
System Expansion Checklist ......................................................55
Planning New Hardware Platforms.............................................56
Planning New Segment Initialization..........................................56
Planning Table Redistribution.....................................................58
Preparing and Adding Nodes...........................................................60
Adding New Nodes to the Trusted Host Environment.................60
Verifying OS Settings.................................................................61
Validating Disk I/O and Memory Bandwidth...............................62
iv
Greenplum Database System Administrator Guide 4.3 – Contents
Integrating New Hardware into the System...............................62
Initializing New Segments...............................................................62
Creating an Input File for System Expansion.............................62
Running gpexpand to Initialize New Segments ..........................65
Rolling Back an Failed Expansion Setup.....................................66
Redistributing Tables.......................................................................66
Ranking Tables for Redistribution ..............................................66
Redistributing Tables Using gpexpand........................................67
Monitoring Table Redistribution..................................................67
Removing the Expansion Schema....................................................68
Chapter 7: Monitoring a Greenplum System...........................69
Monitoring Database Activity and Performance................................69
Monitoring System State.................................................................69
Enabling System Alerts and Notifications...................................70
Checking System State..............................................................76
Checking Disk Space Usage.......................................................77
Checking for Data Distribution Skew..........................................78
Viewing Metadata Information about Database Objects.............79
Viewing Query Workfile Usage Information................................80
Viewing the Database Server Log Files............................................80
Log File Format..........................................................................80
Searching the Greenplum Database Server Log Files.................82
Using gp_toolkit..............................................................................82
Chapter 8: Routine System Maintenance Tasks.....................83
Routine Vacuum and Analyze..........................................................83
Transaction ID Management......................................................83
System Catalog Maintenance.....................................................84
Vacuum and Analyze for Query Optimization.............................85
Routine Reindexing.........................................................................85
Managing Greenplum Database Log Files........................................86
Database Server Log Files.........................................................86
Management Utility Log Files.....................................................86
Chapter 9: Kerberos Authentication..........................................87
Requirements for using Kerberos with Greenplum Database...........88
Installing and Configuring a Kerberos KDC Server...........................89
Creating Greenplum Database Roles in the KDC Database.........89
Installing and Configuring the Kerberos Client.................................90
Setting up Greenplum Database with Kerberos for PSQL...........91
Setting up Greenplum Database with Kerberos for JDBC ...........92
Sample Kerberos Configuration File.................................................93
krb5.conf Configuration File.......................................................93
v
Greenplum Database System Administrator Guide 4.3 – Preface
Preface
This guide provides information for system administrators responsible for
administering a Greenplum Database system.
• About This Guide
• Document Conventions
• Getting Support
About This Guide
This guide describes system administration tasks for Greenplum Database, such as
configuring the server, monitoring system activity, enabling high-availability, backing
up and restoring databases, and other routine system administration tasks.
This guide assumes knowledge of Linux/UNIX system administration and database
management systems. Familiarity with structured query language (SQL) is helpful.
Because Greenplum Database is based on PostgreSQL 8.2.15, this guide assumes
some familiarity with PostgreSQL. References to PostgreSQL documentation are
provided throughout this guide for features that are similar to those in Greenplum
Database.
About the Greenplum Database Documentation Set
The Greenplum Database 4.3 documentation set consists of the following guides.
Table 1 Greenplum Database documentation set
Guide Name Description
Greenplum Database Database Every day DBA tasks such as configuring access control and
Administrator Guide workload management, writing queries, managing data,
defining database objects, and performance troubleshooting.
Greenplum Database System Describes the Greenplum Database architecture and concepts
Administrator Guide such as parallel processing, and system administration tasks
for Greenplum Database such as configuring the server,
monitoring system activity, enabling high-availability, backing
up and restoring databases, and expanding the system.
Greenplum Database Reference Reference information for Greenplum Database systems: SQL
Guide commands, system catalogs, environment variables, character
set support, datatypes, the Greenplum MapReduce
specification, postGIS extension, server parameters, the
gp_toolkit administrative schema, and SQL 2008 support.
Greenplum Database Utility Reference information for command-line utilities, client
Guide programs, and Oracle compatibility functions.
Greenplum Database Information and instructions for installing and initializing a
Installation Guide Greenplum Database system.
About This Guide 1
Greenplum Database System Administrator Guide 4.3 – Preface
Document Conventions
The following conventions are used throughout the Greenplum Database
documentation to help you identify certain types of information.
• Text Conventions
• Command Syntax Conventions
Text Conventions
Table 2 Text Conventions
Text Convention Usage Examples
bold Button, menu, tab, page, and field Click Cancel to exit the page without
names in GUI applications saving your changes.
italics New terms where they are defined The master instance is the postgres
process that accepts client
Database objects, such as schema,
connections.
table, or columns names
Catalog information for Greenplum
Database resides in the pg_catalog
schema.
monospace File names and path names Edit the postgresql.conf file.
Programs and executables Use gpstart to start Greenplum
Database.
Command names and syntax
Parameter names
monospace italics Variable information within file /home/gpadmin/config_file
paths and file names
COPY tablename FROM
Variable information within 'filename'
command syntax
monospace bold Used to call attention to a particular Change the host name, port, and
part of a command, parameter, or database name in the JDBC
code snippet. connection URL:
jdbc:postgresql://host:5432/m
ydb
UPPERCASE Environment variables Make sure that the Java /bin
directory is in your $PATH.
SQL commands
SELECT * FROM my_table;
Keyboard keys
Press CTRL+C to escape.
Document Conventions 2
Greenplum Database System Administrator Guide 4.3 – Preface
Command Syntax Conventions
Table 3 Command Syntax Conventions
Text Convention Usage Examples
{ } Within command syntax, curly FROM { 'filename' | STDIN }
braces group related command
options. Do not type the curly
braces.
[ ] Within command syntax, square TRUNCATE [ TABLE ] name
brackets denote optional
arguments. Do not type the
brackets.
... Within command syntax, an ellipsis DROP TABLE name [, ...]
denotes repetition of a command,
variable, or option. Do not type the
ellipsis.
| Within command syntax, the pipe VACUUM [ FULL | FREEZE ]
symbol denotes an “OR”
relationship. Do not type the pipe
symbol.
$ system_command Denotes a command prompt - do $ createdb mydatabase
# root_system_command not type the prompt symbol. $ and # chown gpadmin -R /datadir
=> gpdb_command # denote terminal command => SELECT * FROM mytable;
=# su_gpdb_command prompts. => and =# denote =# SELECT * FROM pg_database;
Greenplum Database interactive
program command prompts (psql
or gpssh, for example).
Getting Support
EMC support, product, and licensing information can be obtained as follows.
Product information
For product-specific documentation, release notes, or software updates, go to the
EMC Online Support site at support.emc.com.
For information about EMC products, licensing, and service, go to the EMC
Powerlink website (registration required) at http://Powerlink.EMC.com.
Getting Support 3
Greenplum Database System Administrator Guide 4.3 – Preface
Technical support
For technical support, go to EMC Online Support. On the Support page, you will see
several options, including one for making a service request. Note that to open a service
request, you must have a valid support agreement. Please contact your EMC sales
representative for details about obtaining a valid support agreement or with questions
about your account.
Getting Support 4
Greenplum Database System Administrator Guide 4.3 – Chapter 1: About the Greenplum Architecture
1.
About the Greenplum Architecture
Greenplum Database stores and processes large amounts of data by distributing the
data and processing workload across several servers or hosts. Greenplum Database is
an array of individual databases based upon PostgreSQL 8.2 working together to
present a single database image. The master is the entry point to the Greenplum
Database system. It is the database instance to which clients connect and submit SQL
statements. The master coordinates its work with the other database instances in the
system, called segments, which store and process the data.
Figure 1.1 High-Level Greenplum Database Architecture
This section describes the components that make up a Greenplum Database system
and how they work together:
• About the Greenplum Master
• About the Greenplum Segments
• About the Greenplum Interconnect
• About Redundancy and Failover in Greenplum Database
• About Parallel Data Loading
• About Management and Monitoring
5