Beginning Shell Scripting

530 Pages·2005·8.789 MB·English

by  Eric Foster-Johnson, John C. Welch

01_583204 ffirs.qxd 3/18/05 9:30 PM Page i
01_583204 ffirs.qxd 3/18/05 9:30 PM Page i
Beginning Shell Scripting
Eric Foster-Johnson, John C. Welch, and Micah Anderson
01_583204 ffirs.qxd 3/18/05 9:30 PM Page ii
Beginning Shell Scripting
Published by
Wiley Publishing, Inc.
10475 Crosspoint Boulevard
Indianapolis, IN 46256
www.wiley.com
Copyright © 2005 by Wiley Publishing, Inc., Indianapolis, Indiana
Published simultaneously in Canada
ISBN-13: 978-0-7645-8320-9
ISBN-10: 0-7645-8320-4
Manufactured in the United States of America
10 9 8 7 6 5 4 3 2 1
1MA/QW/QU/QV/IN
No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means,
electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of
the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization
through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA
01923, [phone], fax [phone]. Requests to the Publisher for permission should be addressed to the Legal
Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, [phone], fax [phone], or
online at http://www.wiley.com/go/permissions.
LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY:THE PUBLISHER AND THE AUTHOR MAKE NO REPRESEN-
TATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACYOR COMPLETENESS OF THE CONTENTS OF THIS
WORK AND SPECIFICALLYDISCLAIM ALLWARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF
FITNESS FOR APARTICULAR PURPOSE. NO WARRANTYMAYBE CREATED OR EXTENDED BYSALES OR PROMO-
TIONALMATERIALS. THE ADVICE AND STRATEGIES CONTAINED HEREIN MAYNOT BE SUITABLE FOR EVERY
SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN REN-
DERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE IS
REQUIRED, THE SERVICES OF ACOMPETENT PROFESSIONALPERSON SHOULD BE SOUGHT. NEITHER THE PUB-
LISHER NOR THE AUTHOR SHALLBE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGA-
NIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS ACITATION AND/OR APOTENTIALSOURCE OF
FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMA-
TION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER,
READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAYHAVE CHANGED OR DIS-
APPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ.
For general information on our other products and services or to obtain technical support, please contact our Customer
Care Department within the U.S. at [phone], outside the U.S. at [phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available
in electronic books.
Library of Congress Cataloging-in-Publication Data
Foster-Johnson, Eric.
Beginning shell scripting / Eric Foster-Johnson, John C. Welch, and Micah Anderson.
p. cm.
Includes index.
ISBN-13: 978-0-7645-8320-9
ISBN-10: 0-7645-8320-4 (paper/website)
1. Operating systems (Computers) I. Welch, John C., 1967– II. Anderson, Micah, 1974– III. Title.
QA76.76.O63F59717 2005
005.4' 3—dc22
2005002075
Trademarks:Wiley, the Wiley Publishing logo, Wrox, the Wrox logo, Programmer to Programmer and related trade dress are
trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries,
and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley
Publishing, Inc., is not associated with any product or vendor mentioned in this book.
01_583204 ffirs.qxd 3/18/05 9:30 PM Page iii
About the Authors
Eric Foster-Johnson(Arden Hills, MN) is a veteran programmer who works daily with Linux, Unix,
Windows, Mac OS X, and other operating systems. By day, he writes enterprise Java software for
ObjectPartners, a Minnesota consulting firm. He has authored a number of Linux and Unix titles includ-
ing Red Hat RPM Guide, Teach Yourself Linux, Teach Yourself Unix, and Perl Modules.
To Katya and Nalana.
John C. Welch(Boston, MA) has more than ten years of Mac experience in the trenches of the IT world.
He packs the hall at Macworld Expo where he’s a regular speaker, sharing his experiences and knowl-
edge on using Macs effectively in business and enterprise IT environments. John’s articles are frequently
featured in MacTech MagazineandWorkingMac.com, and he is semi-regularly featured on The Mac Show
conducting interviews with the geekier side of the Mac community. He’s recently been touring the coun-
try presenting seminars on Mac OS X administration to IT professionals from coast to coast.
First and foremost, this is dedicated to my son, Alex, who is always the angel on my shoulder.
As an only child, I’ve “adopted” my own family over the years to make up for the one I didn’t grow up
with, so in no particular order: Adrian, Jenny, Melissa, Sly, Web, Karen, Skip, Gypsye, Schoun, Harv,
Jessica, the MacTech Crew, Sam, Steve, Shawn, Hilary, the YMLlist, the list that is never named, and
too many others who help me out in myriad ways.
Oh, and to Randy Milholland, whose work has kept me sane in this very odd world.
Micah Andersonhas been a Unix system administrator for more than a decade. He is a Debian
GNU/Linux developer and works as an independent consultant building alternative communication
infrastructure and supporting the technology needs of positive grassroots alternatives.
This is dedicated to my parents, who got me started hacking on the Timex Sinclair and have always sup-
ported me, no matter what I do. This is also dedicated to those around the world who have struggled for
horizontal self-determination in the face of oppression: those who inspire people to dream of a better
world by doing something about it. Slow and steady wins the race.
01_583204 ffirs.qxd 3/18/05 9:30 PM Page iv
Credits
Acquisitions Editor Vice President and Publisher
Debra Williams Cauley Joseph B. Wikert
Development Editor Project Coordinator
Sara Shlaer Erin Smith
Technical Editor Graphics and Production Specialists
Dilip Thomas Carrie Foster
Lauren Goddard
Copy Editors Denny Hager
Nancy Rapoport Joyce Haughey
Luann Rouff Jennifer Heleine
Barry Offringa
Editorial Manager
Mary Beth Wakefield Quality Control Technician
Laura Albert
Vice President & Executive Group Publisher
Richard Swadley Proofreading and Indexing
TECHBOOKS Production Services
02_583204 ftoc.qxd 3/18/05 9:24 PM Page v
Contents
Introduction xiii
Chapter 1: Introducing Shells 1
What Is a Shell? 2
Why Use Shells? 3
What Kind of Shells Are There? 4
The Bourne Shell 4
The C Shell 4
The Korn Shell 5
Bash,the Bourne Again Shell 6
tcsh,the T C Shell 6
Other Shells 7
Graphical Shells 7
The Official POSIX Shell 8
Default Shells 8
Choosing a Shell 9
Changing Your Default Shell 9
Running a Shell from Within Another Shell 12
Finding Out More About the Shell 13
How Do Shells Fit into a Graphical Environment? 13
Running Shells on Linux 15
Running Shells on Mac OS X 17
Running Shells on Unix Systems 18
Running Shells on Windows—Isn’t command.com Enough? 18
Running Shells on PDAs and Other Systems 19
Entering Commands 20
Determining Which Shell You Are Running 21
Command-Line Options 23
Command Editing 27
Command Substitution 27
Viewing the Command History 31
Calling Up an Editor 33
Using File-Name Completion 34
Working with Wildcards 35
The * Wildcard 35
The ? Wildcard 37
Running Commands in the Background 37
Summary 38
02_583204 ftoc.qxd 3/18/05 9:24 PM Page vi
Contents
Chapter 2: Introducing Shell Scripts 39
What Are Shell Scripts? 40
Alternatives to Shells: Other Scripting Languages 42
Perl 43
Python 45
Tcl 46
MS-DOS Batch Files 47
Tools to Edit Shell Scripts 47
Using Legacy Editors 47
Moving to Graphical Text Editors 59
Writing Scripts 65
Remembering Commands So You Don’t Have To 66
Outputting Text 67
Variables 71
Gathering Input 76
Commenting Your Scripts 78
Continuing Lines 81
Summary 82
Exercises 83
Chapter 3: Controlling How Scripts Run 85
Referencing Variables 85
Looping and Iteration 89
Looping over Files 90
Looping for a Fixed Number of Iterations 93
Looping Like a C Program—the bash Shell 96
Looping in the C Shell 98
Nested Loops 99
Checking Conditions with if 100
Or Else,What? 101
What Is Truth? 102
Redirecting Output 106
Using elif (Short for else if) 111
Nesting if Statements 113
Testing with the test Command 114
Comparing Numbers 114
Comparing Text Strings 117
Testing Files 120
Using the Binary and Not Operators 121
Creating Shorthand Tests with [ 123
Making Complex Decisions with case 125
Handling Problematic Input 127
Using case with the C Shell 129
vi
02_583204 ftoc.qxd 3/18/05 9:24 PM Page vii
Contents
Looping While a Condition Is True 131
Looping Until a Condition Is True 132
Summary 134
Exercises 134
Chapter 4: Interacting with the Environment 135
Examining Environment Variables 135
Reading Environment Variables 136
Setting Environment Variables 150
Customizing Your Account 153
How the Bourne Shell Starts Up 153
How the Korn Shell Starts Up 153
How the C Shell Starts Up 154
How the T C Shell Starts Up 154
How Bash Starts Up 154
Handling Command-Line Arguments 155
Reading Command-Line Arguments with the Bourne Shell 156
Reading Command-Line Arguments with the C Shell 160
Making Scripts Executable 160
Marking Files Executable 160
Setting the #! Magic Line 161
Summary 164
Exercises 165
Chapter 5: Scripting with Files 167
Combining Files into Archives 168
Working with File Modes 169
Testing Files with the test Command 171
Dealing with Mac OS X Files 172
The Legacies of NeXT 172
Mobile File Systems and Mac OS X 173
Naming Issues 175
HFS+ Versus UFS: The Mac OS X Holy War 175
The Terror of the Resource Fork 176
Working with Here Files and Interactive Programs 177
Displaying Messages with Here Files 178
Customizing Here Files 179
Driving Interactive Programs with Here Files 183
Turning Off Variable Substitution 186
Summary 187
Exercises 187
vii
02_583204 ftoc.qxd 3/18/05 9:24 PM Page viii
Contents
Chapter 6: Processing Text with sed 189
Introducing sed 190
sed Versions 190
Installing sed 191
Bootstrap Installation 192
Configuring and Installing sed 193
How sed Works 193
Invoking sed 194
Editing Commands 195
Invoking sed with the -e Flag 196
The -n,--quiet,and --silent Flags 197
sed Errors 199
Selecting Lines to Operate On 199
Address Ranges 200
Address Negation 201
Address Steps 202
Substitution 203
Substitution Flags 204
Using an Alternative String Separator 205
Address Substitution 206
Advanced sed Invocation 207
The comment Command 209
The insert,append,and change Commands 210
Advanced Addressing 211
Regular Expression Addresses 212
Character Class Keywords 215
Regular Expression Address Ranges 216
Combining Line Addresses with regexps 217
Advanced Substitution 217
Referencing Matched regexps with & 218
Back References 219
Hold Space 220
More sed Resources 222
Common One-Line sed Scripts 222
Common sed Commands 224
Less Common sed Commands 224
GNU sed-Specific sed Extensions 225
Summary 226
Exercises 227
Chapter 7: Processing Text with awk 229
What Is awk (Gawk/Mawk/Nawk/Oawk)? 230
Gawk,the GNU awk 230
viii