The Linux Command Line

555 Pages·2019·2.022 MB·English

by  William Shotts

The Linux Command Line
Fifth Internet Edition
William Shotts
A LinuxCommand.org Book
Copyright ©2008-2019, William E. Shotts, Jr.
This work is licensed under the Creative Commons Attribution-Noncommercial-No De-
rivative Works 3.0 United States License. To view a copy of this license, visit the link
above or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042.
A version of this book is also available in printed form, published by No Starch Press.
Copies may be purchased wherever fine books are sold. No Starch Press also offers elec-
tronic formats for popular e-readers. They can be reached at: https://www.nostarch.com.
Linux® is the registered trademark of Linus Torvalds. All other trademarks belong to
their respective owners.
This book is part of the LinuxCommand.org project, a site for Linux education and advo-
cacy devoted to helping users of legacy operating systems migrate into the future. You
may contact the LinuxCommand.org project at http://linuxcommand.org.
Release History
Version Date Description
19.01A January 28, 2019 Fifth Internet Edition (Corrected TOC)
19.01 January 17, 2019 Fifth Internet Edition.
17.10 October 19, 2017 Fourth Internet Edition.
16.07 July 28, 2016 Third Internet Edition.
13.07 July 6, 2013 Second Internet Edition.
09.12 December 14, 2009 First Internet Edition.
Table of Contents
Introduction....................................................................................................xvi
Why Use the Command Line?......................................................................................xvi
What This Book Is About..............................................................................................xvii
Who Should Read This Book.......................................................................................xvii
What's in This Book.....................................................................................................xviii
How To Read This Book................................................................................................xix
Prerequisites............................................................................................................xix
Why I Don't Call It “GNU/Linux”...........................................................................xix
What's New in the Fifth Internet Edition.........................................................................xx
Acknowledgments..........................................................................................................xx
First Internet Edition..................................................................................................xx
Second Internet Edition............................................................................................xxi
Third Internet Edition................................................................................................xxi
Fourth Internet Edition..............................................................................................xxi
Fifth Internet Edition.................................................................................................xxi
Your Feedback Is Needed!............................................................................................xxi
Further Reading............................................................................................................xxi
Colophon......................................................................................................................xxii
Part 1 – Learning the Shell...............................................................1
1 – What Is the Shell?......................................................................................2
Terminal Emulators..........................................................................................................2
Making Your First Keystrokes..........................................................................................2
Command History.......................................................................................................3
Cursor Movement.......................................................................................................3
A Few Words About Mice and Focus.....................................................................3
Try Some Simple Commands..........................................................................................4
Ending a Terminal Session..............................................................................................5
The Console Behind the Curtain...........................................................................5
Summing Up....................................................................................................................5
Further Reading...............................................................................................................6
2 – Navigation...................................................................................................7
Understanding the File System Tree...............................................................................7
The Current Working Directory........................................................................................7
Listing the Contents of a Directory..................................................................................8
i
Changing the Current Working Directory........................................................................9
Absolute Pathnames..................................................................................................9
Relative Pathnames...................................................................................................9
Some Helpful Shortcuts............................................................................................11
Important Facts About Filenames........................................................................11
Summing Up..................................................................................................................12
3 – Exploring the System..............................................................................13
Having More Fun with ls................................................................................................13
Options and Arguments............................................................................................14
A Longer Look at Long Format.................................................................................16
Determining a File's Type with file.................................................................................17
Viewing File Contents with less.....................................................................................17
What Is “Text”?.....................................................................................................17
Less Is More........................................................................................................19
Taking a Guided Tour.....................................................................................................19
Symbolic Links...............................................................................................................23
Hard Links.....................................................................................................................24
Summing Up..................................................................................................................24
Further Reading.............................................................................................................24
4 – Manipulating Files and Directories........................................................25
Wildcards.......................................................................................................................25
Character Ranges................................................................................................27
Wildcards Work in the GUI Too............................................................................27
mkdir – Create Directories.............................................................................................28
cp – Copy Files and Directories....................................................................................28
Useful Options and Examples..................................................................................29
mv – Move and Rename Files.......................................................................................30
Useful Options and Examples..................................................................................30
rm – Remove Files and Directories...............................................................................31
Useful Options and Examples..................................................................................32
Be Careful with rm!..............................................................................................33
ln – Create Links............................................................................................................33
Hard Links................................................................................................................33
Symbolic Links..........................................................................................................34
Let's Build a Playground................................................................................................34
Creating Directories..................................................................................................34
Copying Files............................................................................................................35
Moving and Renaming Files.....................................................................................36
Creating Hard Links..................................................................................................37
Creating Symbolic Links...........................................................................................38
Removing Files and Directories...............................................................................39
Creating Symlinks With The GUI.........................................................................41
Summing Up..................................................................................................................41
Further Reading.............................................................................................................41
5 – Working with Commands........................................................................42
What Exactly Are Commands?......................................................................................42
Identifying Commands...................................................................................................43
ii
type – Display a Command's Type...........................................................................43
which – Display an Executable's Location...............................................................43
Getting a Command's Documentation..........................................................................44
help – Get Help for Shell Builtins..............................................................................44
--help – Display Usage Information..........................................................................45
man – Display a Program's Manual Page................................................................46
apropos – Display Appropriate Commands..............................................................47
whatis – Display One-line Manual Page Descriptions..............................................48
The Most Brutal Man Page Of Them All..............................................................48
info – Display a Program's Info Entry.......................................................................48
README and Other Program Documentation Files................................................50
Creating Our Own Commands with alias......................................................................50
Summing Up..................................................................................................................52
Further Reading.............................................................................................................52
6 – Redirection................................................................................................54
Standard Input, Output, and Error.................................................................................54
Redirecting Standard Output.........................................................................................55
Redirecting Standard Error............................................................................................57
Redirecting Standard Output and Standard Error to One File.................................57
Disposing of Unwanted Output.................................................................................58
/dev/null In Unix Culture......................................................................................58
Redirecting Standard Input............................................................................................58
cat – Concatenate Files............................................................................................59
Pipelines........................................................................................................................61
The Difference Between > and |..........................................................................61
Filters........................................................................................................................62
uniq - Report or Omit Repeated Lines......................................................................62
wc – Print Line, Word, and Byte Counts...................................................................63
grep – Print Lines Matching a Pattern......................................................................63
head / tail – Print First / Last Part of Files................................................................64
tee – Read from Stdin and Output to Stdout and Files............................................65
Summing Up..................................................................................................................66
Linux Is About Imagination..................................................................................66
7 – Seeing the World as the Shell Sees It....................................................68
Expansion......................................................................................................................68
Pathname Expansion...............................................................................................69
Pathname Expansion of Hidden Files.................................................................70
Tilde Expansion........................................................................................................70
Arithmetic Expansion................................................................................................71
Brace Expansion......................................................................................................72
Parameter Expansion...............................................................................................73
Command Substitution.............................................................................................74
Quoting..........................................................................................................................75
Double Quotes..........................................................................................................76
Single Quotes...........................................................................................................78
Escaping Characters................................................................................................78
Backslash Escape Sequences............................................................................79
Summing Up..................................................................................................................79
iii
Further Reading.............................................................................................................80
8 – Advanced Keyboard Tricks.....................................................................81
Command Line Editing..................................................................................................81
Cursor Movement.....................................................................................................81
Modifying Text...........................................................................................................82
Cutting and Pasting (Killing and Yanking) Text.........................................................82
The Meta Key......................................................................................................83
Completion....................................................................................................................83
Programmable Completion..................................................................................85
Using History.................................................................................................................85
Searching History.....................................................................................................86
History Expansion.....................................................................................................88
script....................................................................................................................88
Summing Up..................................................................................................................89
Further Reading.............................................................................................................89
9 – Permissions..............................................................................................90
Owners, Group Members, and Everybody Else............................................................91
Reading, Writing, and Executing...................................................................................92
chmod – Change File Mode.....................................................................................94
What the Heck is Octal?......................................................................................95
Setting File Mode with the GUI.................................................................................98
umask – Set Default Permissions............................................................................98
Some Special Permissions................................................................................100
Changing Identities......................................................................................................101
su – Run a Shell with Substitute User and Group IDs...........................................102
sudo – Execute a Command as Another User.......................................................103
Ubuntu and sudo...............................................................................................104
chown – Change File Owner and Group................................................................104
chgrp – Change Group Ownership.........................................................................106
Exercising Our Privileges............................................................................................106
Changing Your Password............................................................................................108
Summing Up................................................................................................................109
Further Reading..........................................................................................................109
10 – Processes..............................................................................................110
How a Process Works.................................................................................................110
Viewing Processes.......................................................................................................111
Viewing Processes Dynamically with top................................................................113
Controlling Processes..................................................................................................116
Interrupting a Process.............................................................................................116
Putting a Process in the Background.....................................................................117
Returning a Process to the Foreground.................................................................118
Stopping (Pausing) a Process................................................................................118
Signals.........................................................................................................................119
Sending Signals to Processes with kill...................................................................120
Sending Signals to Multiple Processes with killall..................................................122
Shutting Down the System..........................................................................................123
More Process-Related Commands.............................................................................124
iv
Summing Up................................................................................................................124
Part 2 – Configuration and the Environment..............................127
11 – The Environment..................................................................................128
What is Stored in the Environment?............................................................................128
Examining The Environment..................................................................................128
Some Interesting Variables.....................................................................................130
How Is The Environment Established?.......................................................................131
What's in a Startup File?........................................................................................132
Modifying the Environment..........................................................................................134
Which Files Should We Modify?.............................................................................135
Text Editors.............................................................................................................135
Using a Text Editor.................................................................................................136
Why Comments Are Important..........................................................................139
Activating Our Changes..........................................................................................140
Summing Up................................................................................................................140
Further Reading..........................................................................................................140
12 – A Gentle Introduction to vi..................................................................141
Why We Should Learn vi.............................................................................................141
A Little Background.....................................................................................................142
Starting and Stopping vi..............................................................................................142
Compatibility Mode............................................................................................143
Editing Modes..............................................................................................................144
Entering Insert Mode..............................................................................................145
Saving Our Work....................................................................................................145
Moving the Cursor Around...........................................................................................146
Basic Editing................................................................................................................147
Appending Text.......................................................................................................147
Opening a Line.......................................................................................................148
Deleting Text...........................................................................................................149
Cutting, Copying, and Pasting Text........................................................................150
Joining Lines...........................................................................................................152
Search-and-Replace...................................................................................................152
Searching Within a Line..........................................................................................152
Searching the Entire File........................................................................................152
Global Search-and-Replace...................................................................................153
Editing Multiple Files...................................................................................................155
Switching Between Files.........................................................................................156
Opening Additional Files for Editing.......................................................................157
Copying Content from One File into Another..........................................................157
Inserting an Entire File into Another.......................................................................158
Saving Our Work.........................................................................................................159
Summing Up................................................................................................................160
Further Reading..........................................................................................................160
13 – Customizing the Prompt......................................................................162
Anatomy of a Prompt...................................................................................................162
v
Trying Some Alternative Prompt Designs....................................................................164
Adding Color................................................................................................................165
Terminal Confusion............................................................................................165
Moving the Cursor.......................................................................................................168
Saving the Prompt.......................................................................................................169
Summing Up................................................................................................................169
Further Reading..........................................................................................................170
Part 3 – Common Tasks and Essential Tools.............................171
14 – Package Management..........................................................................172
Packaging Systems.....................................................................................................172
How a Package System Works...................................................................................173
Package Files.........................................................................................................173
Repositories............................................................................................................173
Dependencies.........................................................................................................174
High and Low-level Package Tools........................................................................174
Common Package Management Tasks.......................................................................175
Finding a Package in a Repository.........................................................................175
Installing a Package from a Repository..................................................................175
Installing a Package from a Package File..............................................................176
Removing a Package.............................................................................................176
Updating Packages from a Repository...................................................................177
Upgrading a Package from a Package File............................................................177
Listing Installed Packages......................................................................................178
Determining Whether a Package is Installed.........................................................178
Displaying Information About an Installed Package...............................................179
Finding Which Package Installed a File.................................................................179
Summing Up................................................................................................................180
The Linux Software Installation Myth.................................................................180
Further Reading..........................................................................................................181
15 – Storage Media.......................................................................................182
Mounting and Unmounting Storage Devices...............................................................182
Viewing a List of Mounted File Systems.................................................................184
Why Unmounting Is Important...........................................................................187
Determining Device Names....................................................................................188
Creating New File Systems.........................................................................................191
Manipulating Partitions with fdisk...........................................................................192
Creating a New File System with mkfs...................................................................194
Testing and Repairing File Systems............................................................................195
What the fsck?...................................................................................................196
Moving Data Directly to and from Devices..................................................................196
Creating CD-ROM Images..........................................................................................197
Creating an Image Copy of a CD-ROM..................................................................197
Creating an Image From a Collection of Files........................................................197
A Program by Any Other Name.........................................................................198
Writing CD-ROM Images.............................................................................................198
Mounting an ISO Image Directly............................................................................198
vi
Blanking a Rewritable CD-ROM.............................................................................199
Writing an Image....................................................................................................199
Summing Up................................................................................................................199
Further Reading..........................................................................................................199
Extra Credit..................................................................................................................199
16 – Networking............................................................................................201
Examining and Monitoring a Network.........................................................................202
ping.........................................................................................................................202
traceroute...............................................................................................................203
ip.............................................................................................................................204
netstat.....................................................................................................................205
Transporting Files Over a Network..............................................................................206
ftp............................................................................................................................206
lftp – A Better ftp.....................................................................................................209
wget........................................................................................................................209
Secure Communication with Remote Hosts................................................................209
ssh..........................................................................................................................210
Tunneling with SSH...........................................................................................213
scp and sftp............................................................................................................214
An SSH Client for Windows?.............................................................................215
Summing Up................................................................................................................215
Further Reading..........................................................................................................215
17 – Searching for Files...............................................................................217
locate – Find Files the Easy Way................................................................................217
Where Does the locate Database Come From?...............................................219
find – Find Files the Hard Way....................................................................................219
Tests.......................................................................................................................220
Operators................................................................................................................222
Predefined Actions..................................................................................................225
User-Defined Actions..............................................................................................227
Improving Efficiency...............................................................................................228
xargs.......................................................................................................................228
Dealing with Funny Filenames..........................................................................229
A Return to the Playground....................................................................................229
Options...................................................................................................................232
Summing Up................................................................................................................233
Further Reading..........................................................................................................233
18 – Archiving and Backup.........................................................................234
Compressing Files.......................................................................................................234
gzip.........................................................................................................................235
bzip2.......................................................................................................................237
Don’t Be Compressive Compulsive...................................................................238
Archiving Files.............................................................................................................238
tar............................................................................................................................239
zip...........................................................................................................................244
Synchronizing Files and Directories............................................................................246
Using rsync Over a Network...................................................................................249
vii
Summing Up................................................................................................................250
Further Reading..........................................................................................................250
19 – Regular Expressions...........................................................................251
What are Regular Expressions?.............................................................................251
grep.............................................................................................................................251
Metacharacters and Literals........................................................................................253
The Any Character......................................................................................................254
Anchors.......................................................................................................................255
A Crossword Puzzle Helper...............................................................................256
Bracket Expressions and Character Classes..............................................................256
Negation.................................................................................................................257
Traditional Character Ranges.................................................................................257
POSIX Character Classes......................................................................................258
Reverting to Traditional Collation Order............................................................261
POSIX Basic vs. Extended Regular Expressions.......................................................262
POSIX................................................................................................................263
Alternation...................................................................................................................263
Quantifiers...................................................................................................................264
? - Match an Element Zero or One Time................................................................264
* - Match an Element Zero or More Times.............................................................265
+ - Match an Element One or More Times.............................................................266
{ } - Match an Element a Specific Number of Times...............................................266
Putting Regular Expressions to Work..........................................................................267
Validating a Phone List With grep...........................................................................267
Finding Ugly Filenames with find............................................................................269
Searching for Files with locate...............................................................................269
Searching for Text with less and vim......................................................................270
Summing Up................................................................................................................271
Further Reading..........................................................................................................272
20 – Text Processing....................................................................................273
Applications of Text......................................................................................................273
Documents.............................................................................................................274
Web Pages.............................................................................................................274
Email.......................................................................................................................274
Printer Output.........................................................................................................274
Program Source Code............................................................................................274
Revisiting Some Old Friends.......................................................................................274
cat...........................................................................................................................275
MS-DOS Text vs. Unix Text...............................................................................276
sort..........................................................................................................................277
uniq.........................................................................................................................284
Slicing and Dicing........................................................................................................286
cut...........................................................................................................................286
Expanding Tabs.................................................................................................289
paste.......................................................................................................................290
join..........................................................................................................................291
Comparing Text...........................................................................................................293
comm......................................................................................................................294
viii