Table Of Content

Y L F M A E T ® TEAM FLY The WILEY advantage Dear Valued Customer, We realize you’re a busy professional with deadlines to hit. Whether your goal is to learn a new technology or solve a critical problem, we want to be there to lend you a hand. Our primary objective is to provide you with the insight and knowledge you need to stay atop the highly competitive and ever- changing technology industry. Wiley Publishing, Inc., offers books on a wide variety of technical categories, including security, data warehousing, software development tools, and networking — everything you need to reach your peak. Regardless of your level of expertise, the Wiley family of books has you covered. • For Dummies – The funand easyway to learn • The Weekend Crash Course –The fastestway to learn a new tool or technology • Visual – For those who prefer to learn a new topic visually • The Bible – The 100% comprehensivetutorial and reference • The Wiley Professional list – Practicaland reliableresources for IT professionals The book you hold now, UNIX Filesystems: Evolution, Design, and Implementation,is the first book to cover filesystems from all versions of UNIX and Linux. The author gives you details about the file I/O aspects of UNIX programming, describes the various UNIX and Linux operating system internals, and gives cases studies of some of the most popular filesystems including UFS, ext2, and the VERITAS filesystem, VxFS. The book contains numerous examples including a fully working Linux filesystem that you can experiment with. Our commitment to you does not end at the last page of this book. We’d want to open a dialog with you to see what other solutions we can provide. Please be sure to visit us at www.wiley.com/compbooks to re- view our complete title list and explore the other resources we offer. If you have a comment, suggestion, or any other inquiry, please locate the “contact us” link at www.wiley.com. Thank you for your support and we look forward to hearing from you and serving your needs again in the future. Sincerely, Richard K. Swadley Vice President & Executive Group Publisher Wiley Technology Publishing UNIX® Filesystems Evolution, Design, and Implementation (VERITAS Series) UNIX® Filesystems: Evolution, Design, and Implementation (VERITAS Series) Steve D. Pate Publisher: Robert Ipsen Executive Editor: Carol Long Developmental Editor: James H. Russell Managing Editor: Angela Smith Text Design & Composition: Wiley Composition Services This book is printed on acid-free paper. ∞ Copyright © 2003 by Steve Pate. All rights reserved. Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA01923, (978) 750-8400, fax (978) 750-4470. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4447, E-mail: [email protected]. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accu- racy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Trademarks: Wiley, the Wiley Publishing logo and related trade dress are trademarks or registered trademarks of Wiley Publishing, Inc., in the United States and other countries, and may not be used without written permission. Unix is a trademark or registered trademark of Unix Systems Laborato- ries, Inc. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging-in-Publication Data: ISBN: 0-471-16483-6 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 Contents Foreword xvii Introduction xix Chapter 1 UNIX Evolution and Standardization 1 A Brief Walk through Time...................................................................1 How Many Versions of UNIX Are There?..........................................3 Why Is UNIX So Successful?.................................................................3 The Early Days of UNIX........................................................................3 The Early History of the C Language............................................4 Research Editions of UNIX..............................................................5 AT&T’s Commercial Side of UNIX................................................5 The Evolution of BSD UNIX.................................................................7 BSD Networking Releases...............................................................8 UNIX Goes to Court.........................................................................8 The NetBSD Operating System.......................................................8 The FreeBSD Operating System......................................................9 The OpenBSD Operating System...................................................9 Sun Microsystems and SunOS..............................................................9 System V Release 4 and Variants........................................................10 Novell’s Entry into the UNIX Market................................................10 Linux and the Open Source Movement.............................................11 UNIX Standardization.........................................................................11 IEEE and POSIX..............................................................................11 The X/Open Group........................................................................12 The System V Interface Definition................................................12 Spec 11/70 and the Single UNIX Specification...........................13 UNIX International and OSF.........................................................13 The Data Management Interfaces Group....................................14 The Large File Summit...................................................................14 Summary................................................................................................15 vii viii Contents Chapter 2 File-Based Concepts 17 UNIX File Types....................................................................................18 File Descriptors......................................................................................19 Basic File Properties..............................................................................20 The File Mode Creation Mask.............................................................23 Changing File Permissions..................................................................24 Changing File Ownership....................................................................26 Changing File Times.............................................................................28 Truncating and Removing Files..........................................................29 Directories..............................................................................................30 Special Files............................................................................................31 Symbolic Links and Hard Links.........................................................32 Named Pipes..........................................................................................33 Summary................................................................................................34 Chapter 3 User File I/O 35 Library Functions versus System Calls..............................................35 Which Header Files to Use?.................................................................36 The Six Basic File Operations..............................................................37 Duplicate File Descriptors....................................................................40 Seeking and I/O Combined................................................................41 Data and Attribute Caching................................................................42 VxFS Caching Advisories...............................................................43 Miscellaneous Open Options.........................................................46 File and Record Locking......................................................................46 Advisory Locking............................................................................47 Mandatory Locking.........................................................................51 File Control Operations........................................................................51 Vectored Reads and Writes.................................................................52 Asynchronous I/O................................................................................54 Memory Mapped Files.........................................................................59 64-Bit File Access (LFS).........................................................................65 Sparse Files.............................................................................................66 Summary................................................................................................71 Chapter 4 The Standard I/O Library 73 The FILE Structure................................................................................74 Standard Input, Output, and Error.....................................................74 Opening and Closing a Stream...........................................................75 Standard I/O Library Buffering..........................................................77 Reading and Writing to/from a Stream............................................79 Seeking through the Stream................................................................82 Summary................................................................................................84 Contents ix Chapter 5 Filesystem-Based Concepts 85 What’s in a Filesystem?........................................................................85 The Filesystem Hierarchy....................................................................86 Disks, Slices, Partitions, and Volumes...............................................88 Raw and Block Devices........................................................................90 Filesystem Switchout Commands......................................................90 Creating New Filesystems...................................................................92 Mounting and Unmounting Filesystems..........................................94 Mount and Umount System Call Handling................................98 Mounting Filesystems Automatically..........................................98 Mounting Filesystems During Bootstrap....................................99 Repairing Damaged Filesystems......................................................100 The Filesystem Debugger..................................................................101 Per Filesystem Statistics.....................................................................101 User and Group Quotas.....................................................................103 Summary..............................................................................................104 Chapter 6 UNIX Kernel Concepts 105 5th to 7th Edition Internals................................................................105 The UNIX Filesystem.........................................................................106 Filesystem-Related Kernel Structures..............................................107 User Mode and Kernel Mode......................................................107 UNIX Process-Related Structures...............................................109 File Descriptors and the File Table.............................................110 The Inode Cache............................................................................112 The Buffer Cache...........................................................................112 Mounting Filesystems..................................................................115 System Call Handling........................................................................115 Pathname Resolution.........................................................................116 Putting It All Together.......................................................................117 Opening a File...............................................................................118 Reading the File.............................................................................119 Closing the File..............................................................................120 Summary..............................................................................................120 Chapter 7 Development of the SVR4 VFS/Vnode Architecture 121 The Need for Change.........................................................................121 Pre-SVR3 Kernels................................................................................122 The File System Switch......................................................................122 Mounting Filesystems..................................................................123 The Sun VFS/Vnode Architecture...................................................126 The uio Structure...........................................................................129 The VFS Layer...............................................................................129 The Vnode Operations Layer......................................................130