Matrix Algebra for Linear Models

393 Pages·2013·2.1 MB·English

Matrix algebra for
linear Models

Matrix algebra for
linear Models
Marvin H. J. gruber
School of Mathematical Sciences
Rochester Institute of Technology
Rochester, NY
Copyright © 2014 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except
as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment of the appropriate per-copy fee
to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax
[phone], or on the web at www.copyright.com. Requests to the Publisher for permission should
be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken,
NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best
efforts in preparing this book, they make no representations or warranties with respect to the accuracy
or completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be suitable
for your situation. You should consult with a professional where appropriate. Neither the publisher nor
author shall be liable for any loss of profit or any other commercial damages, including but not limited
to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact
our Customer Care Department within the United States at [phone], outside the United States
at [phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Gruber, Marvin H. J., 1941–
Matrix algebra for linear models / Marvin H. J. Gruber, Department of Mathematical Sciences,
Rochester Institute of Technology, Rochester, NY.
pages cm
Includes bibliographical references and index.
ISBN 978-1-118-59255-7 (cloth)
1. Linear models (Statistics) 2. Matrices. I. Title.
QA279.G78 2013
519.5′36–dc23
2013026537
Printed in the United States of America
ISBN: 9781118592557
10 9 8 7 6 5 4 3 2 1
To the memory of my parents, Adelaide Lee Gruber and Joseph George Gruber, who
were always there for me while I was growing up and as a young adult.

Contents
PrefaCe xiii
aCknowledgMents xv
Part i basiC ideas about MatriCes and systeMs
of linear equations 1
section 1 what Matrices are and some basic operations with them 3
1.1 Introduction, 3
1.2 What are Matrices and Why are they Interesting
to a Statistician?, 3
1.3 Matrix Notation, Addition, and Multiplication, 6
1.4 Summary, 10
Exercises, 10
section 2 determinants and solving a system of equations 14
2.1 Introduction, 14
2.2 Definition of and Formulae for Expanding Determinants, 14
2.3 Some Computational Tricks for the Evaluation
of Determinants, 16
2.4 Solution to Linear Equations Using Determinants, 18
2.5 Gauss Elimination, 22
2.6 Summary, 27
Exercises, 27
vii
viii CoNTENTS
section 3 the inverse of a Matrix 30
3.1 Introduction, 30
3.2 The Adjoint Method of Finding the Inverse of a Matrix, 30
3.3 Using Elementary Row operations, 31
3.4 Using the Matrix Inverse to Solve a System of Equations, 33
3.5 Partitioned Matrices and Their Inverses, 34
3.6 Finding the Least Square Estimator, 38
3.7 Summary, 44
Exercises, 44
section 4 special Matrices and facts about Matrices that will
be used in the sequel 47
4.1 Introduction, 47
4.2 Matrices of the Form aI + bJ,47
n n
4.3 orthogonal Matrices, 49
4.4 Direct Product of Matrices, 52
4.5 An Important Property of Determinants, 53
4.6 The Trace of a Matrix, 56
4.7 Matrix Differentiation, 57
4.8 The Least Square Estimator Again, 62
4.9 Summary, 62
Exercises, 63
section 5 vector spaces 66
5.1 Introduction, 66
5.2 What is a Vector Space?, 66
5.3 The Dimension of a Vector Space, 68
5.4 Inner Product Spaces, 70
5.5 Linear Transformations, 73
5.6 Summary, 76
Exercises, 76
section 6 the rank of a Matrix and solutions to systems of equations 79
6.1 Introduction, 79
6.2 The Rank of a Matrix, 79
6.3 Solving Systems of Equations with Coefficient Matrix of Less
than Full Rank, 84
6.4 Summary, 87
Exercises, 87
Part ii eigenvalues, tHe singular value
deCoMPosition, and PrinCiPal CoMPonents 91
section 7 finding the eigenvalues of a Matrix 93
7.1 Introduction, 93
7.2 Eigenvalues and Eigenvectors of a Matrix, 93