Exploratory Data Analysis with MATLAB

363 Pages·2007·7.76 MB·English

Exploratory Data
Analysis
with MATLAB®

Computer Science and Data Analysis Series

© 2005 by CRC Press LLC

Chapman & Hall/CRC
Series in Computer Science and Data Analysis

The interface between the computer and statistical sciences is increasing,
as each discipline seeks to harness the power and resources of the other.
This series aims to foster the integration between the computer sciences
and statistical, numerical and probabilistic methods by publishing a broad
range of reference works, textbooks and handbooks.

SERIES EDITORS
John Lafferty, Carnegie Mellon University
David Madigan, Rutgers University
Fionn Murtagh, Queen’s University Belfast
Padhraic Smyth, University of California Irvine

Proposals for the series should be sent directly to one of the series editors
above, or submitted to:

Chapman & Hall/CRC Press UK
23-25 Blades Court
London SW15 2NU
UK

Published Titles
Bayesian Artificial Intelligence
Kevin B. Korb and Ann E. Nicholson

Exploratory Data Analysis with MATLAB®

Wendy L. Martinez and Angel R. Martinez

Forthcoming Titles
Correspondence Analysis and Data Coding with JAVA and R
Fionn Murtagh

R Graphics
Paul Murrell

Nonlinear Dimensionality Reduction
Vin de Silva and Carrie Grimes

© 2005 by CRC Press LLC

CHAPMAN & HALL/CRC

A CRC Press Company
Boca Raton London New York Washington, D.C.

Wendy L. Martinez
Angel R. Martinez

Exploratory Data
Analysis
with MATLAB®

Computer Science and Data Analysis Series

© 2005 by CRC Press LLC

This book contains information obtained from authentic and highly regarded sources. Reprinted material
is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable
efforts have been made to publish reliable data and information, but the author and the publisher cannot
assume responsibility for the validity of all materials or for the consequences of their use.

Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic
or mechanical, including photocopying, microfilming, and recording, or by any information storage or
retrieval system, without prior permission in writing from the publisher.

The consent of CRC Press does not extend to copying for general distribution, for promotion, for creating
new works, or for resale. Specific permission must be obtained in writing from CRC Press for such
copying.

Direct all inquiries to CRC Press, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431.

Trademark Notice:

Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation, without intent to infringe.

Visit the CRC Press Web site at www.crcpress.com

© 2005 by Chapman & Hall/CRC Press

No claim to original U.S. Government works
International Standard Book Number 1-58488-366-9
Library of Congress Card Number 2004058245
Printed in the United States of America 1 2 3 4 5 6 7 8 9 0
Printed on acid-free paper

Library of Congress Cataloging-in-Publication Data

Martinez, Wendy L.
Exploratory data analysis with MATLAB / Wendy L. Martinez, Angel R. Martinez.
p. cm.
Includes bibliographical references and index.
ISBN 1-58488-366-9 (alk. paper)
1. Multivariate analysis. 2. MATLAB. 3. Mathematical statistics. I. Martinez, Angel R.
II. Title.

QA278.M3735 2004
519.5'35--dc22
2004058245

C3669 disclaimer.fm Page 1 Monday, October 18, 2004 12:24 PM

© 2005 by CRC Press LLC

This book is dedicated to our children:

Angel and Ochida

Deborah and Nataniel

Jeff and Lynn

and

Lisa (Principessa)

EDA.book Page i Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

vii

Table of Contents

Table of Contents ..................................................................................................vii
Preface .................................................................................................................xiii

Part I
Introduction to Exploratory Data Analysis

Chapter 1
Introduction to Exploratory Data Analysis
1.1 What is Exploratory Data Analysis ............................................................. 3
1.2 Overview of the Text ..................................................................................... 6
1.3 A Few Words About Notation ..................................................................... 8
1.4 Data Sets Used in the Book ........................................................................... 9
1.4.1 Unstructured Text Documents ........................................................ 9
1.4.2 Gene Expression Data ..................................................................... 12
1.4.3 Oronsay Data Set ............................................................................. 18
1.4.4 Software Inspection ......................................................................... 19
1.5 Transforming Data ....................................................................................... 20
1.5.1 Power Transformations .................................................................. 21
1.5.2 Standardization ................................................................................ 22
1.5.3 Sphering the Data ............................................................................ 24
1.6 Further Reading ........................................................................................... 25
Exercises .............................................................................................................. 27

Part II
EDA as Pattern Discovery

Chapter 2
Dimensionality Reduction - Linear Methods
2.1 Introduction .................................................................................................. 31
2.2 Principal Component Analysis - PCA ...................................................... 33
2.2.1 PCA Using the Sample Covariance Matrix ................................. 34
2.2.2 PCA Using the Sample Correlation Matrix ................................. 37
2.2.3 How Many Dimensions Should We Keep? ................................. 38
2.3 Singular Value Decomposition - SVD ....................................................... 42
2.4 Factor Analysis ............................................................................................. 46

EDA.book Page vii Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

viii
Exploratory Data Analysis with MATLAB

2.5 Intrinsic Dimensionality .............................................................................. 52
2.6 Summary and Further Reading ................................................................. 57
Exercises .............................................................................................................. 57

Chapter 3
Dimensionality Reduction - Nonlinear Methods
3.1 Multidimensional Scaling - MDS ............................................................... 61
3.1.1 Metric MDS ...................................................................................... 63
3.1.2 Nonmetric MDS ............................................................................... 72
3.2 Manifold Learning ....................................................................................... 81
3.2.1 Locally Linear Embedding ............................................................. 81
3.2.2 Isometric Feature Mapping - ISOMAP ........................................ 83
3.2.3 Hessian Eigenmaps ......................................................................... 85
3.3 Artificial Neural Network Approaches .................................................... 90
3.3.1 Self-Organizing Maps - SOM ......................................................... 90
3.3.2 Generative Topographic Maps - GTM ......................................... 94
3.4 Summary and Further Reading ................................................................. 98
Exercises ............................................................................................................ 100

Chapter 4
Data Tours
4.1 Grand Tour ................................................................................................. 104
4.1.1 Torus Winding Method ................................................................ 105
4.1.2 Pseudo Grand Tour ....................................................................... 107
4.2 Interpolation Tours .................................................................................... 110
4.3 Projection Pursuit ....................................................................................... 112
4.4 Projection Pursuit Indexes ........................................................................ 120
4.4.1 Posse Chi-Square Index ................................................................ 120
4.4.2 Moment Index ................................................................................ 124
4.5 Summary and Further Reading ............................................................... 125
Exercises ............................................................................................................ 126

Chapter 5
Finding Clusters
5.1 Introduction ................................................................................................ 127
5.2 Hierarchical Methods ................................................................................ 129
5.3 Optimization Methods - k-Means ............................................................ 135
5.4 Evaluating the Clusters ............................................................................. 139
5.4.1 Rand Index ..................................................................................... 141
5.4.2 Cophenetic Correlation ................................................................ 143
5.5.3 Upper Tail Rule .............................................................................. 144
5.5.4 Silhouette Plot ................................................................................ 147
5.5.5 Gap Statistic .................................................................................... 149
5.5 Summary and Further Reading ............................................................... 155

EDA.book Page viii Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

Table of Contents
ix

Exercises ............................................................................................................ 158

Chapter 6
Model-Based Clustering
6.1 Overview of Model-Based Clustering .................................................... 163
6.2 Finite Mixtures ........................................................................................... 166
6.2.1 Multivariate Finite Mixtures ........................................................ 167
6.2.2 Component Models - Constraining the Covariances ............... 168
6.3 Expectation-Maximization Algorithm .................................................... 176
6.4 Hierarchical Agglomerative Model-Based Clustering ......................... 181
6.5 Model-Based Clustering ............................................................................ 182
6.6 Generating Random Variables from a Mixture Model ......................... 188
6.7 Summary and Further Reading ............................................................... 192
Exercises ............................................................................................................ 193

Chapter 7
Smoothing Scatterplots
7.1 Introduction ................................................................................................ 197
7.2 Loess ............................................................................................................. 198
7.3 Robust Loess ............................................................................................... 208
7.4 Residuals and Diagnostics ........................................................................ 211
7.4.1 Residual Plots ................................................................................. 212
7.4.2 Spread Smooth ............................................................................... 216
7.4.3 Loess Envelopes - Upper and Lower Smooths ......................... 218
7.5 Bivariate Distribution Smooths ................................................................ 219
7.5.1 Pairs of Middle Smoothings ......................................................... 219
7.5.2 Polar Smoothing ............................................................................ 222
7.6 Curve Fitting Toolbox ............................................................................... 226
7.7 Summary and Further Reading ............................................................... 228
Exercises ............................................................................................................ 229

Part III
Graphical Methods for EDA

Chapter 8
Visualizing Clusters
8.1 Dendrogram ................................................................................................ 233
8.2 Treemaps ..................................................................................................... 235
8.3 Rectangle Plots ........................................................................................... 238
8.4 ReClus Plots ................................................................................................ 244
8.5 Data Image .................................................................................................. 249
8.6 Summary and Further Reading ............................................................... 255
Exercises ............................................................................................................ 256

EDA.book Page ix Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

x
Exploratory Data Analysis with MATLAB

Chapter 9
Distribution Shapes
9.1 Histograms .................................................................................................. 259
9.1.1 Univariate Histograms ................................................................. 259
9.1.2 Bivariate Histograms .................................................................... 266
9.2 Boxplots ....................................................................................................... 268
9.2.1 The Basic Boxplot .......................................................................... 269
9.2.2 Variations of the Basic Boxplot .................................................... 274
9.3 Quantile Plots ............................................................................................. 279
9.3.1 Probability Plots ............................................................................ 279
9.3.2 Quantile-quantile Plot ................................................................... 281
9.3.3 Quantile Plot .................................................................................. 284
9.4 Bagplots ....................................................................................................... 286
9.5 Summary and Further Reading ............................................................... 289
Exercises ............................................................................................................ 289

Chapter 10
Multivariate Visualization
10.1 Glyph Plots ................................................................................................ 293
10.2 Scatterplots ................................................................................................ 294
10.2.1 2-D and 3-D Scatterplots ............................................................. 294
10.2.2 Scatterplot Matrices ..................................................................... 298
10.2.3 Scatterplots with Hexagonal Binning ....................................... 299
10.3 Dynamic Graphics ................................................................................... 301
10.3.1 Identification of Data .................................................................. 301
10.3.2 Linking ......................................................................................... 305
10.3.3 Brushing ........................................................................................ 308
10.4 Coplots ....................................................................................................... 309
10.5 Dot Charts ................................................................................................. 312
10.5.1 Basic Dot Chart ............................................................................ 313
10.5.2 Multiway Dot Chart .................................................................... 314
10.6 Plotting Points as Curves ........................................................................ 318
10.6.1 Parallel Coordinate Plots ............................................................ 318
10.6.2 Andrews’ Curves ......................................................................... 321
10.6.3 More Plot Matrices ...................................................................... 325
10.7 Data Tours Revisited ............................................................................... 326
10.7.1 Grand Tour ................................................................................... 326
10.7.2 Permutation Tour ........................................................................ 328
10.8 Summary and Further Reading ............................................................. 332
Exercises ............................................................................................................ 333

Appendix A
Proximity Measures
A.1 Definitions .................................................................................................. 337
A.1.1 Dissimilarities ............................................................................... 338

EDA.book Page x Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

Table of Contents
xi

A.1.2 Similarity Measures ..................................................................... 340
A.1.3 Similarity Measures for Binary Data ......................................... 340
A.1.4 Dissimilarities for Probability Density Functions ................... 341
A.2 Transformations ........................................................................................ 342
A.3 Further Reading ........................................................................................ 343

Appendix B
Software Resources for EDA
B.1 MATLAB Programs .................................................................................. 345
B.2 Other Programs for EDA .......................................................................... 348
B.3 EDA Toolbox .............................................................................................. 350

Appendix C
Description of Data Sets .................................................................................. 351

Appendix D
Introduction to MATLAB
D.1 What Is MATLAB? .................................................................................... 357
D.2 Getting Help in MATLAB ....................................................................... 358
D.3 File and Workspace Management .......................................................... 358
D.4 Punctuation in MATLAB ......................................................................... 360
D.5 Arithmetic Operators ............................................................................... 361
D.6 Data Constructs in MATLAB .................................................................. 362
Basic Data Constructs ............................................................................ 362
Building Arrays ...................................................................................... 363
Cell Arrays ............................................................................................... 363
Structures ................................................................................................. 364
D.7 Script Files and Functions ........................................................................ 365
D.8 Control Flow .............................................................................................. 366
for Loop ................................................................................................. 366
while Loop ............................................................................................. 366
if-else Statements .............................................................................. 367
switch Statement .................................................................................. 367
D.9 Simple Plotting .......................................................................................... 367
D.10 Where to get MATLAB Information .................................................... 370

Appendix E
MATLAB Functions
E.1 MATLAB ..................................................................................................... 371
E.2 Statistics Toolbox - Versions 4 and 5 ....................................................... 373
E.3 Exploratory Data Analysis Toolbox ........................................................ 374

EDA.book Page xi Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

xii
Exploratory Data Analysis with MATLAB

References ......................................................................................................... 377

EDA.book Page xii Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

xiii

Preface

One of the goals of our first book, Computational Statistics Handbook with
MATLAB® [2002], was to show some of the key concepts and methods of
computational statistics and how they can be implemented in MATLAB.1 A
core component of computational statistics is the discipline known as
exploratory data analysis or EDA. Thus, we see this book as a complement to
the first one with similar goals: to make exploratory data analysis techniques
available to a wide range of users.
Exploratory data analysis is an area of statistics and data analysis, where
the idea is to first explore the data set, often using methods from descriptive
statistics, scientific visualization, data tours, dimensionality reduction, and
others. This exploration is done without any (hopefully!) pre-conceived
notions or hypotheses. Indeed, the idea is to use the results of the exploration
to guide and to develop the subsequent hypothesis tests, models, etc. It is
closely related to the field of data mining, and many of the EDA tools
discussed in this book are part of the toolkit for knowledge discovery and
data mining.
This book is intended for a wide audience that includes scientists,
statisticians, data miners, engineers, computer scientists, biostatisticians,
social scientists, and any other discipline that must deal with the analysis of
raw data. We also hope this book can be useful in a classroom setting at the
senior undergraduate or graduate level. Exercises are included with each
chapter, making it suitable as a textbook or supplemental text for a course in
exploratory data analysis, data mining, computational statistics, machine
learning, and others. Readers are encouraged to look over the exercises,
because new concepts are sometimes introduced in them. Exercises are
computational and exploratory in nature, so there is often no unique answer!
As for the background required for this book, we assume that the reader
has an understanding of basic linear algebra. For example, one should have
a familiarity with the notation of linear algebra, array multiplication, a matrix
inverse, determinants, an array transpose, etc. We also assume that the reader
has had introductory probability and statistics courses. Here one should
know about random variables, probability distributions and density
functions, basic descriptive measures, regression, etc.
In a spirit similar to the first book, this text is not focused on the theoretical
aspects of the methods. Rather, the main focus of this book is on the use of the

1 MATLAB® and Handle Graphics® are registered trademarks of The MathWorks, Inc.

EDA.book Page xiii Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

xiv
Exploratory Data Analysis with MATLAB

EDA methods. Implementation of the methods is secondary, but where
feasible, we show students and practitioners the implementation through
algorithms, procedures, and MATLAB code. Many of the methods are
complicated, and the details of the MATLAB implementation are not
important. In these instances, we show how to use the functions and
techniques. The interested reader (or programmer) can consult the M-files for
more information. Thus, readers who prefer to use some other programming
language should be able to implement the algorithms on their own.
While we do not delve into the theory, we would like to emphasize that the
methods described in the book have a theoretical basis. Therefore, at the end
of each chapter, we provide additional references and resources, so those
readers who would like to know more about the underlying theory will
know where to find the information.
MATLAB code in the form of an Exploratory Data Analysis Toolbox is
provided with the text. This includes the functions, GUIs, and data sets that
are described in the book. This is available for download at

http://lib.stat.cmu.edu

and

http://www.infinityassociates.com

Please review the readme file for installation instructions and information on
any changes. M-files that contain the MATLAB commands for the exercises
are also available for download.
We also make the disclaimer that our MATLAB code is not necessarily the
most efficient way to accomplish the task. In many cases, we sacrificed
efficiency for clarity. Please refer to the example M-files for alternative
MATLAB code, courtesy of Tom Lane of The MathWorks, Inc.
We describe the EDA Toolbox in greater detail in Appendix B. We also
provide website information for other tools that are available for download
(at no cost). Some of these toolboxes and functions are used in the book and
others are provided for informational purposes. Where possible and
appropriate, we include some of this free MATLAB code with the EDA
Toolbox to make it easier for the reader to follow along with the examples and
exercises.
We assume that the reader has the Statistics Toolbox (Version 4 or higher)
from The MathWorks, Inc. Where appropriate, we specify whether the
function we are using is in the main MATLAB software package, Statistics
Toolbox, or the EDA Toolbox. The development of the EDA Toolbox was
mostly accomplished with MATLAB Version 6.5 (Statistics Toolbox, Version
4), so the code should work if this is what you have. However, a new release
of MATLAB and the Statistics Toolbox was introduced in the middle of
writing this book, so we also incorporate information about new
functionality provided in these versions.

EDA.book Page xiv Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

xv

We would like to acknowledge the invaluable help of the reviewers: Chris
Fraley, David Johannsen, Catherine Loader, Tom Lane, David Marchette, and
Jeff Solka. Their many helpful comments and suggestions resulted in a better
book. Any shortcomings are the sole responsibility of the authors. We owe a
special thanks to Jeff Solka for programming assistance with finite mixtures
and to Richard Johnson for allowing us to use his Data Visualization Toolbox
and updating his functions. We would also like to acknowledge all of those
researchers who wrote MATLAB code for methods described in this book
and also made it available for free. We thank the editors of the book series in
Computer Science and Data Analysis for including this text. We greatly
appreciate the help and patience of those at CRC press: Bob Stern, Rob
Calver, Jessica Vakili, and Andrea Demby. Finally, we are indebted to Naomi
Fernandes and Tom Lane at The MathWorks, Inc. for their special assistance
with MATLAB.

Disclaimers

1. Any MATLAB programs and data sets that are included with the book are
provided in good faith. The authors, publishers, or distributors do not
guarantee their accuracy and are not responsible for the consequences of
their use.

2. Some of the MATLAB functions provided with the EDA Toolbox were
written by other researchers, and they retain the copyright. References are
given in Appendix B and in the help section of each function. Unless
otherwise specified, the EDA Toolbox is provided under the GNU license
specifications:

http://www.gnu.org/copyleft/gpl.html

3. The views expressed in this book are those of the authors and do not
necessarily represent the views of the United States Department of Defense
or its components.

Wendy L. and Angel R. Martinez
October 2004

EDA.book Page xv Monday, October 18, 2004 8:31 AM

© 2005 by CRC Press LLC

Part I

Introduction to Exploratory Data Analysis

EDA.book Page 1 Wednesday, October 27, 2004 9:10 PM

© 2005 by CRC Press LLC