Bayesian Networks: With Examples in R

252 Pages·2021·18.76 MB·English

by  Marco Scutari, Jean-Baptiste Denis

Bayesian Networks
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Joseph K. Blitzstein, Harvard University, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Recently Published Titles
Statistical Analysis of Financial Data
With Examples in R
James Gentle
Statistical Rethinking
A Bayesian Course with Examples in R and STAN, Second Edition
Richard McElreath
Statistical Machine Learning
A Model-Based Approach
Richard Golden
Randomization, Bootstrap and Monte Carlo Methods in Biology
Fourth Edition
Bryan F. J. Manly, Jorje A. Navarro Alberto
Principles of Uncertainty, Second Edition
Joseph B. Kadane
Beyond Multiple Linear Regression
Applied Generalized Linear Models and Multilevel Models in R
Paul Roback, Julie Legler
Bayesian Thinking in Biostatistics
Gary L. Rosner, Purushottam W. Laud, and Wesley O. Johnson
Linear Models with Python
Julian J. Faraway
Modern Data Science with R, Second Edition
Benjamin S. Baumer, Daniel T. Kaplan, and Nicholas J. Horton
Probability and Statistical Inference
From Basic Principles to Advanced Models
Miltiadis Mavrakakis and Jeremy Penzer
Bayesian Networks
With Examples in R, Second Edition
Marco Scutari and Jean-Baptiste Denis
For more information about this series, please visit: https://www.crcpress.com/
Chapman--Hall/CRC-Texts-in-Statistical-Science/book-series/CHTEXSTASCI
Bayesian Networks
With Examples in R
Second Edition
Marco Scutari
Jean-Baptiste Denis
Second edition published 2022
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2022 Taylor & Francis Group, LLC
First edition published by CRC Press Taylor & Francis Group, 2014
CRC Press is an imprint of Taylor & Francis Group, LLC
The right of Marco Scutari and Jean-Baptiste Denis to be identified as authors of this work has been
asserted by him/her/them in accordance with sections 77 and 78 of the Copyright, Designs and Pat-
ents Act 1988.
Reasonable efforts have been made to publish reliable data and information, but the author and pub-
lisher cannot assume responsibility for the validity of all materials or the consequences of their use.
The authors and publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.
com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermis-
[email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are
used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
ISBN: 978-0-367-36651-3 (hbk)
ISBN: 978-0-429-34743-6 (ebk)
ISBN: 978-1-032-03849-0 (pbk)
DOI: 10.1201/9780429347436
Typeset in LMR10 font
by KnowledgeWorks Global Ltd.
Visit the companion website/eResources: https://www.bnlearn.com/book-crc-2ed/
To the UK,
my home for the last decade
To my wife, Jeanie

Contents
Preface to the Second Edition xi
Preface to the First Edition xiii
1 The Discrete Case: Multinomial Bayesian Networks 1
1.1 Introductory Example: Train-Use Survey . . . . . . . . . . . 1
1.2 Graphical Representation . . . . . . . . . . . . . . . . . . . . 2
1.3 Probabilistic Representation . . . . . . . . . . . . . . . . . . 6
1.4 Estimating the Parameters: Conditional Probability Tables . 10
1.5 Learning the DAG Structure: Tests and Scores . . . . . . . . 13
1.5.1 Conditional Independence Tests. . . . . . . . . . . . . 14
1.5.2 Network Scores . . . . . . . . . . . . . . . . . . . . . . 17
1.6 Using Discrete Bayesian Networks . . . . . . . . . . . . . . . 19
1.6.1 Using the DAG Structure . . . . . . . . . . . . . . . . 20
1.6.2 Using the Conditional Probability Tables . . . . . . . 22
1.6.2.1 Exact Inference . . . . . . . . . . . . . . . . 22
1.6.2.2 Approximate Inference . . . . . . . . . . . . 26
1.7 Plotting Discrete Bayesian Networks . . . . . . . . . . . . . . 28
1.7.1 Plotting DAGs . . . . . . . . . . . . . . . . . . . . . . 28
1.7.2 Plotting Conditional Probability Distributions . . . . 30
1.8 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 33
2 The Continuous Case: Gaussian Bayesian Networks 37
2.1 Introductory Example: Crop Analysis . . . . . . . . . . . . . 37
2.2 Graphical Representation . . . . . . . . . . . . . . . . . . . . 38
2.3 Probabilistic Representation . . . . . . . . . . . . . . . . . . 42
2.4 Estimating the Parameters: Correlation Coefficients . . . . . 45
2.5 Learning the DAG Structure: Tests and Scores . . . . . . . . 48
2.5.1 Conditional Independence Tests. . . . . . . . . . . . . 48
2.5.2 Network Scores . . . . . . . . . . . . . . . . . . . . . . 51
2.6 Using Gaussian Bayesian Networks . . . . . . . . . . . . . . 52
2.6.1 Exact Inference . . . . . . . . . . . . . . . . . . . . . . 52
2.6.2 Approximate Inference . . . . . . . . . . . . . . . . . . 54
2.7 Plotting Gaussian Bayesian Networks . . . . . . . . . . . . . 56
vii
viii
2.7.1 Plotting DAGs . . . . . . . . . . . . . . . . . . . . . . 56
2.7.2 Plotting Conditional Probability Distributions . . . . 58
2.8 More Properties . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.9 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 62
3 The Mixed Case: Conditional Gaussian Bayesian Networks 63
3.1 Introductory Example: Healthcare Costs . . . . . . . . . . . 63
3.2 Graphical and Probabilistic Representation . . . . . . . . . . 64
3.3 Estimating the Parameters: Mixtures of Regressions . . . . . 69
3.4 Learning the DAG Structure: Tests and Scores . . . . . . . . 73
3.5 Using Conditional Gaussian Bayesian Networks . . . . . . . 74
3.6 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 77
4 Time Series: Dynamic Bayesian Networks 79
4.1 Introductory Example: Domotics . . . . . . . . . . . . . . . . 79
4.2 Graphical Representation . . . . . . . . . . . . . . . . . . . . 81
4.3 Probabilistic Representation . . . . . . . . . . . . . . . . . . 82
4.4 Learning a Dynamic Bayesian Network . . . . . . . . . . . . 84
4.5 Using Dynamic Bayesian Networks . . . . . . . . . . . . . . . 86
4.6 Plotting Dynamic Bayesian Networks . . . . . . . . . . . . . 87
4.7 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 88
5 More Complex Cases: General Bayesian Networks 91
5.1 Introductory Example: A&E Waiting Times . . . . . . . . . 91
5.2 Graphical and Probabilistic Representation . . . . . . . . . . 93
5.3 Building the Model in Stan . . . . . . . . . . . . . . . . . . . 96
5.3.1 Generating Data . . . . . . . . . . . . . . . . . . . . . 97
5.3.2 Exploring the Variables . . . . . . . . . . . . . . . . . 98
5.4 Estimating the Parameters in Stan . . . . . . . . . . . . . . . 99
5.5 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 102
6 Theory and Algorithms for Bayesian Networks 105
6.1 Conditional Independence and Graphical Separation . . . . . 105
6.2 Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . . 107
6.3 Markov Blankets . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.4 Moral Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.5 Bayesian Network Learning . . . . . . . . . . . . . . . . . . . 117
6.5.1 Structure Learning . . . . . . . . . . . . . . . . . . . . 121
6.5.1.1 Constraint-Based Algorithms . . . . . . . . . 122
6.5.1.2 Score-Based Algorithms . . . . . . . . . . . . 127
6.5.1.3 Hybrid Algorithms . . . . . . . . . . . . . . . 130
6.5.2 Parameter Learning . . . . . . . . . . . . . . . . . . . 133
ix
6.6 Bayesian Network Inference . . . . . . . . . . . . . . . . . . . 133
6.6.1 Probabilistic Reasoning and Evidence . . . . . . . . . 134
6.6.2 Algorithms for Belief Updating . . . . . . . . . . . . . 136
6.6.2.1 Exact Inference Algorithms . . . . . . . . . . 136
6.6.2.2 Approximate Inference Algorithms . . . . . . 141
6.7 Causal Bayesian Networks . . . . . . . . . . . . . . . . . . . 145
6.8 Evaluating a Bayesian Network . . . . . . . . . . . . . . . . . 148
6.9 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 152
7 Software for Bayesian Networks 155
7.1 An Overview of R Packages . . . . . . . . . . . . . . . . . . . 155
7.1.1 The deal Package . . . . . . . . . . . . . . . . . . . . 157
7.1.2 The catnet Package . . . . . . . . . . . . . . . . . . . 159
7.1.3 The pcalg Package . . . . . . . . . . . . . . . . . . . 161
7.1.4 The abn Package. . . . . . . . . . . . . . . . . . . . . 162
7.2 Stan and BUGS Software Packages . . . . . . . . . . . . . . 164
7.2.1 Stan: A Feature Overview . . . . . . . . . . . . . . . . 165
7.2.2 Inference Based on MCMC Sampling . . . . . . . . . . 166
7.3 Other Software Packages . . . . . . . . . . . . . . . . . . . . 167
7.3.1 BayesiaLab . . . . . . . . . . . . . . . . . . . . . . . . 167
7.3.2 Hugin . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.3.3 GeNIe . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
8 Real-World Applications of Bayesian Networks 171
8.1 Learning Protein-Signalling Networks . . . . . . . . . . . . . 171
8.1.1 A Gaussian Bayesian Network . . . . . . . . . . . . . 173
8.1.2 Discretising Gene Expressions . . . . . . . . . . . . . . 174
8.1.3 Model Averaging . . . . . . . . . . . . . . . . . . . . . 177
8.1.4 Choosing the Significance Threshold . . . . . . . . . . 181
8.1.5 Handling Interventional Data . . . . . . . . . . . . . . 183
8.1.6 Querying the Network . . . . . . . . . . . . . . . . . . 187
8.2 Predicting the Body Composition . . . . . . . . . . . . . . . 190
8.2.1 Aim of the Study . . . . . . . . . . . . . . . . . . . . . 191
8.2.2 Designing the Predictive Approach . . . . . . . . . . . 192
8.2.2.1 Assessing the Quality of a Predictor . . . . . 192
8.2.2.2 The Saturated BN . . . . . . . . . . . . . . . 193
8.2.2.3 Convenient BNs . . . . . . . . . . . . . . . . 194
8.2.3 Looking for Candidate BNs . . . . . . . . . . . . . . . 196
8.3 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 203
x
A Graph Theory 205
A.1 Graphs, Nodes and Arcs . . . . . . . . . . . . . . . . . . . . 205
A.2 The Structure of a Graph . . . . . . . . . . . . . . . . . . . . 206
A.3 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 208
B Probability Distributions 209
B.1 General Features . . . . . . . . . . . . . . . . . . . . . . . . . 209
B.2 Marginal and Conditional Distributions . . . . . . . . . . . . 210
B.3 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . 212
B.3.1 Binomial Distribution . . . . . . . . . . . . . . . . . . 212
B.3.2 Multinomial Distribution . . . . . . . . . . . . . . . . 213
B.3.3 Other Common Distributions . . . . . . . . . . . . . . 213
B.3.3.1 Bernoulli Distribution . . . . . . . . . . . . . 213
B.3.3.2 Poisson Distribution . . . . . . . . . . . . . . 213
B.4 Continuous Distributions . . . . . . . . . . . . . . . . . . . . 214
B.4.1 Normal Distribution . . . . . . . . . . . . . . . . . . . 214
B.4.2 Multivariate Normal Distribution . . . . . . . . . . . . 214
B.4.3 Other Common Distributions . . . . . . . . . . . . . . 215
B.4.3.1 Chi-Square Distribution . . . . . . . . . . . . 215
B.4.3.2 Student’s t Distribution . . . . . . . . . . . . 216
B.4.3.3 Beta Distribution . . . . . . . . . . . . . . . 216
B.4.3.4 Dirichlet Distribution . . . . . . . . . . . . . 217
B.5 Conjugate Distributions . . . . . . . . . . . . . . . . . . . . . 217
B.6 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 218
C A Note about Bayesian Networks 219
C.1 Bayesian Networks and Bayesian Statistics . . . . . . . . . . 219
Glossary 221
Solutions 227
Bibliography 251
Index 257