Research Directions in Parallel Functional Programming

506 Pages·1999·13.647 MB·English

by  Kevin Hammond, Greg Michaelson (auth.), Kevin Hammond BSc, PhD, Greg Michaelson BA, MSc, PhD (eds.)

Research Directions in Parallel Functional
Programming
Springer
London
Berlin
Heidelberg
New York
Barcelona
Hong Kong
Milan
Paris
Santa Clara
Singapore
Tokyo
Kevin Hammond and Greg Michaelson (Eds)
Research Directions
in Parallel Functional
Programming
, Springer
Kevin Hammond, BSc, PhD
Division of Computer Science, University of St Andrews, North Haugh, St
Andrews, Fife, KY16 9SS, UK
Greg Michaelson, BA, MSc, PhD
Department of Computing and Electrical Engineering, Heriot-Watt University,
Riccarton, Edinburgh, EH14 4AS, UK
ISBN-13:978-1-85233-092-7
British Library Cataloguing in Publication Data
Research directions in parallel functional programming
I.Functional programming (Computer science) 2.Parallel
programming (Computer science)
LHammond, Kevin II.Michaelson, Greg
005.1'14
ISBN-13:978-1-85233-092-7
Library of Congress Cataloging-in-Publication Data
Research directions in parallel functional programming I Kevin Hammond
and Greg Michaelson (eds.).
p. cm.
ISBN-13:978-1-85233-092-7 e-ISBN-13:978-1-4471-0841-2
DOl: 10.1007/978-1-4471-0841-2
1. Parallel programming (Computer science)--Research.
2. Functional programming (Computer sceince)--Research.
L Hammond, Kevin. II. Michaelson, Greg 1953-
QA76.642.R48 1999 99-29081
005.2'75--dc21
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing of the
publishers, or in the case of repro graphic reproduction in accordance with the terms of licences issued
by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be
sent to the publishers.
© Springer-Verlag London Limited 1999
The use of registered names, trademarks etc. in this publication does not imply, even in the absence of a
specific statement, that such names are exempt from the relevant laws and regulations and therefore
free for general use.
The publisher makes no representation, express or implied, with regard to the accuracy of the
information contained in this book and cannot accept any legal responsibility or liability for any errors
or omissions that may be made.
Typesetting: Camera ready by contributors
34/3830-543210 Printed on acid-free paper SPIN 10696976
To Vicki and Nancy
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. XIII
Overview ..................................................... XVII
Part I. Fundamentals
1. Introduction.............................................. 3
1.1 Introduction........................................... 3
1.2 Language Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Architectural Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 14
1.4 Implementation Issues .................................. 20
1.5 Proof Issues ........................................... 24
1.6 Summary.............................................. 29
2. Foundations.............................................. 31
2.1 Introduction........................................... 31
2.2 A Basic Introduction to Haskell. . . . . . . . . . . . . . . . . . . . . . . . .. 31
2.3 A-Calculus............................................. 39
2.4 Strict Evaluation and the SECD Machine. . . . . . . . . . . . . . . . .. 47
2.5 Lazy Evaluation and Graph Reduction. . . . . . . . . . . . . . . . . . .. 50
2.6 Dataflow.............................................. 58
3. Programming Language Constructs. . . . . . . . . . . . . . . . . . . . . .. 63
3.1 Introduction........................................... 63
3.2 Overview.............................................. 63
3.3 Implicit Parallelism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 66
3.4 Controlled Parallelism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 76
3.5 Explicit Parallelism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 81
3.6 Conclusion............................................ 91
4. Proof ..................................................... 93
4.1 Introduction........................................... 93
4.2 The Basis of Functional Programming: Equations. . . . . . . . . .. 95
4.3 Pattern Matching, Cases and Local Definitions. . . . . . . . . . . .. 96
VIII Contents
4.4 Structural Induction and Recursion. . . . . . . . . . . . . . . . . . . . . .. 99
4.5 Case Study: A Compiler Correctness Proof ................ 100
4.6 General Recursion ...................................... 105
4.7 Partial Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107
4.8 Semantic Approaches ................................... 108
4.9 Strong Functional Programming .......................... 115
4.10 Conclusion ............................................ 118
5. Realisations for Strict Languages ......................... 121
5.1 Introduction ........................................... 121
5.2 Concurrent Program Execution .......................... 123
5.3 System Architectures ................................... 131
5.4 Systems Supporting Strict Languages ..................... 136
6. Realisations for Non-Strict Languages .................... 149
6.1 Introduction ........................................... 149
6.2 Worked Example: the G-Machine ......................... 150
6.3 Analecta of Abstract Machines ........................... 162
6.4 Realisations of the Technology ........................... 175
6.5 Conclusion ............................................ 186
Part II. Current Research Areas
7. Data Parallelism .......................................... 191
7.1 Introduction ........................................... 191
7.2 Data Parallel Combinators ............................... 192
7.3 Active Data Structures .................................. 201
8. Cost Modelling ........................................... 207
8.1 Why Model Costs? ..................................... 207
8.2 Why is Cost Modelling Hard? ............................ 208
8.3 Easy Cost Modelling .................................... 211
8.4 An Example ........................................... 214
8.5 Hard Cost Modelling .................................... 214
8.6 Summary .............................................. 218
9. Shaping Distributions ..................................... 219
9.1 Introduction ........................................... 219
9.2 Shape Theory .......................................... 222
9.3 The FISH Language .................................... 223
9.4 Aspects of the Design of GOLDFISH ...................... 227
9.5 Distributions ........................................... 228
9.6 Cost Modelling ......................................... 231
9.7 Conclusion ............................................ 232
Contents IX
10. Performance Monitoring .................................. 233
10.1 Introduction ........................................... 233
10.2 Analysing an Example Using Current Tools ................ 233
10.3 Data Modelling Parallelism .............................. 239
10.4 Conclusions and Future Work ............................ 246
11. Memory Performance of Dataflow Programs .............. 247
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
11.2 The Dataflow Model of Computation ..................... 247
11.3 Programming Languages ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
11.4 Programming Issues .................................... 251
11.5 The Benchmarks ....................................... 251
11.6 Conclusions and Future Work ............................ 264
12. Portability of Performance in the BSP Model ............ 267
12.1 Introduction ........................................... 267
12.2 The BSP Programming Model ........................... 267
12.3 Simplicity by Restriction ................................ 269
12.4 BSP Programming ...................................... 270
12.5 The BSP Cost Calculus ................................. 271
12.6 BSP Design: Parallel Scan ............................... 275
12.7 BSP Design: Sample Sorting ............................. 277
12.8 BSP and Functional Programming ........................ 283
12.9 Conclusion ............................................ 285
13. Algorithmic Skeletons .................................... 289
13.1 Introduction ........................................... 289
13.2 Patterns of Parallel Computation ......................... 289
13.3 Challenges in Skeletal Programming ...................... 290
13.4 From Theory into Practice ............................... 292
13.5 The Way Ahead ........................................ 303
14. Coordination Languages .................................. 305
14.1 Introduction ........................................... 305
14.2 Caliban: A Common Framework .......................... 305
14.3 Simple Example: Ray Tracing ............................ 308
14.4 Implementing Caliban ................................... 311
14.5 Performance and Performance 'lUning ..................... 314
14.6 Discussion ............................................. 320
14.7 Future Directions ....................................... 320
15. Parallel and Distributed Programming
in Concurrent Clean. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
15.1 Introduction ........................................... 323
15.2 Parallel Programming in Clean ........................... 323
x Contents
15.3 Distributed Programming in Clean ....................... 331
15.4 Conclusion ............................................ 338
16. Functional Process Modelling ............................. 339
16.1 Introduction ........................................... 339
16.2 Modelling Functions as Processes ......................... 341
16.3 Decomposition Strategies for Pipelined Parallelism .......... 344
16.4 Parallel Decomposition of Map ........................... 345
16.5 Parallel Decomposition of Directed Reductions ............. 352
16.6 Related work .......................................... 359
16.7 Conclusion ............................................ 359
17. Validating Programs in Concurrent ML .................. 361
17.1 Introduction ........................................... 361
17.2 Program Analysis ...................................... 361
17.3 Concurrent ML ........................................ 363
17.4 Behaviours ............................................ 366
17.5 A Case Study .......................................... 372
17.6 Conclusion ....................................... , .... 377
18. Explicit Parallelism ....................................... 379
18.1 Motivations ............................................ 379
18.2 Design Choices ......................................... 380
18.3 A Caml Interface to MPI ................................ 385
18.4 An Example ........................................... 391
18.5 Lessons and Perspectives ................................ 393
18.6 Related Work .......................................... 395
18.7 Conclusion ............................................ 396
Part III. Conclusion
19. Large Scale Functional Applications ...................... 399
19.1 Introduction ........................................... 399
19.2 Purely Implicit Parallel Applications ...................... 400
19.3 Semi-Explicit Parallelism ................................ 405
19.4 Coordination Languages ................... " ............ 413
19.5 Explicit Parallelism ..................................... 416
19.6 Derivational Approaches ................................. 419
19.7 Conclusion ............................................ 423
20. Summary ................................................. 427
Glossary ...................................................... 465
Contents XI
Index ......................................................... 481