ebook img

Algorithms and parallel computing PDF

365 Pages·2011·10.699 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Algorithms and parallel computing

Algorithms and Parallel Computing Algorithms and Parallel Computing Fayez Gebali University of Victoria, Victoria, BC A John Wiley & Sons, Inc., Publication Copyright © 2011 by John Wiley & Sons, Inc. All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyaright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/ permission. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifi cally disclaim any implied warranties of merchantability or fi tness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profi t or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data Gebali, Fayez. Algorithms and parallel computing/Fayez Gebali. p. cm.—(Wiley series on parallel and distributed computing ; 82) Includes bibliographical references and index. ISBN 978-0-470-90210-3 (hardback) 1. Parallel processing (Electronic computers) 2. Computer algorithms. I. Title. QA76.58.G43 2011 004′.35—dc22 2010043659 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 To my children: Michael Monir, Tarek Joseph, Aleya Lee, and Manel Alia Contents Preface xiii List of Acronyms xix 1 Introduction 1 1.1 Introduction 1 1.2 Toward Automating Parallel Programming 2 1.3 Algorithms 4 1.4 Parallel Computing Design Considerations 12 1.5 Parallel Algorithms and Parallel Architectures 13 1.6 Relating Parallel Algorithm and Parallel Architecture 14 1.7 Implementation of Algorithms: A Two-Sided Problem 14 1.8 Measuring Benefi ts of Parallel Computing 15 1.9 Amdahl’s Law for Multiprocessor Systems 19 1.10 Gustafson–Barsis’s Law 21 1.11 Applications of Parallel Computing 22 2 Enhancing Uniprocessor Performance 29 2.1 Introduction 29 2.2 Increasing Processor Clock Frequency 30 2.3 Parallelizing ALU Structure 30 2.4 Using Memory Hierarchy 33 2.5 Pipelining 39 2.6 Very Long Instruction Word (VLIW) Processors 44 2.7 Instruction-Level Parallelism (ILP) and Superscalar Processors 45 2.8 Multithreaded Processor 49 3 Parallel Computers 53 3.1 Introduction 53 3.2 Parallel Computing 53 3.3 Shared-Memory Multiprocessors (Uniform Memory Access [UMA]) 54 3.4 Distributed-Memory Multiprocessor (Nonuniform Memory Access [NUMA]) 56 vii viii Contents 3.5 SIMD Processors 57 3.6 Systolic Processors 57 3.7 Cluster Computing 60 3.8 Grid (Cloud) Computing 60 3.9 Multicore Systems 61 3.10 SM 62 3.11 Communication Between Parallel Processors 64 3.12 Summary of Parallel Architectures 67 4 Shared-Memory Multiprocessors 69 4.1 Introduction 69 4.2 Cache Coherence and Memory Consistency 70 4.3 Synchronization and Mutual Exclusion 76 5 Interconnection Networks 83 5.1 Introduction 83 5.2 Classifi cation of Interconnection Networks by Logical Topologies 84 5.3 Interconnection Network Switch Architecture 91 6 Concurrency Platforms 105 6.1 Introduction 105 6.2 Concurrency Platforms 105 6.3 Cilk++ 106 6.4 OpenMP 112 6.5 Compute Unifi ed Device Architecture (CUDA) 122 7 Ad Hoc Techniques for Parallel Algorithms 131 7.1 Introduction 131 7.2 Defi ning Algorithm Variables 133 7.3 Independent Loop Scheduling 133 7.4 Dependent Loops 134 7.5 Loop Spreading for Simple Dependent Loops 135 7.6 Loop Unrolling 135 7.7 Problem Partitioning 136 7.8 Divide-and-Conquer (Recursive Partitioning) Strategies 137 7.9 Pipelining 139 8 Nonserial–Parallel Algorithms 143 8.1 Introduction 143 8.2 Comparing DAG and DCG Algorithms 143 8.3 Parallelizing NSPA Algorithms Represented by a DAG 145

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.