Some Sequential Algorithms are Almost Always Parallel

122 Pages·2017·1.16 MB·English

by  Guy Blelloch

Some Sequential Algorithms
are Almost Always Parallel
Guy Blelloch
CarnegieMellonUniversity
July 2017
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 1/35
Joint work with Jeremy Fineman, Phil Gibbons, Yan Gu, Julian
Shun, and Yihan Sun [BFS SPAA’12], [BFGS PPOPP’12],
[SGuBFG SODA’15], [BGuSSu SPAA’16].
Some Sequential Algorithms
?
are Almost Always
?
Parallel
?
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 2/35
Some Sequential Algorithms
?
are Almost Always
?
Parallel
?
Joint work with Jeremy Fineman, Phil Gibbons, Yan Gu, Julian
Shun, and Yihan Sun [BFS SPAA’12], [BFGS PPOPP’12],
[SGuBFG SODA’15], [BGuSSu SPAA’16].
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 2/35
Iterative Sequential Algorithms
for i = 1 to n
do something
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 3/35
Fully Sequential (for arbitrary f)
Each iteration depends on all previous iterations.
Iterative Sequential Algorithms
for i = 1 to n
a[i] = f(a[i-1])
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 4/35
Iterative Sequential Algorithms
for i = 1 to n
a[i] = f(a[i-1])
Fully Sequential (for arbitrary f)
Each iteration depends on all previous iterations.
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 4/35
Fully “parallel”
No dependences among iterations.
Iterative Sequential Algorithms
for i = 1 to n
a[i] = a[i] + 1
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 5/35
Iterative Sequential Algorithms
for i = 1 to n
a[i] = a[i] + 1
Fully “parallel”
No dependences among iterations.
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 5/35
i = 1 2 3 4 5 6 7 8 9
i → j means j depends on i
Partially parallel.
Some dependences, but they are not affected by the data.
√
In this case dependence depth is n.
Iterative Sequential Algorithms
√
for i = n+1 to n
√
a[i] = f(a[i- n])
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 6/35
Partially parallel.
Some dependences, but they are not affected by the data.
√
In this case dependence depth is n.
Iterative Sequential Algorithms
√
for i = n+1 to n
√
a[i] = f(a[i- n])
i = 1 2 3 4 5 6 7 8 9
i → j means j depends on i
SPAA/PODC2017 SomeSequentialAlgorithmsareAlmostAlwaysParallel 6/35