The process of writing large parallel programs is complicated by the need to specify both the parallel behaviour of the program and the algorithm that is to be used to compute its result. This paper introduces evaluation strategies, lazy higher-order functions that control the parallel evaluation of non-strict functional languages. Using evaluation strategies, it is possible to achieve a clean separation between algorithmic and behavioural code. The result is enhanced clarity and shorter parallel programs. Evaluation strategies are a very general concept: this paper shows how they can be used to model a wide range of commonly used programming paradigms, including divideand-conquer, pipeline parallelism, producer/consumer parallelism, and data-oriented parallelism. Because they are based on unrestricted higher-order functions, they can also capture irregular parallel structures. Evaluation strategies are not just of theoretical interest: they have evolved out of our experience in parallelising several large-scale parallel applications, where they have proved invaluable in helping to manage the complexities of parallel behaviour. These applications are described in detail here. The largest application we have studied to date, Lolita, is a 60,000 line natural language parser. Initial results show that for these programs we can achieve acceptable parallel performance, while incurring minimal overhead for using evaluation strategies. 1 Writing Parallel Programs While it is hard to write good sequential programs, it can be considerably harder to write good parallel ones. At Glasgow we have worked on several fairly large parallel programming projects and have slowly, and sometimes painfully, developed a methodology for parallelising sequential programs. The essence of the problem facing the parallel programmer is that, in addition to specifying what value the program should compute, explicitly parallel programs
|
2762
|
Communication and Concurrency
– Milner
- 1989
|
|
382
|
Coordination languages and their significance
– Gelernter, Carriero
- 1992
|
|
357
|
Multilisp: A language for concurrent symbolic computation
– Halstead
- 1985
|
|
216
|
CML: A Higher-order Concurrent Language
– Reppy
- 1991
|
|
200
|
I-structures: Data structures for parallel computing
– Arvind, Pingali
- 1989
|
|
198
|
Lazy task creation: a technique for increasing the granularity of parallel programs
– Mohr, Kranz, et al.
- 1990
|
|
133
|
Supporting Dynamic Data Structures on Distributed Memory Machines
– Rogers, Carlisle, et al.
- 1995
|
|
132
|
Implementation of a portable nested data-parallel language
– Blelloch, Chatterjee, et al.
- 1994
|
|
125
|
Programming parallel algorithms
– Blelloch
- 1996
|
|
94
|
Mul-T: a high-performance parallel Lisp
– Kranz, Halstead, et al.
- 1989
|
|
93
|
Exploiting task and data parallelism on a multicomputer
– Subhlok, Stichnoth, et al.
- 1993
|
|
70
|
Functional Programming for Loosely-Coupled Multiprocessors
– Kelly
- 1989
|
|
66
|
Concurrent Clean
– Nocker, Smetsers, et al.
- 1991
|
|
65
|
C*: An extended C language for data parallel programming
– Rose, Steele
- 1987
|
|
58
|
The design and implementation of programming languages
– Hughes
- 1983
|
|
56
|
Performance Fortran Forum, "High Performance Fortran Language Specification
– High
- 1993
|
|
55
|
GUM: a portable parallel implementation of Haskell
– Trinder
- 1996
|
|
54
|
SISAL: Streams and Iteration in a Single Assignment Language: reference manual version 1.2
– McGraw, Skedzielewski, et al.
- 1985
|
|
45
|
Parallel Programming using Functional Languages (Report CSC 91/R3
– Roe
- 1991
|
|
44
|
Para-Functional Programming
– Hudak
- 1986
|
|
38
|
Global analysis for partitioning non-strict programs into sequential threads
– Traub, Culler, et al.
- 1992
|
|
33
|
Skil: An imperative language with algorithmic skeletons for efficient distributed programming
– Botorog, Kuchen
- 1996
|
|
33
|
Time and Space Profiling for NonStrict, Higher-Order Functional Languages
– Sansom, Jones
- 1995
|
|
32
|
Parallel Skeletons for Structured Composition
– Darlington, Guo, et al.
- 1995
|
|
29
|
Para-functional Programming in Haskell
– Hudak
- 1991
|
|
26
|
Implicit and Explicit Parallel Programming in Haskell
– Jones, Hudak
- 1993
|
|
24
|
Monsoon: An Explicit Token Store Architecture
– Papadopoulos, Culler
- 1990
|
|
23
|
Visualising Granularity in Parallel Programs: A Graphical Winnowing System for Haskell
– Hammond, Loidl, et al.
- 1995
|
|
22
|
Abstract Interpretation and the Parallel Evaluation of Functional Languages
– Burn
- 1987
|
|
22
|
Exploiting Parallelism in Functional Languages: A "Paradigm-Oriented" Approach
– Rabhi
- 1993
|
|
22
|
portability and predictability: The BSP approach to parallel programming
– Scalability
- 1996
|
|
21
|
First-class schedules and virtual maps
– Mirani, Hudak
- 1995
|
|
18
|
Arvind, "*T: A multithreaded massively parallel architecture
– Nikhil, Papadopoulos
- 1992
|
|
17
|
Implementation of Non-strict Functional Programming Languages
– Traub
- 1991
|
|
15
|
Algorithmic Skeletons
– Cole
- 1999
|
|
15
|
Pomset interpretations of parallel function programs
– Hudak, Anderson
- 1987
|
|
14
|
Annotations to Control Parallelism and Reduction Order in the Distributed Evaluation of Functional Programs
– Burton
- 1984
|
|
14
|
Assessing the evaluation transformer model of reduction on the spineless G-machine
– Finne, Burn
- 1993
|
|
12
|
pHluid: The Design of a Parallel Functional Language Implementation on Workstations
– Flanagan, Nikhil
- 1996
|
|
11
|
An Introduction to Database Systems” (4th edition
– Date
- 1976
|
|
11
|
Exploring Parafunctional Programming: Separating the What from the How
– Hudak
- 1988
|
|
9
|
Implementing the evaluation transformer model of reduction on parallel machines
– Burn
- 1991
|
|
5
|
A Compiler Approach to Scalable Concurrent-Program Design
– Foster, Taylor
- 1994
|
|
5
|
pH language proposal
– Nikhil, Arvind, et al.
- 1993
|
|
5
|
A toolkit for parallel functional programming
– Hartel, Hofman, et al.
- 1995
|
|
3
|
Parallel programming with control abstraction
– Crowl, LeBlanc
- 1994
|
|
3
|
Solving Systems of Linear Equations Functionally: a Case Study in Parallelisation
– Loidl, Hammond, et al.
- 1995
|
|
3
|
Modeling speedup of SPMD applications on the Intel Paragon: a case study
– Smirni, Rosti
- 1995
|
|
1
|
Workshop on Functional Programming
– Glasgow
- 1989
|
|
1
|
Computing by Homomorphic Images", in Computer Algebra --- Symbolic and Algebraic
– Lauer
- 1982
|
