A.H. Karp. Programming for parallelism. IEEE Computer, May 1987.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....compilers take a program written in a sequential or sharedmemory parallel language, and based on user specified partitioning of data, generate the target parallel program for a multicomputer. For most compilers, this parallel program corresponds to the SPMD (Single Program Multiple Data) model [39], where all processors execute the same program, but operate on distinct data items, thus enabling the exploitation of data parallelism [28] These research efforts include the Fortran D compiler [30, 31] and the Superb compiler [81] both accepting Fortran 77 as the base language. The Crystal ....

A. H. Karp. Programming for parallelism. Computer, 20(5):43--57, May 1987.

....little knowledge about parallel architecture and parallel programming, are hoping to write parallel programs to solve their daily problems. This situation eagerly calls for models and methodologies which can assist programming parallel computers e ectively and correctly. The data parallel model [HS86, Kar87, HL93] turns out to be one of the most successful ones for programming massively parallel computers [Pra92] To support parallel programming, this model basically consists of two parts: # a parallel data structure to model a uniform collection of data which can be organized in a way that each element ....

A. Karp. Programming for parallelism. IEEE Computer, pages 43-57, May 1987.

....experiment. New problems were introduced that produced deadlock. The students used graph theory techniques to show why deadlocked occurred. The last experiment developed was a message passing system. The solution was run on several communication configurations including linear and hypercube [De90, Ka87]. This allowed the students to see how the structure of the parallel communication structure impacted the efficiency of the solution. The experiments were designed to allow the students to gain experience with parallel processing and some of the problems associated with it. The experiments were ....

Karp, Alan H. "Programming for Parallelism," Computer, 20(5), 1987, pp.43-57.

....c architecture. This is especially typical in languages designed for vector machines, where the desire is usually for high vector performance. 22 This implies very simple and regular structures that t the machine s vector registers. Clarity, portability, and ease of programming are sacri ced [45, 46]. While this can result in ecient code for loops, it is not a general approach, and speci cally does not support the parallelism of divide and conquer algorithms. Expressing the Sharing Patterns As noted above, some parallel languages for the shared memory model allow all activities to access ....

....use it to de ne scoping boundaries. For example, Occam provides closed parallel constructs that are similar to those of ParC, but it does not support shared variables. Fortrans with a parallel do construct typically assume that there are no interactions between iterations (like ParC s lparfor) [45]. Some languages have low level unstructured support for parallelism. Dijkstra has observed that closed parallel constructs (such as those provided by ParC) promote a structured style, just like the closed control structures advocated for sequential languages [17] While this idea has caught on ....

A. H. Karp, `Programming for parallelism'. Computer 20(5), 43-51 (1987).

....different versions of the same parallel algorithm, for example different decompositions, is in general rather cumbersome. Comparably small changes may require major program restructuring. An alternative approach is to automatically generate parallel programs in SPMD (Single Program Multiple Data) [Karp87] format, given a data decomposition specification. This approach has recently gained a lot of attention. It has been applied by [Callahan88, Gerndt89, Kennedy89,90] for applications to Fortran, by [Andre90] to C, by [Rogers89] to Id Nouveau, by [Koelbel90] to Kali Fortran, by [Quinn89] to C , and ....

A.H. Karp, "Programming for parallelism," IEEE Computer, May 1987, pp. 43-57.

....efficient code automatically for arbitrary program machine combinations. This insight has led to the approach of parallelism through program annotations, incorporating explicit data decompositions. From these data decomposition specifications, SPMD (Single Process Multiple Data) code [Karp87] can be generated automatically. This approach is followed by [Callahan88, Gerndt89, Kennedy89, 4 Koelbel89] in FORTRAN, by [Rogers89] in Id Nouveau, and by [Quinn89] in C . This concept is also followed in Booster [Paalvast90] 3. Booster Language concepts Booster is a high level, fourth ....

A.H. Karp, "Programming for parallelism," IEEE Computer, May 1987, pp. 43-57.

....restructuring. In this paper, we describe a different explicit approach, that pairs the flexibility of the implicit with the expressiveness of the explicit. In the approach algorithm description and algorithm decomposition are described separately. Efficient SPMD (Single Program Multiple Data) [Karp87] code, in particular communication and synchronization, is generated automatically by the compiler. Furthermore, the compiler uses a model base of target architectures in order to optimize computation and communication efficiency. The approach of inducing parallelism by explicitly decomposing the ....

A.H. Karp, "Programming for Parallelism," IEEE Computer, May 1987, pp. 43-57.

.... to assign work to processors (the ownership rule is the simplest way to do this [5] and restructure loop nests with the aim of avoiding non local accesses as much as possible, and when necessary, optimize communication to transfer remote data [6] The single program multiple data (SPMD) model [7] is used to generate code. Each processor runs the same program but accesses to different parts of the data. Research in the past years has focussed at finding a matrix theory for program transformations 2 to reveal program parallelism [8, 9] or exploit data locality and block transfers [10, ....

Karp A.H., Programming for Parallelism, Computer, vol. 20, no. 5, May 1987.

.... to assign work to processors (the ownership rule is the simplest way to do this [5] and restructure loop nests with the aim of avoiding non local accesses as much as possible, and when necessary, optimize communication to transfer remote data [6] The single program multiple data (SPMD) model [7] is used to generate code. Each processor runs the same program but accesses to different parts of the data. Research in the past years has focussed at finding a matrix theory for program transformations to reveal program parallelism [8, 9] or exploit data locality and block transfers [10, 11] ....

Karp A.H., Programming for Parallelism, Computer, vol. 20, no. 5, May 1987.

....Delirium [LS91] are valuable when used to coordinate coarse grained functional parallelism. However, these languages do not meet the needs of computational scientists because they are inefficient for capturing fine grain data parallelism (of the type described by Hillis and Steele [HS86] and Karp [Kar87] This is mainly due to the fact that existing parallel languages lack both language and compiler support to assist in efficient data placement [PB90] Parallelism must also be explicitly specified because these languages do not provide compilers that can automatically detect and exploit ....

A. Karp. Programming for parallelism. IEEE Computer, 20(5):43--57, May 1987.

....does not necessarily reduce the complexity of the problem. These strategies are designed to find a balance between the minimization of data movement and the maximization of the exploitable parallelism. Using these strategies, the compiler generates a Single Program Multiple Data (SPMD) program [16] for execution on the target machine. In a software coherent NUMA architecture the compiler translates the global references in HPF into local and non local references satisfied by the appropriate message passing statements, usually respecting the owner computes rule (i.e. the processor owning a ....

A.H. Karp. Programming for parallelism. IEEE Computer, May 1987.

....little knowledge about parallel architecture and parallel programming, are hoping to write parallel programs to solve their daily problems. This situation eagerly calls for models and methodologies which can assist programming parallel computers e ectively and correctly. The data parallel model [HS86, Kar87, HL93] turns out to be one of the most successful ones for programming massively parallel computers [Pra92] To support parallel programming, this model basically consists of two parts: a parallel data structure to model a uniform collection of data which can be organized in a way that each element ....

A. Karp. Programming for parallelism. IEEE Computer, pages 43-57, May 1987.

....and message passing mechanisms. Code produced by automatic parallelization tools so far does not reach the efficiency of programs parallelized manually. The programming paradigm that is best suited to utilize the performance capabilities of massively parallel systems is the SPMD paradigm [Kar87] The development of efficient parallel programs has to be supported in all phases of the program design and coding. In this article we focus on tools that support the optimization process of programs that have been coded and are being executed on massively parallel hardware platforms. We ....

A.H. Karp. Programming for parallelism. IEEE Computer, pages 43 -- 57, May 1987.

....parallel applications are programs with explicitly stated parallelism. The parallelism is either defined in a programming model with explicit message passing or by giving data distributions like in High Performance Fortran. The execution of either kind of program corresponds to the SPMD model [9]. The development of efficient parallel programs has to be supported in all phases of the program design and coding. In this article, however, we focus on the support of the optimization process of programs that have been coded and are being executed on massively parallel hardware platforms. We ....

A.H. Karp. Programming for parallelism. IEEE Computer, pages 43 -- 57, May 1987.

....a data parallel operation can be viewed as a single operation for a collection of data. Secondly, in contrast to the programmer s viewpoint of single threaded execution, there is also a simple and e ective parallel execution model for data parallel programs, which is called SPMD execution model (Karp, 1987). In the SPMD model, the data elements in a collection is scattered over the processors in advance, and every processor in a parallel machine executes the same program on the scattered data elements. The primitive operations on a collection of data are immediately executed in parallel, as each ....

Karp, A. (1987). Programming for parallelism. IEEE computer, 20(5), 43-57.

....Such machines are called Non uniform Memory Access (NUMA) machines and exploiting locality of reference is important on these machines as well. The importance of combining parallelism detection with proper data distribution to achieve locality of reference has been eloquently summarized by Karp[11] as follows: 1 : we see that data organization is the key to parallel algorithms even on shared memory systems. It will take some retraining to get programmers to plan their data first and their program flow later. The importance of data management is also a problem for people writing ....

A. Karp. Programming for parallelism. IEEE Computer, 20(5), May 1987.

....interesting parallel programming paradigms and related compiling effort in the past few years. Among them, the one most relevant to the proposed research is the Single Program Multiple Data model (SPMD) which seems to be a promising approach for a wide class of distributed memory MIMD machines [8]. 3 Our approach The portable parallel programming environment we are developing will support the parallel software development process by combining three ingredients: a parallel programming paradigm, a compiler, and a development environment. We have chosen to develop a parallel programming ....

....C In the EPPP system we intend to support data parallel paradigms for C. Our SPMD model for C will support Fortran style array based data parallel paradigms, as well as more general types of programs that may use dynamic data structures. We also go beyond the traditional SPMD model presented in [8], and study some alternative ways to support data parallel paradigm based on the single program multithreaded control model. The key features of EPPP include (1) exploiting both user directed and automatic data partitioning and distribution for array based SPMD model, 2) exploiting data ....

A. H. Karp. Programming for parallelism. IEEE Comput. Mag., 20(5):43--57, May 1987.

....board positions. Since these sets would occupy an excessive amount of main memory, the data structures are implemented compactly as arrays (and even then occupy hundreds of Mbytes of memory) The arrays and the computations associated with them can be decomposed statically, much as with SPMD style [30] parallelism. The second key issue is grainsize control and load balancing. Here, Orca provides much less support than functional languages, and leaves the problem to the programmer, just like message passing systems do. For some applications (retrograde analysis and arc consistency) load ....

A.H. Karp. Programming for Parallelism. IEEE Computer, 20(5):43--57, May 1987.

....of interest, which can be effectively used as building blocks to write parallel programs. This model not only provides the programmer an easily understandable view of a single execution stream of a parallel program, but also makes the parallelization or vectorization process easier [HS86, Kar87, HL93] Despite these promising features, the application of current data parallel programming suffers from two problems (see Section 3 for the examples) Firstly, the main parallel data structures that most languages support are restricted to simple collection data types like arrays or similar ....

A. Karp. Programming for parallelism. IEEE Computer, pages 43--57, May 1987.

....details of how we have configured our system at the Palo Alto Scientific Center that others might find useful are also included. Index terms: Parallel processing, parallel languages, network computing, RS 6000, LINDA 1 Introduction Many packages for programming parallel processors are available. [6] For shared memory machines there are language extensions, sets of compiler directives, and preprocessors. For message passing systems there are several library packages, some provided by the hardware vendor and some by third parties. Shared memory systems view memory as divided into two pieces, ....

Alan H. Karp. Programming for Parallelism. IEEE Computer, pages 43--57, May 1987.

No context found.

A.H. Karp. Programming for parallelism. IEEE Computer, May 1987.

No context found.

KARP, A.- Programming for Parallelism, IEEE Computer, 20, May 43-57, (1987).

No context found.

Karp, A.: Programming for parallelism. IEEE Computer, (1987) 43--57

No context found.

Karp, A. Programming for Parallelism. IEEE Computer, pp. 43-57, May 1987.

No context found.

Alan H. Karp. Programming for Parallelism. Computer, 20(5):43--57, May 1987.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC