N. Islam, A. Prodromidis, and M. S. Squillante, "Dynamic partitioning in different distributed-memory environments". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 244--270, Springer-Verlag, 1996. Lecture Notes in Computer Science Vol. 1162.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....most cases, however, it is more difficult to support malleability in the way an application is written. One way of attaining a limited form of malleability is by creating as many threads in a job as the largest number of processors that would ever be used, and then using multiplexing (or folding [51,38]) to have the job execute on a lesser number of processors. Alternatively, a job can be made malleable by inserting application specific code at particular synchronization points to repartition the data in response to any change in processor allocation. The latter approach is somewhat more ....

N. Islam, A. Prodromidis, and M. S. Squillante, "Dynamic partitioning in different distributed-memory environments". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 244--270, Springer-Verlag, 1996. Lecture Notes in Computer Science Vol. 1162.

....assists developers of fault tolerant, state sharing distributed programs. This pattern focuses on the issues of performance and fault tolerance in distributed systems, and its usefulness is demonstrated through its successful implementation in two IBM projects the Octopus distributed scheduler [IPS96] and a distributed lock manager for the Calypso file system [MDK91] 50 The Recoverable Distributor is used in systems where state sharing between multiple distributed processes is required. It provides a facility to manage the sharing in an efficient manner, and provides recovery facilities in ....

Islam, N., A. Prodromidis, and M.S. Squillante. Dynamic partitioning in different distributed-memory environments. In Proceedings of the 2 Workshop on Job Scheduling Strategies for Parallel Processing. April 1996. pp. 26-49.

.... and runtime support for program restructuring are active areas of research and appear to be feasible for both shared memory and message passing systems (e.g. 27, 20, 6] Although job reconfiguration can involve substantial cost, particularly if massive data movement is required, the results in [6, 11] show that the benefit of better processor scheduling can outweigh the associated cost. This key issue is explored further for EQS and FB PWS in section 4. We assume that the system knows the maximum number of processing nodes that each job can make productive use of, either because this ....

N. Islam, A. Prodromidis, M. S. Squillante, Dynamic Partitioning in Different Distributed-Memory Environments. Proc. IPPS '96 Workshop on Job Scheduling Strategies for Parallel Systems, Honolulu, Hawaii, April 1996.

....EQ. Folding also gave a better MRT than the RA policy when the workload had poor speedup and variable demand, but not if the workload had good speedup and less variability in demand. The impact of repartitioning overheads on dynamic equipartitioning policies is investigated by Islam et al. IPS96] They implement a malleable job system on NOW and an IBM SP 2, both distributed memory systems where repartitioning could be expected to be expensive. Measurements of this system are then used to create an analytic model Chapter 2. Background 28 that includes the repartitioning overhead of ....

....Job Inter arrival Time The most popular choice for modeling job arrivals has been a Poisson process. Numerous studies have used this distribution [MEB88, LV90, ZM90, GST91, CMV94, PS95, CV96, Fei96, Dow97] although some have experimented with the hyper exponential distribution [RSD 98, CV96, IPS96] An exponential distribution is known to have a CV of 1. However, studies of workload traces from parallel computing sites have tended to find inter arrival CV s considerably larger than 1. In the first widely cited in depth study of a parallel job trace, Feitelson [FN95] reports that ....

N. Islam, A. Prodromidis, and M. S. Squillante. Dynamic partitioning in different distributed-memory environments. Lecture Notes in Computer Science, 1162:244--270, 1996.

....2, except that it has 15 phases instead of 250. This lowers the execution time relative to the scheduling overhead. In this case, RA outperforms Folding. It may also be possible for Equipartitioning to perform better than Folding under certain circumstances. Some pertinent results are presented in [IPS96]. Mary Vernon has suggested [V96] that on parallel systems of the future, the job arrival rates would be very low (30 jobs hour) and at such low arrival rates, Folding and Equipartitioning may have similar performance. Areas for future study include validating the results on a larger ....

N. Islam, A. Prodormidis and M. Squillante, "Dynamic Partitioning in Different Distributed-Memory Environments," In this volume.

....This avoids the inefficiencies of executing the job on an unsuitable number of PEs just to see what happens. One problem with dynamic partitioning is the overhead of re partitioning, and possible need to transfer computation state from one processor to another (in distributed memory machines) [446, 441, 543, 289]. The common solution is to limit the rate of such reallocations [422] or to perform them at predefined points in the application where the state is minimal [630, 629, 185, 521] In addition, it should be noted that proposed dynamic partitioning schemes suffer from the same memory oblivion as ....

N. Islam, A. Prodromidis, and M. S. Squillante, "Dynamic partitioning in different distributed-memory environments". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 244--270, Springer-Verlag, 1996. Lecture Notes in Computer Science Vol. 1162.

....is one that changes of its own accord. However, note that evolving and malleable should usually come together, because one job s evolution will cause others to have to reconfigure. Analyzing the benefits of dynamic partitioning and malleable jobs has been the subject of much recent research [36,7,28,39,34,23]. This research typically compares the cost of reconfiguration with the resulting improvement in overall performance. But such comparisons do not give a full picture. In many cases, changing the number of processors allocated to a job requires complex interactions between the operating system and ....

....that has a balanced workload over a particular partition size is likely to remain balanced after a folding operation. Many speedup curves resemble step functions, with poor speedup values for non powers of two number of processors. However, there is some debate over the benefits of folding [28,39]. 4.2 Preemption and Time Slicing Dynamic partitioning, discussed above, requires certain processors to be preempted and re allocated in order to accommodate load changes. Another type of preemption is that used in order to time slice multiple applications, as is commonly done on uniprocessors. ....

N. Islam, A. Prodromidis, and M. Squillante, "Dynamic partitioning in different distributed-memory environments". In Job Scheduling Strategies for Parallel Processing II, D. G. Feitelson and L. Rudolph (eds.), Springer-Verlag, 1996. Lecture Notes in Computer Science.

....most cases, however, it is more difficult to support malleability in the way an application is written. One way of attaining a limited form of malleability is by creating as many threads in a job as the largest number of processors that would ever be used, and then using multiplexing (or folding [52, 39]) to have the job execute on a lesser number of processors. Alternatively, a job can be made malleable by inserting application specific code at particular synchronization points to repartition the data in response to any change in processor allocation. The latter approach is somewhat more ....

N. Islam, A. Prodromidis, and M. S. Squillante, "Dynamic partitioning in different distributedmemory environments". In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp. 244--270, Springer-Verlag, 1996. Lecture Notes in Computer Science Vol. 1162.

....range of workload conditions and selecting the best one. However, many systems must support a diverse workload and no single alternative is optimal under all conditions. For example, the system architecture, system load, and workload characteristics all affect which scheduling policy is optimal [5, 6]. This implies that the selected scheduling policy may be or become suboptimal. One way of addressing this is to design schedulers that can adapt to different requirements. This brings us to the second approach to optimal job scheduling: optimizing a set of design parameters which correspond to ....

....outperform or match any other job scheduling policy for a variety of system configurations. However, for certain systems the cost of rescheduling outweighs the benefits of dynamic partitioning, and that those benefits depend on the rescheduling cost, the system load, and workload characteristics [5, 6]. Due to this and the diverse workload of practical systems, the optimal policy may change frequently. This calls for a hybrid of the dynamic and static scheduling policies (see Figure 1) that can adapt to changes in system demands. 3 Hybrid Equipartitioning Policies In this section we describe ....

Islam, N., A. Prodromidis, and M. Squillante. 1996. "Dynamic Partitioning in Different DistributedMemory Environments," Job Scheduling Strategies for Parallel Processing II, D.G. Feitelson & L. Rudolph (eds.), Springer-Verlag.

....mechanism, and the associated costs of the activity on the resulting performance need to be studied. 3. SYSTEM AND MODELS Our stochastic models of the parallel systems and workloads of interest in this study are based on the broad spectrum of parallel computing environments found in practice [7, 11, 12, 21]. Similarly, our stochastic models of the above dynamic coscheduling mechanisms are based on actual implementations described in the research literature [19, 15] and the costs used for the various scheduling actions have also been drawn from experimental results. 3.1 Parallel System We consider ....

N. Islam, A. Prodromidis, M.S. Squillante. Dynamic partitioning in different distributed-memory environments. Job Scheduling Strategies for Parallel Processing, D.G. Feitelson & L. Rudolph (eds.), 244--270, 1996. Springer-Verlag LNCS Vol. 1162.

....especially those with lower efficiencies and or those with less variable service time requirements [45, 49, 14, 26, 31, 27] This is because the dynamic policy can maintain very efficient node utilizations by adjusting node allocation according to workload changes. On the other hand, studies [31, 40, 18] have demonstrated that the overheads associated with dynamic partitioning in distributed environments can generally limit and or eliminate its potential performance benefits (due to factors such as data job migration, node preemption coordination, and reconfiguration of the application) and that ....

....that resources in its sub partitions should be re allocated, it then invokes the getResource( method to take a resource from a partition and then invokes the giveResource( method to give the resource to another partition. This approach is a generalization of dynamic partitioning with smoothing [18, 20], an instance of FDP that uses timers as triggers. In this paper, we refer to the interval in which FDP timers fire as a smoothing interval and will often use the term FDP with smoothing meaning that a timer trigger is being employed. For example, enforcing a minimum interval of time between ....

N. Islam, A. Prodromidis, and M. S. Squillante. Dynamic partitioning in different distributed-memory environments. In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pages 244--270. Springer-Verlag, 1996. Lecture Notes in Computer Science Vol. 1162.

....can maintain very efficient node utilizations by adjusting node allocation according to workload changes. However, for other application workloads, the overheads associated with greedy dynamic partitioning in distributed environments can generally eliminate its potential performance benefits [7, 2]. In this section we describe a new approach for dynamically adjusting the partitioning of resources at any level of our resource scheduling system hierarchy in a controllable manner while reducing the overheads associated with dynamic partitioning in large scale parallel environments. One dynamic ....

N. Islam, A. Prodromidis, and M. S. Squillante, Dynamic partitioning in different distributedmemory environments, in Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), Springer-Verlag, 1996. Lecture Notes in Comp. Sci. Vol. 1162.

.... form of space sharing can provide the best system performance for a wide variety of application workloads, especially those with lower efficiencies and or those with less variable service time requirements (as our results show) Octopus uses the dynamic partitioning with smoothing approach [10] to determine the number of partitions in the system, denoted by K. According to this approach, the resource management system attempts to balance the node utilizations by reallocating the resources among the applications. Reallocations are triggered when applications arrive or exit the system but ....

N. Islam, A. L. Prodromidis, and M. S. Squillante. Dynamic partitioning in different distributed-memory environments. In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pp 244-- 270. Springer-Verlag, 1996. Lecture Notes in Comp. Sci. Vol. 1162.

....is assumed that the application and the associated runtime system reconfigure the job so it can continue to execute, from that point on, on the new set of processors. For two approaches toward developing reconfigurable applications, we refer the interested reader to the DRMS [20, 19] and Octopus [9, 11, 10] programming environments. The exact epoch when a job is reconfigured is left up to the job. However, it is expected that a job responds to the scheduler s reconfiguration request quickly relative to its total execution time. In our analysis we take into account the delay in responding to the ....

....to subsequent workload changes. Dynamic partitioning tends to alleviate these problems by modifying the processor partitions allocated to programs throughout their execution, but at the expense of increased overhead which depends upon the parallel architecture and application workload; see [39, 16, 23, 17, 35, 9] and the references therein. Much of this previous work has been primarily concerned with uniform access, shared memory parallel computer systems. In these environments, the runtime costs of a dynamic partitioning policy tend to be relatively small and thus the benefits of dynamic partitioning ....

[Article contains additional citation context not shown here]

N. Islam, A. Prodromidis, and M. S. Squillante. Dynamic partitioning in different distributedmemory environments. In Job Scheduling Strategies for Parallel Processing, D. G. Feitelson and L. Rudolph (eds.), pages 244--270. Springer-Verlag, 1996. Lecture Notes in Computer Science Vol. 1162.

....limited due to its inability to adjust scheduling decisions in response to subsequent workload changes. These potential problems are alleviated under dynamic partitioning, where the size of the partition allocated to a job can be modified during its execution, at the expense of increased overhead [6,15,18,25,27 30,43,46,50,53]. Gang scheduling can be viewed as a hybrid scheduling scheme that combines time sharing with space sharing [9 11,33] Under this scheduling approach, jobs with the same resource requirements are collected in groups, and each group is assigned to a partition of the system for a specific amount ....

N. Islam, A. Prodromidis, and M. S. Squillante. Dynamic partitioning in different distributed-memory environments. In Proceedings of the 2nd Workshop on Job Scheduling Strategies for Parallel Processing, pages 155--170, April 1996.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC