"... In PAGE 8: ... We observe that the remote suspension time vary largely depending on the com- munication patterns. Table1 characterizes the communication pat- terns observed on the MPI process hosted in the migrating VM. As we can see, CG has the longest remote suspension time, be- cause it communicates frequently with relatively large messages, thus likely takes long time waiting for the outstanding communi- cations.... ..."

"... In PAGE 5: ...mple given in section 3.1. The prefetch instruction itself was inserted using the asm facility of dlxcc. To sketch the impact of latency hiding on our bench- mark set, Table1 shows a characterization of each benchmark in terms of used communication patterns. 2A description of how to parallelizethe Livermore Loops can be found in [Feo88].... In PAGE 6: ... To get close to the reality of massively parallel machines we set the number p of physical processors to 1024. However, most of our benchmarks are not in uenced by the number of physical processors, be- cause the communication patterns { array o set, 2-D grid and indirect addressing (see Table1 ) { do not depend on machine size. Only those benchmarks with reduction operations are in uenced by machine size, because the height of the reduction tree and therefore the number of communication operations evaluates to h = logf(p), where f is the fan-in.... In PAGE 8: ... Still, the codefragments show an excellent improve- ment factor of 9 and above with latency hiding. Comparing the simulation results (Table 2) and the classi cation of our benchmark set ( Table1 ), it be- comes evident that benchmarks with similar communi- cations patterns show also roughly the same behavior... ..."

"... In PAGE 5: ...mple given in section 3.1. The prefetch instruction itself was inserted using the asm facility of dlxcc. To sketch the impact of latency hiding on our bench- mark set, Table1 shows a characterization of each benchmark in terms of used communication patterns. 2A description of how to parallelizethe Livermore Loops can be found in [Feo88].... In PAGE 6: ... To get close to the reality of massively parallel machines we set the number p of physical processors to 1024. However, most of our benchmarks are not in uenced by the number of physical processors, be- cause the communication patterns { array o set, 2-D grid and indirect addressing (see Table1 ) { do not depend on machine size. Only those benchmarks with reduction operations are in uenced by machine size, because the height of the reduction tree and therefore the number of communication operations evaluates to h = logf(p), where f is the fan-in.... In PAGE 8: ... Still, the codefragments show an excellent improve- ment factor of 9 and above with latency hiding. Comparing the simulation results (Table 2) and the classi cation of our benchmark set ( Table1 ), it be- comes evident that benchmarks with similar communi- cations patterns show also roughly the same behavior... ..."

"... In PAGE 6: ... Our respondents indicated that EM provided a greater scope of communication, expanded their patterns of communication, and provided more coordination and control in their communications, relative to VM. Splitting out the items measuring Patterns of Communication (see Table3 ), we find that VM is used significantly more often to communicate with peers than with subordinates (3.4, p gt;O.... ..."

"... In PAGE 5: ... Multiple packets of a message are trans- mitted in (possibly) non-consecutive slots and re-assembled at the destination node. We use a set of parallel applications in Table2 that have various sharing and communication patterns. These appli- cations are from SPLASH 2 benchmark suits [9] and we use their default problem sizes.... In PAGE 5: ... These appli- cations are from SPLASH 2 benchmark suits [9] and we use their default problem sizes. The sharing pattern of each program is also shown in the Table2 using the classifica- tion in [10]. Repl is a sharing pattern in which a data struc- ture is accessed by several processors in a read-mostly man- ner.... In PAGE 5: ... The migratory sharing is further divided into read-mostly (MigR) and read-write (MigRW). The third column in Table2 shows the communication patterns of the benchmark programs. Ocean has a nearest- neighbor communication pattern (NN in Table 2).... In PAGE 5: ... The third column in Table 2 shows the communication patterns of the benchmark programs. Ocean has a nearest- neighbor communication pattern (NN in Table2 ). 27% of remote misses are destined to the adjacent nodes.... ..."

"... In PAGE 4: ... FFT shows a small superlinear speedup, due to cache effects. Table2 summarizes the communication patterns and improvements. Figure 1 summarizes inter-cluster traffic of the original applications.... ..."

"... In PAGE 4: ... FFT shows a small superlinear speedup, due to cache effects. Table2 summarizes the communication patterns and improvements. Figure 1 summarizes inter-cluster traffic of the original applications.... ..."

"... In PAGE 20: ... naive optimized replicated TSP 100 102 100 ASP 100 177 109 QR 100 127 109 ACP 100 115 100 LEQ 100 204 102 Average 100 123 102 While implementing the replicated versions we found that replication is used for three different purposes: sharing data, broadcasting, and collective communication. Table3 shows an overview of the communication patterns of the different applications. TSP and ACP use replication to share data that are read very frequently, but written infrequently and at irregular intervals.... ..."