Citation Context

...nal tuning parameters. The user then invokes the MapReduce function, passing it to the specification object. The user’s code is linked together with the MapReduce library (implemented in C++). Our original MapReduce paper contains the full program text for this example [8]. More than ten thousand distinct programs have been implemented using MapReduce at Google, including algorithms for large-scale graph processing, text processing, data mining, machine learning, statistical machine translation, and many other areas. More discussion of specific applications of MapReduce can be found elsewhere [8, 16, 7]. 2.2 Types Even though the previous pseudocode is written in terms of string inputs and outputs, conceptually the map and reduce functions supplied by the user have associated types. map (k1,v1) → list(k2,v2) reduce (k2,list(v2)) → list(v2) That is, the input keys and values are drawn from a different domain than the output keys and values. Furthermore, the intermediate keys and values are from the same domain as the output keys and values. 3. Implementation Many different implementations of the MapReduce interface are possible. The right choice depends on the environment. For example, one im...

Citation Context

...many algorithms that can be expressible in certain “summation form”. Phoenix runtime, presented by Colby Ranger et al., is a MapReduce implementation for multi-core systems and multiprocessor systems =-=[34]-=-. The evaluations used by Ranger et al. comprises of typical use cases found in Google's MapReduce paper such as word count, reverse index and also iterative computations such as Kmeans. Some of our d...

Citation Context

...mework, developers implement their application logic using these two primitives, and the data processing tasks are automatically distributed and performed on multiple machines [8] or a multi-core CPU =-=[23]-=-. Thus, this framework reduces the burden on the developers so that they can exploit the parallelism without worrying about the details of the underlying computing resources. Encouraged by the success...

Citation Context

... Google’s proprietary implementation vs. the open-source Hadoop implementation in Java. And in fact, there are many implementations of MapReduce, e.g., targeted specifically for multi-core processors =-=[127]-=-, for GPGPUs [71], for the CELL architecture [126], etc. There are some differences between the MapReduce programming model implemented in Hadoop and Google’s proprietary implementation, which we will...

Citation Context

...t the number of data points (Both axes are in log scale) times have caused the Hadoop results to be almost hundred times more than that of CGL-MapReduce. Cheng-Tao et al. [12] and Colby Ranger et al. =-=[13]-=- both used Kmeans clustering algorithm to evaluate their MapReduce implementations for multi-core and multiprocessor systems. However, other clustering algorithms such as Deterministic Annealing [14],...

Citation Context

...its time in the kernel at one core, this grows to 23% at 48 cores, indicating scalability limitations. 3.7 MapReduce Metis is a MapReduce [20] library for single multicore servers inspired by Phoenix =-=[45]-=-. We use Metis with an application that generates inverted indices. This workload allocates large amounts of memory to hold temporary tables, stressing the kernel memory allocator and soft page fault ...

Citation Context

...omputations and then automate the task of actually making them run in parallel. MapReduce, in particular, has received a great deal of recent attention. Several implementations are publicly available =-=[1, 38]-=-, and researchers have used MapReduce in domains ranging from natural language processing [17], to machine learning [16], to scientific computing [35]. Extensions have also been proposed to enhance an...

Citation Context

...l for summarization, e.g. finding average household income by zip code from a census dataset, but it is also at the heart of the distributed implementation of algorithms such as matrix multiplication =-=[22, 27]-=-. The ability to perform GroupByAggregate at scale is therefore increasingly important, both for traditional data-mining tasks and also for emerging applications such as web-scale machine learning and...

Citation Context

...uilt as 32-bit executables with version 2.6 of the LLVM compiler. 6.1 Methodology We evaluate the performance and scalability of DTHREADS versus CoreDet and pthreads across the PARSEC [9] and Phoenix =-=[24]-=- benchmark suites. We do not include results for bodytrack, fluidanimate, x.264, facesim, vips, and raytrace benchmarks from PARSEC, since they do not currently work with DTHREADS (note that many of t...