Citation Context

...orkflows and placing replicas asynchronously before execution. Other architectures Sensor networks share our assumption of expensive network bandwidth with respect to compute/storage [32]. The obvious differences in scale (one micro-controller vs. one data center), and the much broader computation model we assume, make most techniques from this space not directly applicable, though relevant as an inspiration. Likewise, stream-processing databases [25, 2] consider a more restrictive model than ours, in which data are always produced at edge nodes and are not replicated. Work in the CDN setting [35] has begun to address geo-distributed analytics, but with much simpler data models. 5. CONCLUSION AND OPEN PROBLEMS The recent explosion of data volumes has reignited the focus on scale-out data analytics, and has fostered the world of Big Data systems. While these paradigms suffice for a single data center, we believe we have reached a new inflection point where the combination of big and geographically distributed data requires new approaches for geo-distributed analytics processing to minimize wide-area bandwidth costs. Centralized approaches together with heuristics such as data reduction ...

Citation Context

...ct ones (e.g., data synopsis) (§2.2). It is thus feasible to incur small additional errors due to approximate fault tolerance, provided that the errors are bounded. Also, the errors can often be amortized after the processing of largevolume and high-speed data streams. Second, although failures are prevalent in distributed systems, their occurrences remain relatively infrequent over the lifetime of stream processing. Thus, incurring errors upon failures should bring limited disturbance. We point out that the power of approximation has been extensively addressed in distributed computing (e.g., [2, 3, 35, 47, 49, 59, 60]). To the best of our knowledge, AF-Stream is the first work that leverages approximation in achieving fault tolerance in distributed stream processing. Our contributions are summarized as follows. First, AF-Stream provides an extensible programming model for general streaming algorithms. In particular, it exports built-in interfaces and primitives that make fault tolerance intrinsically supported and transparent to programmers. Second, AF-Stream realizes approximate fault tolerance, which bounds errors upon failures in two aspects: state divergence and number of lost items. We prove that the ...

Citation Context

...nd not to minimize analytics cost. Spanner [5] supports OLTP workloads on geo-distributed data. Mesa [8] and Riak [2] replicate data across data centers for fault-tolerance but do not address distributed analytics execution. Sensor networks [11] share a similar problem of minimizing the use of expensive network bandwidth, but not our massive scale or the breadth of our computational model. Stream-processing databases process long-standing queries on dispersed data, but the focus of work in this area has been on relatively simple computational models. A representative recent example, JetStream [14], targets a limited computational model centered around aggregation queries and does not, for instance, support joins. 5. CONCLUSION Persistent growth in data volumes and rising interest in sovereignty regulation are rendering the current centralized approaches to geo-scale analytics increasingly untenable. We propose an alternative geo-distributed execution approach that is much better adapted to these requirements. Our system, WANalytics, incorporates several novel techniques inspired by revisiting classical database problems from a networking perspective. This demonstration showcases WANaly...

Citation Context

...p automatically diagnose reported user problems. Analytics in bandwidth-constrained environments As a postdoctoral researcher at Princeton, I am the lead researcher on the (ongoing) JetStream project =-=[9, 10]-=-. We are building a wide-area data analytics engine, designed for bandwidth-constrained environments. Traditional approaches to distributed analytics require users to specify in advance which data wil...