Citation Context

...that group, yielding much more accurate approximations. 5 Continuous Queries MapReduce is often used to analyze streams of constantlyarriving data, such as URL access logs [6] and system console logs =-=[30]-=-. Because of traditional constraints on MapReduce, this is done in large batches that can only provide periodic views of activity. This introduces significant latency into a data analysis process that...

Citation Context

...erformance problems in distributed systems. Whereas many rely on end-to-end tracing, others attempt to infer request flows from existing data sources, such as message send/receive events [27] or logs =-=[38]-=-. These latter techniques trade accuracy of re-constructed request flows for ease of using existing monitoring mechanisms. Other techniques rely on black-box metrics and are limited to localizing prob...

Citation Context

...ied to isolate bottlenecks in (semi-) blackbox systems [1], correlate system metrics with performance behaviors [8, 48], avoid performance problems at runtime [52], and diagnose problems through logs =-=[59]-=-. In distributed systems, a performance problem in a single node could spread into the entire system. Of course, there are also many performance problems unique to distributed environment. Performance...

Citation Context

...periences with the Synoptic tool. An overview of the Synoptic tool from a user’s perspective is given in [4]. Other prior work on mining systems logs focused on detecting dependencies [31], anomalies =-=[22, 32, 46, 49]-=-, and performance debugging [40, 41]. That work does not target the problem of finding a concise model for an arbitrary system generating the log. For instance, SALSA [40] and Mochi [41] extract and v...

Citation Context

...o the final answer in orders of magnitude less time. 4.2 Continuous Queries MapReduce is often used to analyze streams of constantly-arriving data, such as URL access logs [4] and system console logs =-=[11]-=-. Because of traditional constraints on MapReduce, this is done in large batches that can only provide periodic views of activity. This introduces significant latency into a data analysis process that...

Citation Context

.... combine SNMP queries with operational logs to analyze failures in the backbone of a regional service provider [19], and Xu et al. parse syslog records to identify anomalies in datacenter operations =-=[34]-=-), we believe ours is the first effort that uses this information to systematically identify and characterize network failures. 3. DATA SOURCES While we intend for our methodology to be generally appl...

Citation Context

...xtracted from logs. This has enabled us to develop an analysis tool applicable to more than simply request processing applications. Other work in identifying problems in distributed systems from logs =-=[37]-=- is restricted to identifying anomalous local problems, while we believe that poor performance commonly manifests from larger implementation issues. We present DISTALYZER, atooltoanalyzelogsof distrib...

Citation Context

...ng increase in the quantity of logs and the difficulty of manual debugging. Automated log analysis is increasing the amount of information that can be extracted from logs, thus increasing their value =-=[26, 30, 3, 16]-=-. Hence, log collection and processing is increasingly important. Scalable data processing is challenging and so it is very desirable to leverage existing tools. MapReduce is emerging as a standard to...

Citation Context

...essage traces and compute dependency paths. D3S [19] uses binary instrumentation to perform online predicate checks. Recent work shows how access to source code can facilitate tasks like log analysis =-=[43]-=- and distributed diagnosis [12]. CarrierIQ17 collects detailed measurements by integrating with the mobile platform, and has drawn criticism for the intrusiveness of their implementation18. Unlike the...