Results 1  10
of
24
Spatiotemporal compressive sensing and internet traffic matrices
 In SIGCOMM ’09: Proceedings of the ACM SIGCOMM 2009 conference on Data communication
, 2009
"... Many basic network engineering tasks (e.g., traffic engineering, capacity planning, anomaly detection) rely heavily on the availability and accuracy of traffic matrices. However, in practice it is challenging to reliably measure traffic matrices. Missing values are common. This observation brings us ..."
Abstract

Cited by 78 (13 self)
 Add to MetaCart
(Show Context)
Many basic network engineering tasks (e.g., traffic engineering, capacity planning, anomaly detection) rely heavily on the availability and accuracy of traffic matrices. However, in practice it is challenging to reliably measure traffic matrices. Missing values are common. This observation brings us into the realm of compressive sensing, a generic technique for dealing with missing values that exploits the presence of structure and redundancy in many realworld systems. Despite much recent progress made in compressive sensing, existing compressivesensing solutions often perform poorly for traffic matrix interpolation, because real traffic matrices rarely satisfy the technical conditions required for these solutions. To address this problem, we develop a novel spatiotemporal compressive sensing framework with two key components: (i) a new technique called SPARSITY REGULARIZED MATRIX FACTORIZATION (SRMF) that leverages the sparse or lowrank nature of realworld traffic matrices and their spatiotemporal properties, and (ii) a mechanism for combining lowrank approximations with local interpolation procedures. We illustrate our new framework and demonstrate its superior performance in problems involving interpolation with real traffic matrices where we can successfully replace up to 98 % of the values. Evaluation in applications such as network tomography, traffic prediction, and anomaly detection confirms the flexibility and effectiveness of our approach.
Anatomy of a Large European IXP
"... The largest IXPs carry on a daily basis traffic volumes in the petabyte range, similar to what some of the largest global ISPs reportedly handle. This littleknown fact is due to a few hundreds of member ASes exchanging traffic with one another over the IXP’s infrastructure. This paper reports on a ..."
Abstract

Cited by 55 (16 self)
 Add to MetaCart
(Show Context)
The largest IXPs carry on a daily basis traffic volumes in the petabyte range, similar to what some of the largest global ISPs reportedly handle. This littleknown fact is due to a few hundreds of member ASes exchanging traffic with one another over the IXP’s infrastructure. This paper reports on a firstofitskind and indepth analysis of one of the largest IXPs worldwide based on nine months ’ worth of sFlow records collected at that IXP in 2011. A main finding of our study is that the number of actual peering links at this single IXP exceeds the number of total AS links of the peerpeer type in the entire Internet known as of 2010! To explain such a surprisingly rich peering fabric, we examine in detail this IXP’s ecosystem and highlight the diversity of networks that are members at this IXP and connect there with other member ASes for reasons that are similarly diverse, but can be partially inferred from their business types and observed traffic patterns. In the process, we investigate this IXP’s traffic matrix and illustrate what its temporal and structural properties can tell us about the member ASes that generated the traffic in the first place. While our results suggest that these large IXPs can be viewed as a microcosm of the Internet ecosystem itself, they also argue for a reassessment of the mental picture that our community has about this ecosystem.
Network Topologies: Inference, Modelling and Generation
 IEEE COMMUNICATIONS SURVEYS & TUTORIALS
"... Accurate measurement, inference and modelling techniques are fundamental to Internet topology research. Spatial analysis of the Internet is needed to develop network planning, optimal routing algorithms and failure detection measures. A first step towards achieving such goals is the availability of ..."
Abstract

Cited by 38 (11 self)
 Add to MetaCart
Accurate measurement, inference and modelling techniques are fundamental to Internet topology research. Spatial analysis of the Internet is needed to develop network planning, optimal routing algorithms and failure detection measures. A first step towards achieving such goals is the availability of network topologies at different levels of granularity, facilitating realistic simulations of new Internet systems. The main objective of this survey is to familiarize the reader with research on network topology over the past decade. We study techniques for inference, modelling and generation of the Internet topology at both router and administrative level. We also compare the mathematical models assigned to various topologies and the generation tools based on them. We conclude with a look at emerging areas of research and potential future research directions.
Catching the ‘Network Science’ Bug: Insight and Opportunities for the Operations Researchers
 Operations Research
, 2009
"... Accepted for publication by ..."
(Show Context)
Internet Traffic Matrices: A Primer
"... The increasing demand of various services from the Internet has led to an exponential growth of Internet traffic in the last decade, and that growth is likely to continue. With this demand comes the increasing importance of network operations management, planning, provisioning and traffic engineerin ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
The increasing demand of various services from the Internet has led to an exponential growth of Internet traffic in the last decade, and that growth is likely to continue. With this demand comes the increasing importance of network operations management, planning, provisioning and traffic engineering. A key input into these processes is the traffic matrix, and this is the focus of this chapter. The traffic matrix represents the volumes of traffic from sources to destinations in a network. Here, we first explore the various issues involved in measuring and characterising these matrices. The insights obtained are used to develop models of the traffic, depending on the properties of traffic to be captured: temporal, spatial or spatiotemporal properties. The models are then used in various applications, such as the recovery of traffic matrices, network optimisation and engineering activities, anomaly detection and the synthesis of artificial traffic matrices for testing routing protocols. We conclude the chapter by summarising open questions in Internet traffic matrix research and providing a list resources useful for the researcher and practitioner. 1
On Traffic Matrix Completion in the Internet
"... The ability of an ISP to infer traffic volumes that are not directly measurable can be useful for research, engineering, and business intelligence. Previous work has shown that traffic matrix completion is possible, but there is as yet no clear understanding of which ASes are likely to be able to pe ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
The ability of an ISP to infer traffic volumes that are not directly measurable can be useful for research, engineering, and business intelligence. Previous work has shown that traffic matrix completion is possible, but there is as yet no clear understanding of which ASes are likely to be able to perform TM completion, and which traffic flows can be inferred. In this paper we investigate the relationship between the ASlevel topology of the Internet and the ability of an individual AS to perform traffic matrix completion. We take a threestage approach, starting from abstract analysis on idealized topologies, and then adding realistic routing and topologies, and finally incorporating realistic traffic on which we perform actual TM completion. Our first set of results identifies which ASes are bestpositioned to perform TM completion. We show, surprisingly, that for TM completion it does not help for an AS to have many peering links. Rather, the most important factor enabling an AS to perform TM completion is the number of direct customers it has. Our second set of results focuses on which flows can be inferred. We show that topologically close flows are easier to infer, and that flows passing through customers are particularly well suited for inference.
GATEway: Symbiotic InterDomain Traffic Engineering
, 2008
"... There are a group of problems in networking that can most naturally be described as optimization problems (network design, traffic engineering, etc.). There has been a great deal of research devoted to solving these problems, but this research has been concentrated on intradomain problems where one ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
There are a group of problems in networking that can most naturally be described as optimization problems (network design, traffic engineering, etc.). There has been a great deal of research devoted to solving these problems, but this research has been concentrated on intradomain problems where one network operator has complete information and control. An emerging field is interdomain engineering, for instance, traffic engineering between large autonomous networks. Extending intradomain optimization techniques to interdomain problems is often impossible without the measurements and control available within a domain. This paper presents an alternative: we propose a method for traffic engineering that doesn’t require sharing of important information across domains. The method extends the idea of genetic algorithms to allow symbiotic evolution between two parties. Both parties may improve their performance without revealing their data, other than what would be easily observed in any case. We show the method provides large reductions in network congestion, close to the optimal shortest path routing across a pair of networks. The results are highly robust to measurement noise, the method is very flexible, and it can be applied using existing routing.
Topology Discovery of Sparse Random Graphs With Few Participants ∗
, 2011
"... We considerthe taskoftopologydiscoveryofsparserandomgraphsusing endtoendrandom measurements(e.g., delay)between a subset ofnodes, referredto as the participants. The rest of the nodes are hidden, and do not provide any information for topology discovery. We consider topology discovery under two ro ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
We considerthe taskoftopologydiscoveryofsparserandomgraphsusing endtoendrandom measurements(e.g., delay)between a subset ofnodes, referredto as the participants. The rest of the nodes are hidden, and do not provide any information for topology discovery. We consider topology discovery under two routing models: (a) the participants exchange messages along the shortest paths and obtain endtoend measurements, and (b) additionally, the participants exchange messages along the second shortest path. For scenario (a), our proposed algorithm results in a sublinear editdistance guarantee using a sublinear number of uniformly selected participants. For scenario (b), we obtain a much stronger result, and show that we can achieve consistent reconstruction when a sublinear number of uniformly selected nodes participate. This implies that accurate discovery of sparse random graphs is tractable using an extremely small number of participants. We finally obtain a lower bound on the number of participants required by any algorithm to reconstruct the original random graph up to a given edit distance. We also demonstrate that while consistent discovery is tractable for sparse random graphs using a small number of participants, in general, there are graphs which cannot be discovered by any algorithm even with a significant number of participants, and with the availability of endtoend information along all the paths between the participants.
Towards a Meaningful MRA of Traffic Matrices
"... Most research on traffic matrices (TM) has focused on finding models that help with inference, but not with other important tasks such as synthesis of TMs, traffic prediction, or anomaly detection. In this paper we approach the problem of a general model for traffic matrices, and argue that such a m ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Most research on traffic matrices (TM) has focused on finding models that help with inference, but not with other important tasks such as synthesis of TMs, traffic prediction, or anomaly detection. In this paper we approach the problem of a general model for traffic matrices, and argue that such a model must be sparse, i.e., have a small number of parameters in comparison to the size of the TM. A MultiResolution Analysis (MRA) of TMs can provide such a sparse representation. The Diffusion Wavelet (DW) transform is a good choice as a MRA tool here, because it inherently adapts to the structure of the underlying network. The paper describes our construction of the twodimensional version of the DW transform and shows how to use it for our proposed MRA of TMs. The results obtained with operational networks confirm the sparseness of the DWbased TM analysis approach and its applicability to other TMrelated tasks. Categories and Subject Descriptors