IP flows have heavy-tailed packet and byte size distributions. This make them poor candidates for uniform sampling---i.e. selecting 1 in N flows---since omission or inclusion of a large flow can have a large effect on estimated total traffic. Flows selected in this manner are thus unsuitable for use in usage sensitive billing. We propose instead using a size-dependent sampling scheme which gives priority to the larger contributions to customer usage. This turns the heavy tails to our advantage; we can obtain accurate estimates of customer usage from a relatively small number of important samples. The sampling scheme allows us to control error when charging is sensitive to estimated usage only above a given base level. A refinement allows us to strictly limit the chance that a customers estimated usage will exceed their actual usage. Furthermore, we show that a secondary goal, that of controlling the rate at which samples are produced, can be fulfilled provided the billing cycle is sufficiently long. All these claims are supported by experiments on flow traces gathered from a commercial network.

Many routers can generate and export statistics on flows of packets that traverse them. Increasingly, high end routers form flow statistics from only a sampled packet stream in order to manage resource consumption involved.

Due to the importance and complexity of the packet classification problem, a myriad of algorithms and resulting implementations exist. The performance and capacity of many algorithms and classification devices, including TCAMs, depend upon properties of the filter set and query patterns. Unlike microprocessors in the field of computer architecture, there are no standard performance evaluation tools or techniques available to evaluate packet classification algorithms and products. Network service providers are reluctant to distribute copies of real filter sets for security and confidentiality reasons, hence realistic test vectors are a scarce commodity. The small subset of the research community who obtain real filter sets either limit performance evaluation to the small sample space or employ ad hoc methods of modifying those filter sets. In response to this problem, we present ClassBench, a suite of tools for benchmarking packet classification algorithms and devices. ClassBench includes a Filter Set Generator that produces synthetic filter sets that accurately model the characteristics of real filter sets. Along with varying the size of the filter sets, we provide high-level control over the composition of the filters in the resulting filter set. The tools suite also includes a Trace Generator that produces a sequence of packet headers to exercise the synthetic filter set. Along with specifying the relative size of the trace, we provide a simple mechanism for controlling locality of reference in the trace. While we have already found ClassBench to be very useful in our own research, we seek to initiate a broader discussion and solicit input from the community to guide the refinement of the tools and codification of a formal benchmarking methodology.

objects 289-43596 . We wish to estimate the sums !#" %$ &('*)+& , of the sizes of objects of a given color , from a sampled subset of objects. How should the sampling distribution be chosen in order to jointly control both the variance of the estimators - ./ and the number of samples taken? This problem is motivated from network measurement, in which the are the byte sizes of traffic flows reported by routers, and the are the common properties of the packet of the flow, e.g., source and destination IP address. In this paper we propose a sampling scheme that optimally controls the volume of the measurements, and the variance of unbiased usage estimates - 0/ , while retaining usage detail down to the finest level of granularity in the colors. We provide algorithms for dynamic control of sample volumes and evaluate them on flow data gathered from a commercial IP network. The algorithms are simple to implement and robust to variation in network conditions. The work reported here has been applied in the measurement infrastructure of the commercial IP network. To not have employed sampling would have entailed an order of magnitude greater capital expenditure to accommodate the measurement traffic and its processing.

One of the most vexing questions facing researchers interested in the World Wide Web is why users often experience long delays in document retrieval. The Internet's size, complexity, and continued growth make this a difficult question to answer. We describe the Wide Area Web Measurement project (WAWM) which uses an infrastructure distributed across the Internet to study Web performance. The infrastructure enables simultaneous measurements of Web client performance, network performance and Web server performance. The infrastructure uses a Web traffic generator to create representativeworkloads on servers, and both active and passivetools to measure performance characteristics. Initial results based on a prototype installation of the infrastructure are presented in this paper.

In this paper we perform the statistical analysis of an In-ternet communication channel. Our study is based on a Hidden Markov Model (HMM). The channel switches be-tween different states; to each state corresponds the proba-bility that a packet sent by the transmitter will be lost. The transition between the different states of the channel is gov-erned by a Markov chain; this Markov chain is not observed directly, but the received packet flow provides some proba-bilistic information about the current state of the channel, as well as some information about the parameters of the model. In this paper we detail some useful algorithms for the estimation of the channel parameters, and for making inference about the state of the channel. We discuss the relevance of the Markov model of the channel; we also dis-cuss how many states are required to pertinently model a real communication channel.

In this paper we describe an analytical approach for estimating the queuing delay distribution on an Internet link carrying realistic TCP traffic, such as that produced by a large number of finite-size connections transferring files whose sizes are taken from a long-tail distribution.The analytical predictions are validated against detailed simulation experiments and real network measurements.Despite its simplicity, our model proves to be accurate and robust under a variety of operating conditions, and offers novel insights into the impact on the network of long-tail flow length distributions.Our contribution is a performance evaluation methodology that could be usefully employed in network dimensioning and engineering.

crucial so that adaptive applications can make use of Grid environments. Although a large number of systems and tools have been developed to provide such measurement services, the diversity of Grid resources and lack of central control prevent the development of a single monitoring system that can be deployed to answer every application's resource queries for connections between any pair of machines it can use. We propose a standard for representing network entities and measurements of their properties. Our standard enables the exchange of measurements and will allow applications to function even in environments without the particular measurement system for which they were developed. We present an overview of our measurement representation and evaluate its usefulness. We have used the measurement hierarchy to store and exchange measurement data between several systems, and we discuss its usefulness in comparing the output of several measurement tools.

Abstract. Sampling has become an integral part of passive network measurement. This role is driven by the need to control the consumption of resources in the measurement infrastructure under increasing traffic rates and the demand for detailed measurements from applications and service providers. Classical sampling methods play an important role in the current practice of Internet measurement. The aims of this review are (i) to explain the classical sampling methodology in the context of the Internet to readers who are not necessarily acquainted with either, (ii) to give an account of newer applications and sampling methods for passive measurement and (iii) to identify emerging areas that are ripe for the application of statistical expertise. Key words and phrases: Traffic measurement, network management, sampling methods, estimation, packets, flows.

This paper introduces a methodology for interpreting measurement obtained over Internet. The paper is motivated by the fact that a large number of published papers in empirical networking analysis follow a generic framework that might be formalized and generalized to a large class of problem. The objective of this paper is to present an interpretation framework and to illustrate it by examples coming from the networking literature. The aim of the paper is rather to give to the researcher who is confronted to measurements coming from a network some guidelines on how to formalize the way to address interpretation of observations.