by Dong Lu, Yi Qiao, Peter A. Dinda, Fabian E. Bustamante, Dong Lu, Yi Qiao, Peter A. Dinda, Fabian E. Bustamante
http://www.cs.northwestern.edu/~pdinda/Papers/NWU-CS-04-34.pdf
Add To MetaCart
Abstract:
The packet pair mechanism has been shown to be a reliable method to measure the bottleneck link capacity and available bandwidth on a network path, and has been widely deployed in tools such as nettimer, IGI, and PTR. However, the available bandwidth is different from the TCP throughput that an application can achieve and the difference can be huge. TCP throughput benchmarking techniques are widely used to probe the TCP throughput for applications, for example in the Network Weather Service (NWS). Unfortunately recent research shows that these techniques often cannot predict TCP throughput well for large transfers. This paper addresses this issue. We begin by statistically characterizing the TCP throughput on the Internet, exploring the strong correlation between TCP flow size and throughput, and the transient end-to-end throughput distribution. We then analyze why benchmarking fails to predict large transfers, and propose a novel yet simple prediction model based on our observations. Our prototype, dualPats, is an application level TCP throughput prediction framework that combines our model with simple time series models and a dynamic probing rate adjustment algorithm that relates intrusiveness to path dynamics. Our analysis and evaluation is based on large scale Internet-based measurements and experiments involving many sites distributed all over the world. Effort sponsored by the National Science Foundation under Grants ANI-0093221, ACI-0112891, ANI-0301108, EIA-0130869, and EIA-0224449. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation (NSF).
Citations
|
452
|
End-to-end routing behavior in the Internet
– Paxson
- 1997
|
|
425
|
The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing
– Wolski, Spring, et al.
- 1998
|
|
319
|
A control-theoretic approach to flow control
– KESHAV
- 1993
|
|
209
|
Using pathchar to estimate internet link characteristics
– DOWNEY
- 1999
|
|
208
|
Measuring bottleneck link speed in packet-switched networks
– CARTER, CROVELLA
- 1996
|
|
200
|
Dynamically forecasting network performance using the network weather service
– Wolski
- 1998
|
|
185
|
SPAND: Shared passive network performance discovery
– SESHAN, STEMM, et al.
- 1997
|
|
184
|
End-to-end available bandwidth: Measurement methodolody, dynamics, and relation with tcp throughput
– JAIN, DOVROLIS
- 2002
|
|
166
|
What do packet dispersion techniques measure
– DOVROLIS, RAMANATHAN, et al.
|
|
146
|
On the Constancy of Internet Path Properties
– Zhang, Duffield, et al.
- 2001
|
|
137
|
Nettimer: A tool for measuring bottleneck link bandwidth
– LAI, BAKER
|
|
120
|
TCP Behavior of a Busy Internet Server: Analysis and Improvements
– Seshan, Balakrishnan, et al.
- 1998
|
|
115
|
SSH — secure login connections over the internet
– YLONEN
- 1996
|
|
100
|
A measurement study of available bandwidth estimation tools
– STRAUSS, KATABI, et al.
- 2003
|
|
91
|
Analyzing Stability in Wide-Area Network Performance
– BALAKRISHNAN, SESHAN, et al.
- 1997
|
|
91
|
pathchirp: Efficient available bandwidth estimation for network paths
– RIBEIRO, RIEDI, et al.
- 2003
|
|
88
|
Evaluation and characterization of available bandwidth probing techniques
– HU, STEENKISTE
- 2003
|
|
79
|
On the Characteristics and Origins of Internet flow rates
– ZHANG, BRESLAU, et al.
- 2002
|
|
53
|
Pathload: A measurement tool for end-to-end available bandwidth
– JAIN, DOVROLIS
- 2002
|
|
46
|
Performance characteristics of mirror servers on the internet
– MYERS, DINDA, et al.
- 1999
|
|
38
|
Dynamics of internet routing information
– CHINOY
- 1993
|
|
35
|
Predicting the performance of wide area data transfers
– VAZHKUDAI, SCHOPF, et al.
- 2002
|
|
30
|
Online prediction of the running time of tasks
– DINDA
- 2002
|
|
29
|
A resource monitoring system for network-aware applications
– LOWEKAMP, MILLER, et al.
- 1998
|
|
27
|
A framework for defining empirical bulk transfer capacity metrics, rfc3148
– MATHIS, ALLMAN
- 2001
|
|
26
|
The statistical properties of host load. Scientific Programming 7, 3,4
– DINDA
- 1999
|
|
20
|
Multivariate resource performance forecasting in the network weather service
– SWANY, WOLSKI
- 2002
|
|
19
|
GridFTP: Protocol extensions to ftp for the grid
– ALLCOCK, BESTER, et al.
- 2001
|
|
17
|
Network characterization service (ncs
– JIN, YANG, et al.
- 2001
|
|
16
|
Representing dynamic performance information in grid environments with the network weather service
– SWANY, WOLSKI
- 2002
|
|
13
|
Predicting sporadic grid data transfers
– VAZHKUDAI, SCHOPF
- 2002
|
|
7
|
The livny and plank-beck problems: Studies in data movement on the computational grid
– ALLEN, WOLSKI
- 2003
|
|
5
|
The macroscopic behavior of the tcp congestionavoidance algorithm
– MATHIS, SEMKE, et al.
- 1997
|
|
5
|
Comparing passive network monitoring of grid application traffic with active probes
– ZANGRILLI, LOWEKAMP
- 2003
|
