Dinda, P. A. and D. R. O'Hallaron: 2000, `Host Load Prediction Using Linear Models'. Cluster Computing 3(4). An earlier version appeared in HPDC '99.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....that this is a useful predictor for CPU resources, for example. Instead of selecting the number of recent measurements to use in a prediction we also consider using only a set of measurements from a previous window of time. Unlike other systems where measurements are taken at regular intervals [8, 34] we make this distinction because our measurements can be spaced irregularly in time. The lack of regular samples means that this approach can reflect trends more accurately than selecting a specific number of previous measurements since it uses temporal windows that capture recent fluctuations ....

....factor of data transfer size, and the same twelve using previous data partitioned by file size. These predictors are summarized in Table 2. As noted earlier, we have primarily considered mean based and median based predictors for our work. Of course, many other variants for predictors are possible [8, 29, 34]. We plan to examine these in future work. Also, rather than choosing just a single prediction technique, we could also evaluate a number of them and choose the most appropriate one on the fly, as is done by NWS. We are considering this strategy for future work. Average based Median based All ....

P. Dinda, D. O'Hallaron, Host Load Prediction Using Linear Models, Cluster Computing, Volume 3, Number 4, 2000.

....Confidence intervals provide a simple abstraction to the application, but still provide sufficient information to enable valid statistical reasoning in the scheduling process. The RTA s response is computed from host load predictions, a topic we have thoroughly reported on in previous papers [5, 8, 7, 6]. We have implemented an extremely low overhead online host load prediction system based on our results and our general purpose RPS Toolkit. In this paper, we describe the algorithm the RTA uses to compute a confidence interval for the running time of a compute bound task from such host load ....

....included self similarity and epochal behavior, simple linear time series models proved to be most appropriate for host load prediction. In fact, the best all around model for host load predictions of 1 30 seconds into the future, in terms of predictive power and low overhead, was the AR(16) model [8]. In this paper, we shall present results that use the AR(16) model, as well as the simple LAST model (last measurement is prediction for all future measurements) and the MEAN model (prediction is the long term arithmetic average of the load signal. It is important to note that linear time ....

[Article contains additional citation context not shown here]

Dinda, P. A. and D. R. O'Hallaron: 2000a, `Host Load Prediction Using Linear Models'. Cluster Computing 3(4). An earlier version of this paper appeared in HPDC '99.

....that this is a useful predictor for CPU resources, for example. Instead of selecting the number of recent measurements to use in a prediction, we also consider using only a set of measurements from a previous window of time. Unlike other systems where measurements are taken at regular intervals [11, 42], our measurements can be spaced irregularly in time. Using temporal windows for irregular samples can reflect trends more accurately than selecting a specific number of previous measurements because they capture recent fluctuations, thereby helping to ensure that recent (and, one hopes, more ....

....over our data sets: 15 predictors each over the entire data set ignoring the context sensitive factor of data transfer size, and the same 15 using previous data partitioned by file size. These predictors are summarized in Figure 4. Of course, many other variants for predictors are possible [11, 36, 42]. Also, rather than choosing just a single prediction technique, we could also evaluate a number of them and choose the most appropriate one on the fly, as is done by the NWS. 5. Delivery Infrastructure Gathering the data is just the first step in building a service to provide predictions for ....

P. Dinda and D. O'Hallaron, Host Load Prediction Using Linear Models, Cluster Computing, Volume 3, no. 4, 2000.

.... to be run as needed, as part of a service (as it is now) to insure uninterruptible performance monitoring, to feed custom GUIs that present the data in new ways, to drive a closely coupled analysis tool such as Argus, or to provide sensors for systems such as Remos [19] NWS [43] and RPS [6, 7]. 2.4 Overhead WatchTower s overhead is similar to that of perfmon. Figure 3 shows the overhead (as of CPU time used) of perfmon and WatchTower as a function of the number of counters and Figure 4 as a function of the measurement rate. With 128 counters, WatchTower can monitor at a peak rate ....

....traces of machine behavior. We also plan to use these traces to study the dynamic resource demands of interactive applications. We will consider more complex data reduction algorithms as needed. 5 Related Work Performance monitoring and prediction systems such as Remos [19] NWS [43] and RPS [6, 7] have limited or nonexistent Windows support. This is not because the code is difficult to port, but rather because of the complex nature of sensors on Windows and the lack of the ability to embed perfmon like tools. WatchTower provides a simple interface to Windows sensors and can be embedded ....

DINDA, P. A., AND O'HALLARON, D. R. Host load prediction using linear models. Cluster Computing 3, 4 (2000).

No context found.

DINDA, P. A., AND O'HALLARON, D. R. Host load prediction using linear models. Cluster Computing 3, 4 (2000), 265--280. To Appear. An earlier version appeared in HPDC'99.

....the injected traffic could become serialized through the shared media. We don t believe that happened often, but the chances would be even lower in a modern switched network. We are also considering what meaning our results have for current efforts to predict resource availability in networks [5, 20]. ....

P. A. Dinda and D. R. O'Hallaron. Host load prediction using linear models. Cluster Computing, 3(4), 2000.

.... or predict such time series data include Remos [9] the Network Weather Service [15] and RPS [5] We focus here specifically on host load, a signal with which the running time of tasks strongly correlates, and that has been shown to be quite predictable over the short range (1 to 30 seconds) [3, 6]. A tension exists between sensors and the applications and schedulers that they serve because different applications are interested in the behavior of the signal over different time scales. For example, a real time scheduling advisor for an interactive application [4, Chapter 6] may be ....

P. A. Dinda and D. R. O'Hallaron. Host load prediction using linear models. Cluster Computing, 3(4), 2000. An earlier version of this paper appeared in HPDC '99.

....scheduling process. Figure 1 shows the structure of the Running Time Advisor (or RTA) system, the broader context of which it is a part, and the queries and responses at each level. The RTA s response is computed from host load predictions, a topic we have thoroughly studied in previous papers [1, 4]. We have found that host load, specifically the Digital Unix 5 second load average sampled at 1 Hz, can be usefully predicted to a 30 second horizon using simple AR(16) models. We have implemented an extremely low overhead online host load prediction system, based on a general purpose toolkit [3] ....

P. A. Dinda and D. R. O'Hallaron. Host load prediction using linear models. Cluster Computing, 3(4), 2000.

....tools. NSClient [15] exposes Windows performance counters as a plug in to the NetSaint [6] monitoring system. None of tools mentioned include a notion of data reduction to capture only the important dynamics in the data. Performance monitoring and prediction systems such as Remos [9] RPS [4][5], and NWS [21] have limited or nonexistent Windows support. This is not because the code is difficult to port, but rather because of the different nature of sensors on Windows verses UNIX, and the lack of the ability to easily embed Perfmonlike tools. WatchTower provides a simple interface to ....

P. Dinda, D. O'Hallaron, Host Load Prediction Using Linear Models, Cluster Computing, Vol. 3, No. 4, 2000.

....Confidence intervals provide a simple abstraction to the application, but still provide sufficient information to enable valid statistical reasoning in the scheduling process. The RTA s response is computed from host load predictions, a topic we have thoroughly reported on in previous papers [5, 8, 7, 6]. We have implemented an extremely low overhead online host load prediction system based on our results and our general purpose RPS Toolkit. In this paper, we describe the algorithm the RTA uses to compute a confidence interval for the running time of a compute bound task from such host load ....

....included self similarity and epochal behavior, simple linear time series models proved to be most appropriate for host load prediction. In fact, the best all around model for host load predictions of 1 30 seconds into the future, in terms of predictive power and low overhead, was the AR(16) model [8]. In this paper, we shall present results that use the AR(16) model, as well as the simple LAST model (last measurement is prediction for all future measurements) and the MEAN model (prediction is the longterm arithmetic average of the load signal. It is important to note that linear time ....

[Article contains additional citation context not shown here]

P. A. Dinda and D. R. O'Hallaron. Host load prediction using linear models. Cluster Computing, 3(4), 2000. An earlier version of this paper appeared in HPDC '99.

....task s CPU demands, predictions of the load on a host, and a confidence level, the RTA predicts, as a confidence interval, the running time of the task on the host. The details of load measurement, prediction, and how the RTA computes its predictions of running time have been thoroughly documented [4, 6, 7, 5]. The Network Weather Service also provides load prediction [18] The RTSA is similar in spirit to the focused addressing algorithm described by Ramamritham, et al. [15] but it is based on sophisticated prediction techniques, makes no assumptions about host cooperation, and is designed to run on ....

....of this, at which point we expect the application will back off and increase sf . The above describes the prediction based scheduling strategy, which is parameterized by the host load predictor that is used. In keeping with results of our study of host load prediction we use the AR(16) predictor [7] and thus we refer to this as the AR(16) strategy. We also studied two additional scheduling strategies: RANDOM and MEASURE. RANDOM simply recommends a randomly selected host. There is little chance of contention among RANDOM based advisors. AR(16) degenerates to RANDOM when all the hosts are ....

[Article contains additional citation context not shown here]

P. A. Dinda and D. R. O'Hallaron. Host load prediction using linear models. Cluster Computing, 3(4), 2000.

....using the RPS Toolkit [6] random: a random server is chosen, load measurement: the server with the lowest one minute load average is chosen, Fig. 3. Speedup and efficiency to 16 processors (8 dual processor nodes) load prediction: the server with the lowest predicted load [7] over the next 10 seconds is chosen, and RTSA: our prediction based real time scheduling advisor [5] is used to select the server. The RTSA attempts simultaneously to help the client s task meet a deadline and to avoid congestion. 2 way SMP (a) Speedup Distributed 2 way SMP Fig. 4. ....

DINDA, P. A., AND O'HALLARON, D. R. Host load prediction using linear models. Cluster Computing 3, 4 (2000). Earlier version in HPDC 1999.

....it does not yet include nonlinear models such as TARs. The choice of models (system identification) is a complex topic in general [4, 5, 1, 28] and also within the context of distributed systems. We have found AR models of order 16 or better to be appropriate for prediction of host load [10], despite load s complex behavior [8] Others have also found that simple models are sufficient for host load [32] Once a model has been chosen, fitted to historical data, and is in use, its error must be monitored to verify that the fit continues to hold. In RPS, this continuous testing (done by ....

....a profound influence on an application s performance, providing this information as well as site to site performance measurements has proven useful for predicting application performance. Research into resource prediction has focused on determining appropriate predictive models for host behavior [10, 32, 22], and network behavior [30, 2, 12] The RPS toolbox used by Remos incorporates many of the models studied by this research. RPS is also available as an independent tool for other research requiring predictive models. One of the products of the Grid Forum is the Grid Monitoring Architecture [25] ....

[Article contains additional citation context not shown here]

P. A. Dinda and D. R. O'Hallaron. Host load prediction using linear models. Cluster Computing, 3(4), 2000. An earlier version appeared in HPDC '99.

No context found.

Dinda, P. A. and D. R. O'Hallaron: 2000, `Host Load Prediction Using Linear Models'. Cluster Computing 3(4). An earlier version appeared in HPDC '99.

No context found.

DINDA, P. A., AND O'HALLARON, D. R. Host load prediction using linear models. Cluster Computing 3, 4 (2000).

No context found.

P. A. Dinda and D. R. O'Hallaron, "Host load prediction using linear models," Cluster Comput., vol. 3, no. 4, pp. 265--280, Jun. 2000.

No context found.

P. Dinda, D. O'Hallaron, "Host load prediction using linear models," Cluster Computing, Vol. 3, No. 4, pp. 265--280, 2000

No context found.

Dinda, P.A. and O'Hallaron, D.R., Host Load Prediction Using Linear Models, Cluster Computing, 3 (2000).

No context found.

P. Dinda and D. O'Hallaron, "Host Load Prediction Using Linear Models," Cluster Computing, 3(4), 2000.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC