Article accepted for publication in the Elsevier “Performance Evaluation” journal

The article results from an ongoing collaboration with Giuliano Casale from Imperial College, London. Elsevier Performance Evaluation is a leading journal in the area of modeling, measurement, and evaluation of performance aspects of computing and communication systems. According to the ISI Science Citation Index (SCI), the journal has a 5-year impact factor of 1.312 (in 2014).

Simon Spinner, Giuliano Casale, Fabian Brosig, and Samuel Kounev. Evaluating Approaches to Resource Demand Estimation. Performance Evaluation, 92:51 - 71, 2015, Elsevier B.V. [ DOI | http | .pdf ]

The article is also available free-of-charge here (until September 27, 2015).

Abstract:
Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.