Publications

SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation. Grohmann, Johannes; Eismann, Simon; Bauer, Andr{é}; Spinner, Simon; Blum, Johannes; Herbst, Nikolas; Kounev, Samuel; in ACM Transactions on Autonomous and Adaptive Systems (2021). 15(2) Association for Computing Machinery, New York, NY, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

@article{GrEiBaSpBlHeKo2021-TaaS-SARDE,
  abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.},
  address = {New York, NY, USA},
  author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andr\'{e} and Spinner, Simon and Blum, Johannes and Herbst, Nikolas and Kounev, Samuel},
  journal = {ACM Transactions on Autonomous and Adaptive Systems},
  keywords = {t_journalmagazine},
  month = {06},
  number = 2,
  publisher = {Association for Computing Machinery},
  title = {SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation},
  volume = 15,
  year = 2021
}

%0 Journal Article
%1 GrEiBaSpBlHeKo2021-TaaS-SARDE
%A Grohmann, Johannes
%A Eismann, Simon
%A Bauer, Andr{é}
%A Spinner, Simon
%A Blum, Johannes
%A Herbst, Nikolas
%A Kounev, Samuel
%C New York, NY, USA
%D 2021
%I Association for Computing Machinery
%J ACM Transactions on Autonomous and Adaptive Systems
%N 2
%T SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation
%U https://doi.org/10.1145/3463369
%V 15
%X Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}. Bauer, Andr{é}; Herbst, Nikolas; Spinner, Simon; Ali-Eldin, Ahmed; Kounev, Samuel; in IEEE Transactions on Parallel and Distributed Systems (2019). 30(4) 800–813. IEEE.

[ BibTeX ]
[ EndNote ]
[ Download ]

@article{BaHeSpAEKo2019-TPDS-ChameleonAutoscaler,
  author = {Bauer, Andr{\'e} and Herbst, Nikolas and Spinner, Simon and Ali-Eldin, Ahmed and Kounev, Samuel},
  journal = {IEEE Transactions on Parallel and Distributed Systems},
  keywords = {Self-adaptive-systems},
  month = {04},
  number = 4,
  pages = {800–813},
  publisher = {IEEE},
  title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}},
  volume = 30,
  year = 2019
}

%0 Journal Article
%1 BaHeSpAEKo2019-TPDS-ChameleonAutoscaler
%A Bauer, Andr{é}
%A Herbst, Nikolas
%A Spinner, Simon
%A Ali-Eldin, Ahmed
%A Kounev, Samuel
%D 2019
%I IEEE
%J IEEE Transactions on Parallel and Distributed Systems
%N 4
%P 800--813
%T {Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}
%V 30

Online model learning for self-aware computing infrastructures. Spinner, Simon; Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in Journal of Systems and Software (2019). 147 1–16.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]
[ Download ]

Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.

@article{SpGrEiKo2019-JSS-ModelLearning,
  abstract = {Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.},
  author = {Spinner, Simon and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel},
  journal = {Journal of Systems and Software},
  keywords = {Virtualization},
  pages = {1–16},
  title = {Online model learning for self-aware computing infrastructures},
  volume = 147,
  year = 2019
}

%0 Journal Article
%1 SpGrEiKo2019-JSS-ModelLearning
%A Spinner, Simon
%A Grohmann, Johannes
%A Eismann, Simon
%A Kounev, Samuel
%D 2019
%J Journal of Systems and Software
%P 1--16
%R 10.1016/j.jss.2018.09.089
%T Online model learning for self-aware computing infrastructures
%U https://doi.org/10.1016/j.jss.2018.09.089
%V 147
%X Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.

Using Machine Learning for Recommending Service Demand Estimation Approaches. Grohmann, Johannes; Herbst, Nikolas; Spinner, Simon; Kounev, Samuel; in Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018) (2018). 473–480. SciTePress.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]
[ Download ]

Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.

@inproceedings{GrHeSpKo-Closer18-RecServDemEst,
  abstract = {Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.},
  author = {Grohmann, Johannes and Herbst, Nikolas and Spinner, Simon and Kounev, Samuel},
  booktitle = {Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018)},
  keywords = {Optimization},
  month = {03},
  organization = {INSTICC},
  pages = {473–480},
  publisher = {SciTePress},
  title = {Using Machine Learning for Recommending Service Demand Estimation Approaches},
  year = 2018
}

%0 Conference Paper
%1 GrHeSpKo-Closer18-RecServDemEst
%A Grohmann, Johannes
%A Herbst, Nikolas
%A Spinner, Simon
%A Kounev, Samuel
%B Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018)
%D 2018
%I SciTePress
%P 473--480
%R 10.5220/0006761104730480
%T Using Machine Learning for Recommending Service Demand Estimation Approaches
%U https://doi.org/10.5220/0006761104730480
%X Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.
%@ 978-989-758-295-0

{Self-Tuning Resource Demand Estimation}. Grohmann, Johannes; Herbst, Nikolas; Spinner, Simon; Kounev, Samuel; in Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017) (2017). 21–26.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]
[ Download ]

The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.

@inproceedings{GrHeSpKo2017-ICAC-RDE,
  abstract = {The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.},
  author = {Grohmann, Johannes and Herbst, Nikolas and Spinner, Simon and Kounev, Samuel},
  booktitle = {Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017)},
  keywords = {Self-adaptive-systems},
  month = {07},
  pages = {21–26},
  title = {{Self-Tuning Resource Demand Estimation}},
  year = 2017
}

%0 Conference Paper
%1 GrHeSpKo2017-ICAC-RDE
%A Grohmann, Johannes
%A Herbst, Nikolas
%A Spinner, Simon
%A Kounev, Samuel
%B Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017)
%D 2017
%P 21--26
%R 10.1109/ICAC.2017.19
%T {Self-Tuning Resource Demand Estimation}
%U https://doi.org/10.1109/ICAC.2017.19
%X The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.

Self-Aware Resource Management in Virtualized Data Centers. Spinner, Simon; (2017, July). University of W{ü}rzburg, Germany.

[ BibTeX ]
[ EndNote ]
[ URL ]

@phdthesis{Spinner2017-phd,
  author = {Spinner, Simon},
  keywords = {descartes},
  month = {07},
  school = {University of W{\"u}rzburg, Germany},
  title = {Self-Aware Resource Management in Virtualized Data Centers},
  year = 2017
}

%0 Thesis
%1 Spinner2017-phd
%A Spinner, Simon
%D 2017
%T Self-Aware Resource Management in Virtualized Data Centers
%U https://opus.bibliothek.uni-wuerzburg.de/frontdoor/index/index/docId/15375

{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}. Kounev, Samuel; Huber, Nikolaus; Brosig, Fabian; Spinner, Simon; Baehr, Manuel; in Jan J{ü}rjens, Kurt Schneider (Hrsg.): Software Engineering 2017 (SE 2017), Fachtagung des GI-Fachbereichs Softwaretechnik, 21.-24. Februar 2017, Hannover, Germany (2017). Gesellschaft f{ü}r Informatik (GI), Hannover, Germany.

[ BibTeX ]
[ EndNote ]
[ Download ]

@inproceedings{KoHuBrSpBa2017-SE-DML,
  address = {Hannover, Germany},
  author = {Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian and Spinner, Simon and Baehr, Manuel},
  booktitle = {Jan J{\"u}rjens, Kurt Schneider (Hrsg.): Software Engineering 2017 (SE 2017), Fachtagung des GI-Fachbereichs Softwaretechnik, 21.-24. Februar 2017, Hannover, Germany},
  keywords = {descartes},
  month = {02},
  publisher = {Gesellschaft f{\"u}r Informatik (GI)},
  series = {Lecture Notes in Informatics (LNI)},
  title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}},
  year = 2017
}

%0 Conference Paper
%1 KoHuBrSpBa2017-SE-DML
%A Kounev, Samuel
%A Huber, Nikolaus
%A Brosig, Fabian
%A Spinner, Simon
%A Baehr, Manuel
%B Jan J{ü}rjens, Kurt Schneider (Hrsg.): Software Engineering 2017 (SE 2017), Fachtagung des GI-Fachbereichs Softwaretechnik, 21.-24. Februar 2017, Hannover, Germany
%C Hannover, Germany
%D 2017
%I Gesellschaft f{ü}r Informatik (GI)
%T {Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}

{Reference Scenarios for Self-Aware Computing}. Kephart, Jeffrey O.; Maggio, Martina; Diaconescu, Ada; Giese, Holger; Hoffmann, Henry; Kounev, Samuel; Koziolek, Anne; Lewis, Peter; Robertsson, Anders; Spinner, Simon; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ BibTeX ]
[ EndNote ]
[ Download ]

@incollection{KeMaDiGiHoKoKoLeRoSp2017-SelfAwareComputing-ReferenceScenarios,
  address = {{Berlin Heidelberg, Germany}},
  author = {Kephart, Jeffrey O. and Maggio, Martina and Diaconescu, Ada and Giese, Holger and Hoffmann, Henry and Kounev, Samuel and Koziolek, Anne and Lewis, Peter and Robertsson, Anders and Spinner, Simon},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Reference Scenarios for Self-Aware Computing}},
  year = 2017
}

%0 Book Section
%1 KeMaDiGiHoKoKoLeRoSp2017-SelfAwareComputing-ReferenceScenarios
%A Kephart, Jeffrey O.
%A Maggio, Martina
%A Diaconescu, Ada
%A Giese, Holger
%A Hoffmann, Henry
%A Kounev, Samuel
%A Koziolek, Anne
%A Lewis, Peter
%A Robertsson, Anders
%A Spinner, Simon
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Reference Scenarios for Self-Aware Computing}

{Run-time Models for Online Performance and Resource Management in Data Centers}. Spinner, Simon; Filieri, Antonio; Kounev, Samuel; Maggio, Martina; Robertsson, Anders; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ BibTeX ]
[ EndNote ]
[ Download ]

@incollection{SpFiKoMaRo2016-SelfawareComputing-RuntimeModels,
  address = {{Berlin Heidelberg, Germany}},
  author = {Spinner, Simon and Filieri, Antonio and Kounev, Samuel and Maggio, Martina and Robertsson, Anders},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Run-time Models for Online Performance and Resource Management in Data Centers}},
  year = 2017
}

%0 Book Section
%1 SpFiKoMaRo2016-SelfawareComputing-RuntimeModels
%A Spinner, Simon
%A Filieri, Antonio
%A Kounev, Samuel
%A Maggio, Martina
%A Robertsson, Anders
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Run-time Models for Online Performance and Resource Management in Data Centers}

{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}. Huber, Nikolaus; Brosig, Fabian; Spinner, Simon; Kounev, Samuel; B{ä}hr, Manuel; in IEEE Transactions on Software Engineering (TSE) (2017). 43(5) 432–452. IEEE Computer Society.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.

@article{HuBrSpKoBa2017-TSE-DML,
  abstract = {Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.},
  author = {Huber, Nikolaus and Brosig, Fabian and Spinner, Simon and Kounev, Samuel and B{\"a}hr, Manuel},
  journal = {IEEE Transactions on Software Engineering (TSE)},
  keywords = {DQL},
  number = 5,
  pages = {432–452},
  publisher = {IEEE Computer Society},
  title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}},
  volume = 43,
  year = 2017
}

%0 Journal Article
%1 HuBrSpKoBa2017-TSE-DML
%A Huber, Nikolaus
%A Brosig, Fabian
%A Spinner, Simon
%A Kounev, Samuel
%A B{ä}hr, Manuel
%D 2017
%I IEEE Computer Society
%J IEEE Transactions on Software Engineering (TSE)
%N 5
%P 432--452
%T {Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}
%U http://dx.doi.org/10.1109/TSE.2016.2613863
%V 43
%X Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.

{Self-Awareness of Cloud Applications}. Iosup, Alexandru; Zhu, Xiaoyun; Merchant, Arif; Kalyvianaki, Eva; Maggio, Martina; Spinner, Simon; Abdelzaher, Tarek; Mengshoel, Ole; Bouchenak, Sara; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ BibTeX ]
[ EndNote ]
[ Download ]

@incollection{IoZhMeKaMaSpAbMeBo2016-SelfawareComputing-CloudApplications,
  address = {{Berlin Heidelberg, Germany}},
  author = {Iosup, Alexandru and Zhu, Xiaoyun and Merchant, Arif and Kalyvianaki, Eva and Maggio, Martina and Spinner, Simon and Abdelzaher, Tarek and Mengshoel, Ole and Bouchenak, Sara},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {t_bookchapter},
  publisher = {{Springer Verlag}},
  title = {{Self-Awareness of Cloud Applications}},
  year = 2017
}

%0 Book Section
%1 IoZhMeKaMaSpAbMeBo2016-SelfawareComputing-CloudApplications
%A Iosup, Alexandru
%A Zhu, Xiaoyun
%A Merchant, Arif
%A Kalyvianaki, Eva
%A Maggio, Martina
%A Spinner, Simon
%A Abdelzaher, Tarek
%A Mengshoel, Ole
%A Bouchenak, Sara
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Self-Awareness of Cloud Applications}

{Online Learning of Run-time Models for Performance and Resource Management in Data Centers}. Walter, J{ü}rgen; Marco, Antinisca Di; Spinner, Simon; Inverardi, Paola; Kounev, Samuel; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

@incollection{WaMaSpInKo-learning,
  abstract = {In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.},
  address = {{Berlin Heidelberg, Germany}},
  author = {Walter, J{\"u}rgen and Marco, Antinisca Di and Spinner, Simon and Inverardi, Paola and Kounev, Samuel},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Online Learning of Run-time Models for Performance and Resource Management in Data Centers}},
  year = 2017
}

%0 Book Section
%1 WaMaSpInKo-learning
%A Walter, J{ü}rgen
%A Marco, Antinisca Di
%A Spinner, Simon
%A Inverardi, Paola
%A Kounev, Samuel
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Online Learning of Run-time Models for Performance and Resource Management in Data Centers}
%X In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

{Automated Parameterization of Performance Models from Measurements}. Casale, Giuliano; Spinner, Simon; Wang, Weikun; in {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)} (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Estimating parameters of performance models from empirical measurements is a critical task, which often has a major influence on the predictive accuracy of a model. This tutorial presents the problem of parameter estimation in queueing systems and queueing networks. The focus is on reliable estimation of the {\em arrival rates} of the requests and of the {\em service demands} they place at the servers. The tutorial covers common estimation techniques such as regression methods, maximum-likelihood estimation, and moment-matching, discussing their sensitivity with respect to data and model characteristics. The tutorial also demonstrates the automated estimation of model parameters using new open source tools.

@inproceedings{CaSpWa2016-ICPE-AutomatedParameterizationTutorial,
  abstract = {Estimating parameters of performance models from empirical measurements is a critical task, which often has a major influence on the predictive accuracy of a model. This tutorial presents the problem of parameter estimation in queueing systems and queueing networks. The focus is on reliable estimation of the {\em arrival rates} of the requests and of the {\em service demands} they place at the servers. The tutorial covers common estimation techniques such as regression methods, maximum-likelihood estimation, and moment-matching, discussing their sensitivity with respect to data and model characteristics. The tutorial also demonstrates the automated estimation of model parameters using new open source tools.},
  author = {Casale, Giuliano and Spinner, Simon and Wang, Weikun},
  booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}},
  keywords = {LibReDE},
  month = {03},
  title = {{Automated Parameterization of Performance Models from Measurements}},
  year = 2016
}

%0 Conference Paper
%1 CaSpWa2016-ICPE-AutomatedParameterizationTutorial
%A Casale, Giuliano
%A Spinner, Simon
%A Wang, Weikun
%B {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}
%D 2016
%T {Automated Parameterization of Performance Models from Measurements}
%X Estimating parameters of performance models from empirical measurements is a critical task, which often has a major influence on the predictive accuracy of a model. This tutorial presents the problem of parameter estimation in queueing systems and queueing networks. The focus is on reliable estimation of the {\em arrival rates} of the requests and of the {\em service demands} they place at the servers. The tutorial covers common estimation techniques such as regression methods, maximum-likelihood estimation, and moment-matching, discussing their sensitivity with respect to data and model characteristics. The tutorial also demonstrates the automated estimation of model parameters using new open source tools.

{A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}. Spinner, Simon; Walter, J{ü}rgen; Kounev, Samuel; in {Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)} (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.

@inproceedings{SpWaKo2015-WOSPC-ReferenceArchitecture,
  abstract = {Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.},
  author = {Spinner, Simon and Walter, J{\"u}rgen and Kounev, Samuel},
  booktitle = {{Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}},
  keywords = {t_workshop},
  month = {03},
  title = {{A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}},
  year = 2016
}

%0 Conference Paper
%1 SpWaKo2015-WOSPC-ReferenceArchitecture
%A Spinner, Simon
%A Walter, J{ü}rgen
%A Kounev, Samuel
%B {Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}
%D 2016
%T {A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}
%X Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.

{Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}. M{ü}ller, Christoph; Rygielski, Piotr; Spinner, Simon; Kounev, Samuel; in Electronic Notes in Theoretical Computer Science (2016). 327 71–91. Elsevier.

The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

@article{MuRySpKo2016-PASM-QPN-LQN-Transformation,
  abstract = {Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.},
  author = {M{\"u}ller, Christoph and Rygielski, Piotr and Spinner, Simon and Kounev, Samuel},
  journal = {Electronic Notes in Theoretical Computer Science},
  keywords = {QPN},
  note = {The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016},
  pages = {71–91},
  publisher = {Elsevier},
  title = {{Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}},
  volume = 327,
  year = 2016
}

%0 Journal Article
%1 MuRySpKo2016-PASM-QPN-LQN-Transformation
%A M{ü}ller, Christoph
%A Rygielski, Piotr
%A Spinner, Simon
%A Kounev, Samuel
%D 2016
%I Elsevier
%J Electronic Notes in Theoretical Computer Science
%P 71--91
%T {Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}
%U http://www.sciencedirect.com/science/article/pii/S157106611630069X
%V 327
%X Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

{Getting the Data}. Happe, Jens; Klatt, Benjamin; K{ü}ster, Martin; Brosig, Fabian; Wert, Alexander; Spinner, Simon; Stier, Christian; Koziolek, Heiko; in {Modeling and Simulating Software Architectures - The Palladio Approach}, R. H. Reussner, S. Becker, J. Happe, R. Heinrich, A. Koziolek, H. Koziolek, M. Kramer, K. Krogmann (eds.) (2016). MIT Press, Cambridge, MA.

In Press

[ BibTeX ]
[ EndNote ]

@incollection{HaKlKuBrWeSpStKo-2016-MITPress-Palladio,
  address = {Cambridge, MA},
  author = {Happe, Jens and Klatt, Benjamin and K{\"u}ster, Martin and Brosig, Fabian and Wert, Alexander and Spinner, Simon and Stier, Christian and Koziolek, Heiko},
  booktitle = {{Modeling and Simulating Software Architectures - The Palladio Approach}},
  editor = {Reussner, Ralf H. and Becker, Steffen and Happe, Jens and Heinrich, Robert and Koziolek, Anne and Koziolek, Heiko and Kramer, Max and Krogmann, Klaus},
  keywords = {t_bookchapter},
  note = {In Press},
  publisher = {MIT Press},
  title = {{Getting the Data}},
  year = 2016
}

%0 Book Section
%1 HaKlKuBrWeSpStKo-2016-MITPress-Palladio
%A Happe, Jens
%A Klatt, Benjamin
%A K{ü}ster, Martin
%A Brosig, Fabian
%A Wert, Alexander
%A Spinner, Simon
%A Stier, Christian
%A Koziolek, Heiko
%B {Modeling and Simulating Software Architectures - The Palladio Approach}
%C Cambridge, MA
%D 2016
%E Reussner, Ralf H.
%E Becker, Steffen
%E Happe, Jens
%E Heinrich, Robert
%E Koziolek, Anne
%E Koziolek, Heiko
%E Kramer, Max
%E Krogmann, Klaus
%I MIT Press
%T {Getting the Data}

{Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques}. Willnecker, Felix; Dlugi, Markus; Brunnert, Andreas; Spinner, Simon; Kounev, Samuel; Krcmar, Helmut; in Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015), M. Beltr{á}n, W. Knottenbelt, J. Bradley (eds.) (2015). (Vol. 9272) 115–129. Springer.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.

@inproceedings{WiDlBrSpKoKr2015-EPEW-LibredePMWT,
  abstract = {Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.},
  author = {Willnecker, Felix and Dlugi, Markus and Brunnert, Andreas and Spinner, Simon and Kounev, Samuel and Krcmar, Helmut},
  booktitle = {Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015)},
  editor = {Beltr{\'a}n, Marta and Knottenbelt, William and Bradley, Jeremy},
  keywords = {t_workshop},
  month = {{August}},
  pages = {115-129},
  publisher = {Springer},
  series = {Lecture Notes in Computer Science},
  title = {{Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques}},
  volume = 9272,
  year = 2015
}

%0 Conference Paper
%1 WiDlBrSpKoKr2015-EPEW-LibredePMWT
%A Willnecker, Felix
%A Dlugi, Markus
%A Brunnert, Andreas
%A Spinner, Simon
%A Kounev, Samuel
%A Krcmar, Helmut
%B Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015)
%D 2015
%E Beltr{á}n, Marta
%E Knottenbelt, William
%E Bradley, Jeremy
%I Springer
%P 115-129
%T {Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques}
%V 9272
%X Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.

{Resource Demand Estimation in Distributed, Service-Oriented Applications using LibReDE}. Spinner, Simon; (2015, November).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{talk-Spinner-SSP15,
  author = {Spinner, Simon},
  howpublished = {Technical Talk at Symposion on Software Performance (SSP) 2015},
  keywords = {descartes},
  month = 11,
  title = {{Resource Demand Estimation in Distributed, Service-Oriented Applications using LibReDE}},
  year = 2015
}

%0 Generic
%1 talk-Spinner-SSP15
%A Spinner, Simon
%D 2015
%T {Resource Demand Estimation in Distributed, Service-Oriented Applications using LibReDE}

{Evaluating Approaches to Resource Demand Estimation}. Spinner, Simon; Casale, Giuliano; Brosig, Fabian; Kounev, Samuel; in Performance Evaluation (2015). 92 51–71. Elsevier B.V.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.

@article{SpCaBrKo2015-PEVA-ResDemEstSurvey,
  abstract = {Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.},
  author = {Spinner, Simon and Casale, Giuliano and Brosig, Fabian and Kounev, Samuel},
  journal = {Performance Evaluation},
  keywords = {Instrumentation_profiling_and_workload_characterization},
  month = 10,
  pages = {51–71},
  publisher = {Elsevier B.V.},
  title = {{Evaluating Approaches to Resource Demand Estimation}},
  volume = 92,
  year = 2015
}

%0 Journal Article
%1 SpCaBrKo2015-PEVA-ResDemEstSurvey
%A Spinner, Simon
%A Casale, Giuliano
%A Brosig, Fabian
%A Kounev, Samuel
%D 2015
%I Elsevier B.V.
%J Performance Evaluation
%P 51--71
%T {Evaluating Approaches to Resource Demand Estimation}
%U http://www.sciencedirect.com/science/article/pii/S0166531615000711
%V 92
%X Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.

Performance-oriented {DevOps}: {A} Research Agenda. (SPEC-RG-2015-01), Brunnert, Andreas; van Hoorn, Andre; Willnecker, Felix; Danciu, Alexandru; Hasselbring, Wilhelm; Heger, Christoph; Herbst, Nikolas; Jamshidi, Pooyan; Jung, Reiner; von Kistowski, Joakim; Koziolek, Anne; Kross, Johannes; Spinner, Simon; V{ö}gele, Christian; Walter, J{ü}rgen; Wert, Alexander; (2015). SPEC Research Group --- DevOps Performance Working Group, Standard Performance Evaluation Corporation (SPEC).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]

DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.

@techreport{SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda,
  abstract = {DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.},
  author = {Brunnert, Andreas and van Hoorn, Andre and Willnecker, Felix and Danciu, Alexandru and Hasselbring, Wilhelm and Heger, Christoph and Herbst, Nikolas and Jamshidi, Pooyan and Jung, Reiner and von Kistowski, Joakim and Koziolek, Anne and Kross, Johannes and Spinner, Simon and V{\"o}gele, Christian and Walter, J{\"u}rgen and Wert, Alexander},
  institution = {SPEC Research Group — DevOps Performance Working Group, Standard Performance Evaluation Corporation (SPEC)},
  keywords = {SPEC},
  month = {08},
  number = {SPEC-RG-2015-01},
  title = {Performance-oriented {DevOps}: {A} Research Agenda},
  year = 2015
}

%0 Report
%1 SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda
%A Brunnert, Andreas
%A van Hoorn, Andre
%A Willnecker, Felix
%A Danciu, Alexandru
%A Hasselbring, Wilhelm
%A Heger, Christoph
%A Herbst, Nikolas
%A Jamshidi, Pooyan
%A Jung, Reiner
%A von Kistowski, Joakim
%A Koziolek, Anne
%A Kross, Johannes
%A Spinner, Simon
%A V{ö}gele, Christian
%A Walter, J{ü}rgen
%A Wert, Alexander
%D 2015
%N SPEC-RG-2015-01
%T Performance-oriented {DevOps}: {A} Research Agenda
%U https://research.spec.org/fileadmin/user_upload/documents/wg_devops/endorsed_publications/SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda.pdf
%X DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.

{Parallel Simulation of Queueing Petri Nets}. Walter, J{ü}rgen; Spinner, Simon; Kounev, Samuel; in EAI Endorsed Transactions on Industrial Networks and Intelligent Systems (2015). 16(8) ACM.

[ BibTeX ]
[ EndNote ]

@article{WaSpKo2016-EAI-parallelSimulation,
  author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel},
  journal = {EAI Endorsed Transactions on Industrial Networks and Intelligent Systems},
  keywords = {se2},
  month = {08},
  number = 8,
  publisher = {ACM},
  title = {{Parallel Simulation of Queueing Petri Nets}},
  volume = 16,
  year = 2015
}

%0 Journal Article
%1 WaSpKo2016-EAI-parallelSimulation
%A Walter, J{ü}rgen
%A Spinner, Simon
%A Kounev, Samuel
%D 2015
%I ACM
%J EAI Endorsed Transactions on Industrial Networks and Intelligent Systems
%N 8
%T {Parallel Simulation of Queueing Petri Nets}
%V 16

{Parallel Simulation of Queueing Petri Nets}. Walter, J{ü}rgen; Spinner, Simon; Kounev, Samuel; in Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015) (2015).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

@inproceedings{WaSpKo-SIMUTools-QPNParallelSimulation,
  abstract = {Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.},
  author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel},
  booktitle = {Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015)},
  keywords = {QPN},
  month = {08},
  title = {{Parallel Simulation of Queueing Petri Nets}},
  year = 2015
}

%0 Conference Paper
%1 WaSpKo-SIMUTools-QPNParallelSimulation
%A Walter, J{ü}rgen
%A Spinner, Simon
%A Kounev, Samuel
%B Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015)
%D 2015
%T {Parallel Simulation of Queueing Petri Nets}
%X Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

{Proactive Memory Scaling of Virtualized Applications}. Spinner, Simon; Zhu, Xiaoyun; Lu, Lei; Uysal, Mustafa; Griffith, Rean; Herbst, Nikolas; Kounev, Samuel; in VMware R&D Innovation Offsite (RADIO) (2015).

[ BibTeX ]
[ EndNote ]

@inproceedings{SpinnerZLUGHK-2015-RADIO-MemoryScaling,
  author = {Spinner, Simon and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Griffith, Rean and Herbst, Nikolas and Kounev, Samuel},
  booktitle = {VMware R&D Innovation Offsite (RADIO)},
  keywords = {descartes},
  month = {07},
  title = {{Proactive Memory Scaling of Virtualized Applications}},
  year = 2015
}

%0 Conference Paper
%1 SpinnerZLUGHK-2015-RADIO-MemoryScaling
%A Spinner, Simon
%A Zhu, Xiaoyun
%A Lu, Lei
%A Uysal, Mustafa
%A Griffith, Rean
%A Herbst, Nikolas
%A Kounev, Samuel
%B VMware R&D Innovation Offsite (RADIO)
%D 2015
%T {Proactive Memory Scaling of Virtualized Applications}

{Proactive Memory Scaling of Virtualized Applications}. Spinner, Simon; Herbst, Nikolas; Kounev, Samuel; Zhu, Xiaoyun; Lu, Lei; Uysal, Mustafa; Griffith, Rean; in Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015) (2015). 277–284. IEEE.

Acceptance Rate: 15%

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

{Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.}

@inproceedings{SpHeKoZhLuUyGr2015-CLOUD-ProactiveMemoryScaling,
  abstract = {{Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.}},
  author = {Spinner, Simon and Herbst, Nikolas and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Griffith, Rean},
  booktitle = {Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015)},
  keywords = {Self-adaptive-systems},
  month = {06},
  note = {Acceptance Rate: 15%},
  pages = {277–284},
  publisher = {IEEE},
  title = {{Proactive Memory Scaling of Virtualized Applications}},
  year = 2015
}

%0 Conference Paper
%1 SpHeKoZhLuUyGr2015-CLOUD-ProactiveMemoryScaling
%A Spinner, Simon
%A Herbst, Nikolas
%A Kounev, Samuel
%A Zhu, Xiaoyun
%A Lu, Lei
%A Uysal, Mustafa
%A Griffith, Rean
%B Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015)
%D 2015
%I IEEE
%P 277--284
%T {Proactive Memory Scaling of Virtualized Applications}
%X {Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.}

{Parallel Simulation of Queueing Petri Nets}. Walter, J{ü}rgen; Spinner, Simon; Kounev, Samuel; (2014, November).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{talk-Walter-SOSP14,
  author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel},
  howpublished = {Technical Talk at Symposion on Software Performance (SOSP) 2014},
  keywords = {QPN},
  month = 11,
  title = {{Parallel Simulation of Queueing Petri Nets}},
  year = 2014
}

%0 Generic
%1 talk-Walter-SOSP14
%A Walter, J{ü}rgen
%A Spinner, Simon
%A Kounev, Samuel
%D 2014
%T {Parallel Simulation of Queueing Petri Nets}

{LibReDE: A Library for Resource Demand Estimation}. Spinner, Simon; Walter, J{ü}rgen; (2014, November).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{talk-Spinner-SOSP14,
  author = {Spinner, Simon and Walter, J{\"u}rgen},
  howpublished = {Technical Talk at Symposion on Software Performance (SOSP) 2014},
  keywords = {Talks},
  month = 11,
  title = {{LibReDE: A Library for Resource Demand Estimation}},
  year = 2014
}

%0 Generic
%1 talk-Spinner-SOSP14
%A Spinner, Simon
%A Walter, J{ü}rgen
%D 2014
%T {LibReDE: A Library for Resource Demand Estimation}

{Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation}. Spinner, Simon; Kounev, Samuel; Zhu, Xiaoyun; Lu, Lei; Uysal, Mustafa; Holler, Anne; Griffith, Rean; in Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO) (2014). 157–166. IEEE.

Acceptance Rate (Full Papers): 26%

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.

@inproceedings{SpKoZhLuUyHoGr2014-SASO,
  abstract = {Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.},
  author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Holler, Anne and Griffith, Rean},
  booktitle = {Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO)},
  keywords = {Self-adaptive-systems},
  month = {09},
  note = {Acceptance Rate (Full Papers): 26%},
  pages = {157–166},
  publisher = {IEEE},
  title = {{Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation}},
  year = 2014
}

%0 Conference Paper
%1 SpKoZhLuUyHoGr2014-SASO
%A Spinner, Simon
%A Kounev, Samuel
%A Zhu, Xiaoyun
%A Lu, Lei
%A Uysal, Mustafa
%A Holler, Anne
%A Griffith, Rean
%B Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO)
%D 2014
%I IEEE
%P 157--166
%T {Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation}
%X Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.

{Model-Based Runtime Performance Management for Virtualized Applications}. Spinner, Simon; Kounev, Samuel; Zhu, Xiaoyun; Lu, Lei; Uysal, Mustafa; Holler, Anne; Griffith, Rean; Padala, Pradeep; in VMware R&D Innovation Offsite (RADIO) (2014).

[ BibTeX ]
[ EndNote ]

@inproceedings{SpinnerKZLUHGP-2014-RADIO-ModelBasedScaling,
  author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Holler, Anne and Griffith, Rean and Padala, Pradeep},
  booktitle = {VMware R&D Innovation Offsite (RADIO)},
  keywords = {se2},
  month = {07},
  title = {{Model-Based Runtime Performance Management for Virtualized Applications}},
  year = 2014
}

%0 Conference Paper
%1 SpinnerKZLUHGP-2014-RADIO-ModelBasedScaling
%A Spinner, Simon
%A Kounev, Samuel
%A Zhu, Xiaoyun
%A Lu, Lei
%A Uysal, Mustafa
%A Holler, Anne
%A Griffith, Rean
%A Padala, Pradeep
%B VMware R&D Innovation Offsite (RADIO)
%D 2014
%T {Model-Based Runtime Performance Management for Virtualized Applications}

{LibReDE - A library for Resource Demand Estimation}. Spinner, Simon; (2014, July).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{talk-Spinner-SPECdevops14,
  author = {Spinner, Simon},
  howpublished = {Technical Talk at SPEC DevOps Performance Working Group},
  keywords = {descartes},
  month = {07},
  title = {{LibReDE - A library for Resource Demand Estimation}},
  year = 2014
}

%0 Generic
%1 talk-Spinner-SPECdevops14
%A Spinner, Simon
%D 2014
%T {LibReDE - A library for Resource Demand Estimation}

{Resource Usage Control In Multi-Tenant Applications}. Krebs, Rouven; Spinner, Simon; Ahmed, Nadia; Kounev, Samuel; in Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014) (2014). 122–131. IEEE/ACM.

Acceptance Rate: 19\% (54/283)

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

{Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.}

@inproceedings{KrSpAhKo2014_CCGrid_ResourceIsolation,
  abstract = {{Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.}},
  author = {Krebs, Rouven and Spinner, Simon and Ahmed, Nadia and Kounev, Samuel},
  booktitle = {Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014)},
  keywords = {Self-adaptive-systems},
  month = {05},
  note = {Acceptance Rate: 19\% (54/283)},
  pages = {122-131},
  publisher = {IEEE/ACM},
  title = {{Resource Usage Control In Multi-Tenant Applications}},
  year = 2014
}

%0 Conference Paper
%1 KrSpAhKo2014_CCGrid_ResourceIsolation
%A Krebs, Rouven
%A Spinner, Simon
%A Ahmed, Nadia
%A Kounev, Samuel
%B Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014)
%D 2014
%I IEEE/ACM
%P 122-131
%T {Resource Usage Control In Multi-Tenant Applications}
%X {Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.}

{LibReDE: A Library for Resource Demand Estimation}. Spinner, Simon; Casale, Giuliano; Zhu, Xiaoyun; Kounev, Samuel; in Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 227–228. ACM Press, New York, NY, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

When creating a performance model, it is necessary to quantify the amount of resources consumed by an application serving individual requests. In distributed enterprise systems, these resource demands usually cannot be observed directly, their estimation is a major challenge. Different statistical approaches to resource demand estimation based on monitoring data have been proposed, e.g., using linear regression or Kalman filtering techniques. In this paper, we present LibReDE, a library of ready-to-use implementations of approaches to resource demand estimation that can be used for online and offline analysis. It is the first publicly available tool for this task and aims at supporting performance engineers during performance model construction. The library enables the quick comparison of the estimation accuracy of different approaches in a given context and thus helps to select an optimal one.

@inproceedings{SpCaZhKo2014-ICPEDemo-LibReDE,
  abstract = {When creating a performance model, it is necessary to quantify the amount of resources consumed by an application serving individual requests. In distributed enterprise systems, these resource demands usually cannot be observed directly, their estimation is a major challenge. Different statistical approaches to resource demand estimation based on monitoring data have been proposed, e.g., using linear regression or Kalman filtering techniques. In this paper, we present LibReDE, a library of ready-to-use implementations of approaches to resource demand estimation that can be used for online and offline analysis. It is the first publicly available tool for this task and aims at supporting performance engineers during performance model construction. The library enables the quick comparison of the estimation accuracy of different approaches in a given context and thus helps to select an optimal one.},
  address = {New York, NY, USA},
  author = {Spinner, Simon and Casale, Giuliano and Zhu, Xiaoyun and Kounev, Samuel},
  booktitle = {Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)},
  keywords = {Instrumentation_profiling_and_workload_characterization},
  month = {03},
  pages = {227–228},
  publisher = {ACM Press},
  title = {{LibReDE: A Library for Resource Demand Estimation}},
  year = 2014
}

%0 Conference Paper
%1 SpCaZhKo2014-ICPEDemo-LibReDE
%A Spinner, Simon
%A Casale, Giuliano
%A Zhu, Xiaoyun
%A Kounev, Samuel
%B Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)
%C New York, NY, USA
%D 2014
%I ACM Press
%P 227--228
%T {LibReDE: A Library for Resource Demand Estimation}
%U http://doi.acm.org/10.1145/2568088.2576093
%X When creating a performance model, it is necessary to quantify the amount of resources consumed by an application serving individual requests. In distributed enterprise systems, these resource demands usually cannot be observed directly, their estimation is a major challenge. Different statistical approaches to resource demand estimation based on monitoring data have been proposed, e.g., using linear regression or Kalman filtering techniques. In this paper, we present LibReDE, a library of ready-to-use implementations of approaches to resource demand estimation that can be used for online and offline analysis. It is the first publicly available tool for this task and aims at supporting performance engineers during performance model construction. The library enables the quick comparison of the estimation accuracy of different approaches in a given context and thus helps to select an optimal one.

{Towards Online Performance Model Extraction in Virtualized Environments}. Spinner, Simon; Kounev, Samuel; Zhu, Xiaoyun; Uysal, Mustafa; in Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013), N. Bencomo, R. France, S. G{ö}tz, B. Rumpe (eds.) (2013). 89–95. CEUR-WS.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.

@inproceedings{SpKoZhUy2013-mrt-TowardsModelExtraction,
  abstract = {Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.},
  author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Uysal, Mustafa},
  booktitle = {Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013)},
  editor = {Bencomo, Nelly and France, Robert and G{\"o}tz, Sebastian and Rumpe, Bernhard},
  keywords = {Virtualization},
  month = {09},
  pages = {89–95},
  publisher = {CEUR-WS},
  title = {{Towards Online Performance Model Extraction in Virtualized Environments}},
  year = 2013
}

%0 Conference Paper
%1 SpKoZhUy2013-mrt-TowardsModelExtraction
%A Spinner, Simon
%A Kounev, Samuel
%A Zhu, Xiaoyun
%A Uysal, Mustafa
%B Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013)
%D 2013
%E Bencomo, Nelly
%E France, Robert
%E G{ö}tz, Sebastian
%E Rumpe, Bernhard
%I CEUR-WS
%P 89--95
%T {Towards Online Performance Model Extraction in Virtualized Environments}
%U http://ceur-ws.org/Vol-1079/mrt13_submission_4.pdf
%X Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.

{Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0}. Spinner, Simon; Kounev, Samuel; Meier, Philipp; in Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012), S. Haddad, L. Pomello (eds.) (2012). (Vol. 7347) 388–397. Springer-Verlag, Berlin, Heidelberg.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.

@inproceedings{SpKoMe2012-PETRINETS-QPME,
  abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.},
  address = {Berlin, Heidelberg},
  author = {Spinner, Simon and Kounev, Samuel and Meier, Philipp},
  booktitle = {Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012)},
  editor = {Haddad, Serge and Pomello, Lucia},
  keywords = {QPN},
  month = {06},
  pages = {388–397},
  publisher = {Springer-Verlag},
  series = {Lecture Notes in Computer Science (LNCS)},
  title = {{Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0}},
  volume = 7347,
  year = 2012
}

%0 Conference Paper
%1 SpKoMe2012-PETRINETS-QPME
%A Spinner, Simon
%A Kounev, Samuel
%A Meier, Philipp
%B Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012)
%C Berlin, Heidelberg
%D 2012
%E Haddad, Serge
%E Pomello, Lucia
%I Springer-Verlag
%P 388--397
%T {Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0}
%U http://dx.doi.org/10.1007/978-3-642-31131-4_21
%V 7347
%X Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.

Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies (Tutorial Paper). Kounev, Samuel; Spinner, Simon; Meier, Philipp; in Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 2012), Boston, USA, 22-25 April 2012 (2012). 9–18. ACM, New York, NY, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and evaluating their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. This tutorial presents an introduction to queueing Petri nets first introducing the modeling formalism itself and then summarizing the results of several modeling case studies which demonstrate how queueing Petri nets can be used for performance modeling and analysis. As part of the tutorial, we present QPME (Queueing Petri net Modeling Environment), an open-source tool for stochastic modeling and analysis of systems using queueing Petri nets. Finally, we briefly present a model-to-model transformation automatically generating a queueing Petri net model from a higher-level software architecture model annotated with performance relevant information.

@inproceedings{KoSpMe2012-icpe-QPME_Tutorial,
  abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and evaluating their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. This tutorial presents an introduction to queueing Petri nets first introducing the modeling formalism itself and then summarizing the results of several modeling case studies which demonstrate how queueing Petri nets can be used for performance modeling and analysis. As part of the tutorial, we present QPME (Queueing Petri net Modeling Environment), an open-source tool for stochastic modeling and analysis of systems using queueing Petri nets. Finally, we briefly present a model-to-model transformation automatically generating a queueing Petri net model from a higher-level software architecture model annotated with performance relevant information.},
  address = {New York, NY, USA},
  author = {Kounev, Samuel and Spinner, Simon and Meier, Philipp},
  booktitle = {Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 2012), Boston, USA, 22-25 April 2012},
  keywords = {QPN},
  month = {04},
  pages = {9–18},
  publisher = {ACM},
  title = {Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies (Tutorial Paper)},
  year = 2012
}

%0 Conference Paper
%1 KoSpMe2012-icpe-QPME_Tutorial
%A Kounev, Samuel
%A Spinner, Simon
%A Meier, Philipp
%B Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 2012), Boston, USA, 22-25 April 2012
%C New York, NY, USA
%D 2012
%I ACM
%P 9--18
%T Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies (Tutorial Paper)
%X Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and evaluating their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. This tutorial presents an introduction to queueing Petri nets first introducing the modeling formalism itself and then summarizing the results of several modeling case studies which demonstrate how queueing Petri nets can be used for performance modeling and analysis. As part of the tutorial, we present QPME (Queueing Petri net Modeling Environment), an open-source tool for stochastic modeling and analysis of systems using queueing Petri nets. Finally, we briefly present a model-to-model transformation automatically generating a queueing Petri net model from a higher-level software architecture model annotated with performance relevant information.

Model-based Techniques for Performance Engineering of Business Information Systems. Kounev, Samuel; Huber, Nikolaus; Spinner, Simon; Brosig, Fabian; in Business Modeling and Software Design, B. Shishkov (ed.) (2012). (Vol. 0109) 19–37. Springer-Verlag, Berlin, Heidelberg.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{With the increasing adoption of virtualization and the transition towards Cloud Computing platforms, modern business information systems are becoming increasingly complex and dynamic. This raises the challenge of guaranteeing system performance and scalability while at the same time ensuring efficient resource usage. In this paper, we present a historical perspective on the evolution of model-based performance engineering techniques for business information systems focusing on the major developments over the past several decades that have shaped the field. We survey the state-of-the-art on performance modeling and management approaches discussing the ongoing efforts in the community to increasingly bridge the gap between high-level business services and low level performance models. Finally, we wrap up with an outlook on the emergence of self-aware systems engineering as a new research area at the intersection of several computer science disciplines.}

@incollection{KoHuSpBr2012-business-inf-sys,
  abstract = {{With the increasing adoption of virtualization and the transition towards Cloud Computing platforms, modern business information systems are becoming increasingly complex and dynamic. This raises the challenge of guaranteeing system performance and scalability while at the same time ensuring efficient resource usage. In this paper, we present a historical perspective on the evolution of model-based performance engineering techniques for business information systems focusing on the major developments over the past several decades that have shaped the field. We survey the state-of-the-art on performance modeling and management approaches discussing the ongoing efforts in the community to increasingly bridge the gap between high-level business services and low level performance models. Finally, we wrap up with an outlook on the emergence of self-aware systems engineering as a new research area at the intersection of several computer science disciplines.}},
  address = {Berlin, Heidelberg},
  author = {Kounev, Samuel and Huber, Nikolaus and Spinner, Simon and Brosig, Fabian},
  booktitle = {Business Modeling and Software Design},
  editor = {Shishkov, Boris},
  keywords = {t_bookchapter},
  pages = {19–37},
  publisher = {Springer-Verlag},
  series = {Lecture Notes in Business Information Processing (LNBIP)},
  title = {Model-based Techniques for Performance Engineering of Business Information Systems},
  volume = {0109},
  year = 2012
}

%0 Book Section
%1 KoHuSpBr2012-business-inf-sys
%A Kounev, Samuel
%A Huber, Nikolaus
%A Spinner, Simon
%A Brosig, Fabian
%B Business Modeling and Software Design
%C Berlin, Heidelberg
%D 2012
%E Shishkov, Boris
%I Springer-Verlag
%P 19--37
%T Model-based Techniques for Performance Engineering of Business Information Systems
%U http://dx.doi.org/10.1007/978-3-642-29788-5_2
%V 0109
%X {With the increasing adoption of virtualization and the transition towards Cloud Computing platforms, modern business information systems are becoming increasingly complex and dynamic. This raises the challenge of guaranteeing system performance and scalability while at the same time ensuring efficient resource usage. In this paper, we present a historical perspective on the evolution of model-based performance engineering techniques for business information systems focusing on the major developments over the past several decades that have shaped the field. We survey the state-of-the-art on performance modeling and management approaches discussing the ongoing efforts in the community to increasingly bridge the gap between high-level business services and low level performance models. Finally, we wrap up with an outlook on the emergence of self-aware systems engineering as a new research area at the intersection of several computer science disciplines.}

{Evaluating Approaches to Resource Demand Estimation}. Spinner, Simon; (2011, July). Karlsruhe Institute of Technology (KIT), Am Fasanengarten 5, 76131 Karlsruhe, Germany.

Best Graduate Award from the Faculty of Informatics

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Proactively managing the performance and resource efficiency of running software systems requires techniques to predict system performance and resource consumption. Typically, performance predictions are based on performance models that capture the performance-relevant aspects of the considered software system. Building performance models involves the estimation of resource demands, i.e., estimating the time a unit of work spends obtaining service from a resource. A number of approaches to estimating the resource demands of a system already exist, e.g., based on regression analysis or stochastic filtering. These approaches differ in their accuracy, their robustness and their applicability. For instance, there are notable differences in the amount and type of measurement data that is required as input. However, to the best of our knowledge, a comprehensive evaluation and comparison of these approaches in a representative context does not exist. In this thesis, we give an overview of the state-of-the-art in resource demand estimation and develop a classification scheme for approaches to resource demand estimation. We implement a sub-set of these estimation approaches and evaluate them in a representative environment. We analyze the influence of various factors of the environment on the estimation accuracy, considering the impact of current technologies, such as multi-core processors and virtualization. Our work improves the comparability of existing estimation approaches and facilitates the selection of an approach in a given application scenario. Additionally, it shows possible directions for future research in the field of resource demand estimation.

@mastersthesis{Spinner2011a,
  abstract = {Proactively managing the performance and resource efficiency of running software systems requires techniques to predict system performance and resource consumption. Typically, performance predictions are based on performance models that capture the performance-relevant aspects of the considered software system. Building performance models involves the estimation of resource demands, i.e., estimating the time a unit of work spends obtaining service from a resource. A number of approaches to estimating the resource demands of a system already exist, e.g., based on regression analysis or stochastic filtering. These approaches differ in their accuracy, their robustness and their applicability. For instance, there are notable differences in the amount and type of measurement data that is required as input. However, to the best of our knowledge, a comprehensive evaluation and comparison of these approaches in a representative context does not exist. In this thesis, we give an overview of the state-of-the-art in resource demand estimation and develop a classification scheme for approaches to resource demand estimation. We implement a sub-set of these estimation approaches and evaluate them in a representative environment. We analyze the influence of various factors of the environment on the estimation accuracy, considering the impact of current technologies, such as multi-core processors and virtualization. Our work improves the comparability of existing estimation approaches and facilitates the selection of an approach in a given application scenario. Additionally, it shows possible directions for future research in the field of resource demand estimation.},
  address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany},
  author = {Spinner, Simon},
  keywords = {QPME_Bibliography},
  month = {07},
  note = {Best Graduate Award from the Faculty of Informatics},
  school = {Karlsruhe Institute of Technology (KIT)},
  title = {{Evaluating Approaches to Resource Demand Estimation}},
  year = 2011
}

%0 Thesis
%1 Spinner2011a
%A Spinner, Simon
%C Am Fasanengarten 5, 76131 Karlsruhe, Germany
%D 2011
%T {Evaluating Approaches to Resource Demand Estimation}
%X Proactively managing the performance and resource efficiency of running software systems requires techniques to predict system performance and resource consumption. Typically, performance predictions are based on performance models that capture the performance-relevant aspects of the considered software system. Building performance models involves the estimation of resource demands, i.e., estimating the time a unit of work spends obtaining service from a resource. A number of approaches to estimating the resource demands of a system already exist, e.g., based on regression analysis or stochastic filtering. These approaches differ in their accuracy, their robustness and their applicability. For instance, there are notable differences in the amount and type of measurement data that is required as input. However, to the best of our knowledge, a comprehensive evaluation and comparison of these approaches in a representative context does not exist. In this thesis, we give an overview of the state-of-the-art in resource demand estimation and develop a classification scheme for approaches to resource demand estimation. We implement a sub-set of these estimation approaches and evaluate them in a representative environment. We analyze the influence of various factors of the environment on the estimation accuracy, considering the impact of current technologies, such as multi-core processors and virtualization. Our work improves the comparability of existing estimation approaches and facilitates the selection of an approach in a given application scenario. Additionally, it shows possible directions for future research in the field of resource demand estimation.

QPME 2.0 User’s Guide Kounev, Samuel; Spinner, Simon; Walter, J{ü}rgen; (2011). Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131 Karlsruhe, Germany.

[ BibTeX ]
[ EndNote ]

@manual{KoSp2011-QPME20-UserGuide,
  address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany},
  author = {Kounev, Samuel and Spinner, Simon and Walter, J{\"u}rgen},
  keywords = {QPN},
  month = {05},
  organization = {Karlsruhe Institute of Technology},
  title = {QPME 2.0 User's Guide},
  year = 2011
}

%0 Generic
%1 KoSp2011-QPME20-UserGuide
%A Kounev, Samuel
%A Spinner, Simon
%A Walter, J{ü}rgen
%C Am Fasanengarten 5, 76131 Karlsruhe, Germany
%D 2011
%T QPME 2.0 User's Guide

{QPME 2.0 - A Tool for Stochastic Modeling and Analysis Using Queueing Petri Nets}. Kounev, Samuel; Spinner, Simon; Meier, Philipp; in From Active Data Management to Event-Based Systems and More, K. Sachs, I. Petrov, P. Guerrero (eds.) (2010). (Vol. 6462) 293–311. Springer-Verlag, Berlin, Heidelberg.

[ BibTeX ]
[ EndNote ]

@incollection{KoSpMe2010-Festschrift-QPME2,
  address = {Berlin, Heidelberg},
  author = {Kounev, Samuel and Spinner, Simon and Meier, Philipp},
  booktitle = {From Active Data Management to Event-Based Systems and More},
  editor = {Sachs, Kai and Petrov, Ilia and Guerrero, Pablo},
  keywords = {QPN},
  pages = {293–311},
  publisher = {Springer-Verlag},
  series = {Lecture Notes in Computer Science},
  title = {{QPME 2.0 - A Tool for Stochastic Modeling and Analysis Using Queueing Petri Nets}},
  volume = 6462,
  year = 2010
}

%0 Book Section
%1 KoSpMe2010-Festschrift-QPME2
%A Kounev, Samuel
%A Spinner, Simon
%A Meier, Philipp
%B From Active Data Management to Event-Based Systems and More
%C Berlin, Heidelberg
%D 2010
%E Sachs, Kai
%E Petrov, Ilia
%E Guerrero, Pablo
%I Springer-Verlag
%P 293--311
%T {QPME 2.0 - A Tool for Stochastic Modeling and Analysis Using Queueing Petri Nets}
%V 6462

Publications of Simon Spinner

Data privacy protection

Data privacy protection

Picture credits