Publications

2020 [ nach oben ]

Systems Benchmarking Kounev, Samuel; Lange, Klaus-Dieter; von Kistowski, Jóakim; (2020). (1st ed.) Springer International Publishing.
[ BibTeX ] [ EndNote ]
 
@book{KoLaKi-2020-SystemsBenchmarking, author = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, Jóakim}, edition = 1, keywords = {myown select descartes t_book}, publisher = {Springer International Publishing}, title = {{Systems Benchmarking}}, year = 2020 }
%0 Book %1 KoLaKi-2020-SystemsBenchmarking %A Kounev, Samuel %A Lange, Klaus-Dieter %A von Kistowski, Jóakim %D 2020 %I Springer International Publishing %T Systems Benchmarking %7 1

2017 [ nach oben ]

Self-Aware Computing Systems Kounev, Samuel; Kephart, Jeffrey O.; Milenkoski, Aleksandar; Zhu, Xiaoyun; (2017). Springer Verlag, Berlin Heidelberg, Germany.
[ BibTeX ] [ EndNote ] [ URL ]
 
@book{KoKeMiZh2017-self, address = {{Berlin Heidelberg, Germany}}, author = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Statistical_estimation_and_machine_learning myown Metrics_and_benchmarking_methodologies Resource_management Design_of_software_and_systems descartes t_book se2_book Survey Online_monitoring_and_forecasting Self-aware-computing Application_quality_of_service_management Application-aware Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Self-Aware Computing Systems}}, year = 2017 }
%0 Book %1 KoKeMiZh2017-self %A Kounev, Samuel %A Kephart, Jeffrey O. %A Milenkoski, Aleksandar %A Zhu, Xiaoyun %C Berlin Heidelberg, Germany %D 2017 %I Springer Verlag %T Self-Aware Computing Systems %U http://www.springer.com/de/book/9783319474724

2015 [ nach oben ]

Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8--10, 2015 Kounev, Samuel; (P. Lalanda; A. Diaconescu; L. Cherkasova; S. Kounev, eds.) (2015). IEEE Computer Society Conference Publishing Services (CPS), Los Alamitos, California.
[ BibTeX ] [ EndNote ]
 
@proceedings{proceedings-icac-2015, address = {Los Alamitos, California}, author = {Kounev, Samuel}, editor = {Lalanda, Philippe and Diaconescu, Ada and Cherkasova, Lucy and Kounev, Samuel}, keywords = {myown descartes t_proceedings se2}, month = {July}, publisher = {IEEE Computer Society Conference Publishing Services (CPS)}, title = {Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8–10, 2015}, year = 2015 }
%0 Conference Proceedings %1 proceedings-icac-2015 %A Kounev, Samuel %C Los Alamitos, California %D 2015 %E Lalanda, Philippe %E Diaconescu, Ada %E Cherkasova, Lucy %E Kounev, Samuel %I IEEE Computer Society Conference Publishing Services (CPS) %T Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8--10, 2015

Model-driven Algorithms and Architectures for Self-Aware Computing Systems Kounev, Samuel; in Dagstuhl Reports, (S. Kounev; X. Zhu; J. O. Kephart; M. Kwiatkowska; A. M. [Assistant Editor], eds.) (2015). Dagstuhl, Germany.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ]
 
This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.
@proceedings{KoZhKeKw2015, abstract = {{This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.}}, address = {Dagstuhl, Germany}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Zhu, Xiaoyun and Kephart, Jeffrey O. and Kwiatkowska, Marta and Editor], Aleksandar Milenkoski [Assistant}, keywords = {Statistical_estimation_and_machine_learning myown Metrics_and_benchmarking_methodologies Resource_management Design_of_software_and_systems Analytical_and_simulation-based_analysis descartes Formal_architecture_modeling Automated_model_learning se2_book Survey Online_monitoring_and_forecasting Self-aware-computing Application-aware t_proceedings Self-adaptive-systems}, month = {January}, series = {Dagstuhl Reports}, title = {{Model-driven Algorithms and Architectures for Self-Aware Computing Systems}}, year = 2015 }
%0 Conference Proceedings %1 KoZhKeKw2015 %A Kounev, Samuel %B Dagstuhl Reports %C Dagstuhl, Germany %D 2015 %E Kounev, Samuel %E Zhu, Xiaoyun %E Kephart, Jeffrey O. %E Kwiatkowska, Marta %E Editor], Aleksandar Milenkoski [Assistant %T Model-driven Algorithms and Architectures for Self-Aware Computing Systems %U http://drops.dagstuhl.de/opus/volltexte/2015/5038/ %X This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.

2013 [ nach oben ]

Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) Kounev, Samuel; (S. Kounev; S. Zschaler; K. Sachs, eds.) (2013). ACM.
[ BibTeX ] [ EndNote ]
 
@proceedings{KoZsSa2013-HOTTOPICS-Proceedings, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Zschaler, Steffen and Sachs, Kai}, keywords = {myown SOA descartes Cloud t_proceedings}, month = {April}, publisher = {ACM}, title = {Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)}, year = 2013 }
%0 Conference Proceedings %1 KoZsSa2013-HOTTOPICS-Proceedings %A Kounev, Samuel %D 2013 %E Kounev, Samuel %E Zschaler, Steffen %E Sachs, Kai %I ACM %T Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)

2011 [ nach oben ]

ICPE’11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14--16, 2011 Kounev, Samuel; (S. Kounev; V. Cortellessa; R. Mirandola; D. J. Lilja, eds.) (2011). ACM, New York, NY, USA.
[ BibTeX ] [ EndNote ]
 
@proceedings{DBLP-conf-wosp-2011, address = {New York, NY, USA}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Cortellessa, Vittorio and Mirandola, Raffaela and Lilja, David J.}, keywords = {myown Metrics_and_benchmarking_methodologies Resource_management Performance Analytical_and_simulation-based_analysis descartes Prediction Formal_architecture_modeling Automated_model_learning Meta-models Instrumentation_profiling_and_workload_characterization Optimization Online_monitoring_and_forecasting Simulation Application_quality_of_service_management t_proceedings SPEC Power-energy_efficient_computing Power}, month = {March}, publisher = {ACM}, title = {ICPE'11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14–16, 2011}, year = 2011 }
%0 Conference Proceedings %1 DBLP-conf-wosp-2011 %A Kounev, Samuel %C New York, NY, USA %D 2011 %E Kounev, Samuel %E Cortellessa, Vittorio %E Mirandola, Raffaela %E Lilja, David J. %I ACM %T ICPE'11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14--16, 2011

2008 [ nach oben ]

Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28 Kounev, Samuel; in Lecture Notes in Computer Science (LNCS), (S. Kounev; I. Gorton; K. Sachs, eds.) (2008). (Vol. 5119) Springer, Heidelberg, Germany.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ]
 
This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.
@proceedings{kounev2008c, abstract = {This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.}, address = {Heidelberg, Germany}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Gorton, Ian and Sachs, Kai}, keywords = {myown Metrics_and_benchmarking_methodologies Resource_management Performance Analytical_and_simulation-based_analysis descartes Prediction Formal_architecture_modeling Automated_model_learning Meta-models Instrumentation_profiling_and_workload_characterization Optimization Online_monitoring_and_forecasting Simulation Application_quality_of_service_management t_proceedings SPEC Power-energy_efficient_computing Power}, month = {June}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28}}, volume = 5119, year = 2008 }
%0 Conference Proceedings %1 kounev2008c %A Kounev, Samuel %B Lecture Notes in Computer Science (LNCS) %C Heidelberg, Germany %D 2008 %E Kounev, Samuel %E Gorton, Ian %E Sachs, Kai %I Springer %T Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28 %U http://www.springer.com/computer/programming/book/978-3-540-69813-5 %V 5119 %X This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.

2005 [ nach oben ]

Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction Kounev, Samuel; (2005). Shaker Verlag, Ph.D. Thesis, Technische Universität Darmstadt, Germany.
Distinguished Dissertation Award
[ BibTeX ] [ EndNote ] [ Download ]
 
@book{Ko2005-TUD-PhD_Thesis, author = {Kounev, Samuel}, keywords = {myown Metrics_and_benchmarking_methodologies Resource_management Performance Analytical_and_simulation-based_analysis descartes Prediction Formal_architecture_modeling t_book Instrumentation_profiling_and_workload_characterization QPN QPME se2_book Survey Simulation Award Application_quality_of_service_management SPEC QPME_Bibliography}, month = {December}, note = {<b>Distinguished Dissertation Award</b>}, publisher = {Shaker Verlag, Ph.D. Thesis, Technische Universit{ä}t Darmstadt, Germany}, title = {Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction}, year = 2005 }
%0 Book %1 Ko2005-TUD-PhD_Thesis %A Kounev, Samuel %D 2005 %I Shaker Verlag, Ph.D. Thesis, Technische Universität Darmstadt, Germany %T Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction

2022 [ nach oben ]

A case study on the stability of performance tests for serverless applications. Eismann, Simon; Costa, Diego; Liao, Lizhi; Bezemer, Cor-Paul; Shang, Weiyi; van Hoorn, André; Kounev, Samuel; in Journal of Systems and Software (JSS) (2022). 189
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.
@article{eismann2022study, abstract = {Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.}, author = {Eismann, Simon and Costa, Diego and Liao, Lizhi and Bezemer, Cor-Paul and Shang, Weiyi and van Hoorn, André and Kounev, Samuel}, journal = {Journal of Systems and Software (JSS)}, keywords = {descartes t_journalmagazine myown}, title = {A case study on the stability of performance tests for serverless applications}, volume = 189, year = 2022 }
%0 Journal Article %1 eismann2022study %A Eismann, Simon %A Costa, Diego %A Liao, Lizhi %A Bezemer, Cor-Paul %A Shang, Weiyi %A van Hoorn, André %A Kounev, Samuel %D 2022 %J Journal of Systems and Software (JSS) %T A case study on the stability of performance tests for serverless applications %U https://www.sciencedirect.com/science/article/abs/pii/S0164121222000498 %V 189 %X Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.

2021 [ nach oben ]

Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms. Lesch, Veronika; König, Maximilian; Kounev, Samuel; Stein, Anthony; Krupitzer, Christian; in Applied Intelligence (2021).
[ BibTeX ] [ EndNote ]
 
@article{lesch2021tackling, author = {Lesch, Veronika and König, Maximilian and Kounev, Samuel and Stein, Anthony and Krupitzer, Christian}, journal = {Applied Intelligence}, keywords = {cyber-physical_systems descartes intelligent_transportation_systems optimization t_journalmagazine myown}, title = {Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms}, year = 2021 }
%0 Journal Article %1 lesch2021tackling %A Lesch, Veronika %A König, Maximilian %A Kounev, Samuel %A Stein, Anthony %A Krupitzer, Christian %D 2021 %J Applied Intelligence %T Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms

An Overview on Approaches for Coordination of Platoons. Lesch, Veronika; Breitbach, Martin; Segata, Michele; Becker, Christian; Kounev, Samuel; Krupitzer, Christian; in IEEE Transactions on Intelligent Transportation Systems (2021).
Early Access on IEEE Xplore
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{lesch2021overview, author = {Lesch, Veronika and Breitbach, Martin and Segata, Michele and Becker, Christian and Kounev, Samuel and Krupitzer, Christian}, journal = {IEEE Transactions on Intelligent Transportation Systems}, keywords = {cyber-physical_systems descartes intelligent_transportation_systems internet_of_things platooning t_journalmagazine myown}, note = {Early Access on IEEE Xplore}, title = {An Overview on Approaches for Coordination of Platoons}, year = 2021 }
%0 Journal Article %1 lesch2021overview %A Lesch, Veronika %A Breitbach, Martin %A Segata, Michele %A Becker, Christian %A Kounev, Samuel %A Krupitzer, Christian %D 2021 %J IEEE Transactions on Intelligent Transportation Systems %T An Overview on Approaches for Coordination of Platoons

A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools. Züfle, Marwin; Moog, Felix; Lesch, Veronika; Krupitzer, Christian; Kounev, Samuel; in ISA Transactions: The Journal of Automation (2021).
(in press)
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ]
 
Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.
@article{zuefle2021anomalydetection, abstract = {Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.}, author = {Züfle, Marwin and Moog, Felix and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, journal = {ISA Transactions: The Journal of Automation}, keywords = {Cyber-Physical_Systems Industry_4_0 descartes prediction statistical_estimation_and_machine_learning t_interdisciplinary t_journalmagazine myown}, note = {(in press)}, title = {A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools}, year = 2021 }
%0 Journal Article %1 zuefle2021anomalydetection %A Züfle, Marwin %A Moog, Felix %A Lesch, Veronika %A Krupitzer, Christian %A Kounev, Samuel %D 2021 %J ISA Transactions: The Journal of Automation %T A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools %U https://www.sciencedirect.com/science/article/pii/S0019057821003773 %X Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.

SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation. Grohmann, Johannes; Eismann, Simon; Bauer, André; Spinner, Simon; Blum, Johannes; Herbst, Nikolas; Kounev, Samuel; in ACM Transactions on Autonomous and Adaptive Systems (2021). 15(2) Association for Computing Machinery, New York, NY, USA.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.
@article{GrEiBaSpBlHeKo2021-TaaS-SARDE, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andr{é} and Spinner, Simon and Blum, Johannes and Herbst, Nikolas and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {sys:relevantfor:se-group Self-adaptive algorithms demand descartes estimation learning machine optimization performance prediction resource self-tuning statistical_estimation_and_machine_learning systems t_journalmagazine myown}, month = {jun}, number = 2, publisher = {Association for Computing Machinery}, title = {SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation}, volume = 15, year = 2021 }
%0 Journal Article %1 GrEiBaSpBlHeKo2021-TaaS-SARDE %A Grohmann, Johannes %A Eismann, Simon %A Bauer, André %A Spinner, Simon %A Blum, Johannes %A Herbst, Nikolas %A Kounev, Samuel %C New York, NY, USA %D 2021 %I Association for Computing Machinery %J ACM Transactions on Autonomous and Adaptive Systems %N 2 %T SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation %U https://doi.org/10.1145/3463369 %V 15 %X Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot. Spiess, Florian; Reinhart, Lucas; Strobel, Norbert; Kaiser, Dennis; Kounev, Samuel; Kaupp, Tobias; in Journal of Image and Graphics, (J. of Image; Graphics, eds.) (2021). 9(4) 135–139.
[ BibTeX ] [ EndNote ]
 
@article{spiess2021people, author = {Spiess, Florian and Reinhart, Lucas and Strobel, Norbert and Kaiser, Dennis and Kounev, Samuel and Kaupp, Tobias}, editor = {of Image, Journal and Graphics}, journal = {Journal of Image and Graphics}, keywords = {sys:relevantfor:se-group descartes t_interdisciplinary t_journalmagazine myown}, month = {December 2021}, number = 4, pages = {135-139}, title = {People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot}, volume = 9, year = 2021 }
%0 Journal Article %1 spiess2021people %A Spiess, Florian %A Reinhart, Lucas %A Strobel, Norbert %A Kaiser, Dennis %A Kounev, Samuel %A Kaupp, Tobias %D 2021 %E of Image, Journal %E Graphics, %J Journal of Image and Graphics %N 4 %P 135-139 %T People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot %V 9

A Comparison of Mechanisms for Compensating Negative Impacts of System Integration. Lesch, Veronika; Krupitzer, Christian; Stubenrauch, Kevin; Keil, Nico; Becker, Christian; Kounev, Samuel; Segata, Michele; in Future Generation Computer Systems (2021). 116 117–131.
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{lesch2020comparison, author = {Lesch, Veronika and Krupitzer, Christian and Stubenrauch, Kevin and Keil, Nico and Becker, Christian and Kounev, Samuel and Segata, Michele}, journal = {Future Generation Computer Systems}, keywords = {compensation cyber-physical_systems descartes intelligent_transportation_systems internet_of_things platooning system_integration t_journalmagazine myown}, month = {March}, pages = {117-131}, title = {A Comparison of Mechanisms for Compensating Negative Impacts of System Integration}, volume = 116, year = 2021 }
%0 Journal Article %1 lesch2020comparison %A Lesch, Veronika %A Krupitzer, Christian %A Stubenrauch, Kevin %A Keil, Nico %A Becker, Christian %A Kounev, Samuel %A Segata, Michele %D 2021 %J Future Generation Computer Systems %P 117-131 %T A Comparison of Mechanisms for Compensating Negative Impacts of System Integration %V 116

2020 [ nach oben ]

Towards Self-Aware Multirotor Formations. Kaiser, Dennis; Lesch, Veronika; Rothe, Julian; Strohmeier, Michael; Spiess, Florian; Krupitzer, Christian; Montenegro, Sergio; Kounev, Samuel; in Computers (2020). 9(7) MDPI.
Special Issue on Self-Aware Computing
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{Kaiser2020Towards, author = {Kaiser, Dennis and Lesch, Veronika and Rothe, Julian and Strohmeier, Michael and Spiess, Florian and Krupitzer, Christian and Montenegro, Sergio and Kounev, Samuel}, journal = {Computers}, keywords = {Analytical_and_simulation-based_analysis Cyber-Physical_Systems Design_of_software_and_systems Intelligent_Transportation_Systems Internet_of_Things Self-aware-computing Simulation descartes t_interdisciplinary t_journalmagazine myown}, month = {Februar}, note = {Special Issue on Self-Aware Computing}, number = 7, publisher = {MDPI}, title = {{Towards Self-Aware Multirotor Formations}}, volume = 9, year = 2020 }
%0 Journal Article %1 Kaiser2020Towards %A Kaiser, Dennis %A Lesch, Veronika %A Rothe, Julian %A Strohmeier, Michael %A Spiess, Florian %A Krupitzer, Christian %A Montenegro, Sergio %A Kounev, Samuel %D 2020 %I MDPI %J Computers %N 7 %T Towards Self-Aware Multirotor Formations %V 9

A Taxonomy of Techniques for SLO Failure Prediction in Software Systems. Grohmann, Johannes; Herbst, Nikolas; Chalbani, Avi; Arian, Yair; Peretz, Noam; Kounev, Samuel; in Computers (2020). 9(1) 10. Multidisciplinary Digital Publishing Institute (MDPI).
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ]
 
Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.
@article{grohmann2020-MDPI-taxonomy, abstract = {Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.}, author = {Grohmann, Johannes and Herbst, Nikolas and Chalbani, Avi and Arian, Yair and Peretz, Noam and Kounev, Samuel}, journal = {Computers}, keywords = {automated_model_learning descartes performance se2 t_journalmagazine myown}, number = 1, pages = 10, publisher = {Multidisciplinary Digital Publishing Institute (MDPI)}, title = {A Taxonomy of Techniques for SLO Failure Prediction in Software Systems}, volume = 9, year = 2020 }
%0 Journal Article %1 grohmann2020-MDPI-taxonomy %A Grohmann, Johannes %A Herbst, Nikolas %A Chalbani, Avi %A Arian, Yair %A Peretz, Noam %A Kounev, Samuel %D 2020 %I Multidisciplinary Digital Publishing Institute (MDPI) %J Computers %N 1 %P 10 %T A Taxonomy of Techniques for SLO Failure Prediction in Software Systems %U https://doi.org/10.3390/computers9010010 %V 9 %X Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.

Time Series Forecasting for Self-Aware Systems. Bauer, André; Züfle, Marwin; Herbst, Nikolas; Zehe, Albin; Hotho, Andreas; Kounev, Samuel; in Proceedings of the IEEE (2020). 108(7) 1068–1093.
[ BibTeX ] [ EndNote ]
 
@article{BaZuHeZeHoKo-PIEEE-Forecasting, author = {Bauer, Andr{é} and Z{ü}fle, Marwin and Herbst, Nikolas and Zehe, Albin and Hotho, Andreas and Kounev, Samuel}, journal = {Proceedings of the IEEE}, keywords = {Elasticity Metrics_and_benchmarking_methodologies Online_monitoring_and_forecasting Prediction Self-aware-computing Telescope descartes t_journalmagazine myown}, month = 7, number = 7, pages = {1068 - 1093}, title = {{Time Series Forecasting for Self-Aware Systems}}, volume = 108, year = 2020 }
%0 Journal Article %1 BaZuHeZeHoKo-PIEEE-Forecasting %A Bauer, André %A Züfle, Marwin %A Herbst, Nikolas %A Zehe, Albin %A Hotho, Andreas %A Kounev, Samuel %D 2020 %J Proceedings of the IEEE %N 7 %P 1068 - 1093 %T Time Series Forecasting for Self-Aware Systems %V 108

The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study. Zinner, Christoph; Matzka, Manuel; Leppich, Robert; Kounev, Samuel; Holmberg, HC; Sperlich, Billy; in Frontiers in Sports and Active Living (2020). 2 127. Frontiers.
[ BibTeX ] [ EndNote ]
 
@article{zinner2020impact, author = {Zinner, Christoph and Matzka, Manuel and Leppich, Robert and Kounev, Samuel and Holmberg, HC and Sperlich, Billy}, journal = {Frontiers in Sports and Active Living}, keywords = {covid19 descartes health myown t_interdisciplinary t_journalmagazine}, pages = 127, publisher = {Frontiers}, title = {The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study}, volume = 2, year = 2020 }
%0 Journal Article %1 zinner2020impact %A Zinner, Christoph %A Matzka, Manuel %A Leppich, Robert %A Kounev, Samuel %A Holmberg, HC %A Sperlich, Billy %D 2020 %I Frontiers %J Frontiers in Sports and Active Living %P 127 %T The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study %V 2

Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data. Spiess, Florian; Friesslich, Jonas; Tobias, Kaupp; Kounev, Samuel; Strobel, Norbert; in International Journal of Mechanical Engineering and Robotics Research (IJMERR), (P. R. (Chunhui) Yang, ed.) (2020). 9(12) 1532–1540.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ]
 
This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.
@article{yang2020survey, abstract = {This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.}, author = {Spiess, Florian and Friesslich, Jonas and Tobias, Kaupp and Kounev, Samuel and Strobel, Norbert}, editor = {Yang, Prof Richard (Chunhui)}, journal = {International Journal of Mechanical Engineering and Robotics Research (IJMERR)}, keywords = {comparison descartes rgb robotics survey t_interdisciplinary t_journalmagazine myown}, month = {dec}, number = 12, pages = {1532-1540}, title = {Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data}, volume = 9, year = 2020 }
%0 Journal Article %1 yang2020survey %A Spiess, Florian %A Friesslich, Jonas %A Tobias, Kaupp %A Kounev, Samuel %A Strobel, Norbert %D 2020 %E Yang, Prof Richard (Chunhui) %J International Journal of Mechanical Engineering and Robotics Research (IJMERR) %N 12 %P 1532-1540 %T Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data %U http://www.ijmerr.com/index.php?m=content&c=index&a=show&catid=185&id=1529 %V 9 %X This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.

2019 [ nach oben ]

Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field. Bauer, André; Herbst, Nikolas; Spinner, Simon; Ali-Eldin, Ahmed; Kounev, Samuel; in IEEE Transactions on Parallel and Distributed Systems (2019). 30(4) 800–813. IEEE.
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{BaHeSpAEKo2019-TPDS-ChameleonAutoscaler, author = {Bauer, Andr{é} and Herbst, Nikolas and Spinner, Simon and Ali-Eldin, Ahmed and Kounev, Samuel}, journal = {IEEE Transactions on Parallel and Distributed Systems}, keywords = {Application-aware Application_quality_of_service_management BUNGEE Chameleon Cloud Elasticity LIMBO Metrics_and_benchmarking_methodologies Online_monitoring_and_forecasting PRISMA Performance Prediction Resource_management SPEC Self-adaptive-systems Self-aware-computing Tool descartes t_journalmagazine myown}, month = {April}, number = 4, pages = {800&ndash;813}, publisher = {IEEE}, title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}}, volume = 30, year = 2019 }
%0 Journal Article %1 BaHeSpAEKo2019-TPDS-ChameleonAutoscaler %A Bauer, André %A Herbst, Nikolas %A Spinner, Simon %A Ali-Eldin, Ahmed %A Kounev, Samuel %D 2019 %I IEEE %J IEEE Transactions on Parallel and Distributed Systems %N 4 %P 800--813 %T Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field %V 30

Measuring and Rating the Energy-Efficiency of Servers. von Kistowski, Jóakim; Lange, Klaus-Dieter; Arnold, Jeremy A.; Beckett, John; Block, Hansfried; Tricker, Mike; Sharma, Sanjay; Pais, Johann; Kounev, Samuel; in Future Generation Computer Systems (2019). 100 579–589.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ]
 
Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.
@article{KiLaArBeBlTrShPaKo-FGCS2019-ServerEfficiency, abstract = {Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04\% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.}, author = {von Kistowski, J{ó}akim and Lange, Klaus-Dieter and Arnold, Jeremy A. and Beckett, John and Block, Hansfried and Tricker, Mike and Sharma, Sanjay and Pais, Johann and Kounev, Samuel}, journal = {Future Generation Computer Systems}, keywords = {Metrics_and_benchmarking_methodologies Performance Power Power-energy_efficient_computing SPEC Tool descartes t_journalmagazine myown}, pages = {579 - 589}, title = {{Measuring and Rating the Energy-Efficiency of Servers}}, volume = 100, year = 2019 }
%0 Journal Article %1 KiLaArBeBlTrShPaKo-FGCS2019-ServerEfficiency %A von Kistowski, Jóakim %A Lange, Klaus-Dieter %A Arnold, Jeremy A. %A Beckett, John %A Block, Hansfried %A Tricker, Mike %A Sharma, Sanjay %A Pais, Johann %A Kounev, Samuel %D 2019 %J Future Generation Computer Systems %P 579 - 589 %T Measuring and Rating the Energy-Efficiency of Servers %U http://www.sciencedirect.com/science/article/pii/S0167739X18312123 %V 100 %X Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.

Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud. Metzger, Florian; Hoßfeld, Tobias; Bauer, André; Kounev, Samuel; Heegaard, Poul. E.; in Proceedings of the IEEE (2019). 107(4) 679–694.
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{MeHoBaKoHe2019-PIEEE-IoTTraffic, author = {Metzger, Florian and Hoßfeld, Tobias and Bauer, Andr{é} and Kounev, Samuel and Heegaard, Poul. E.}, journal = {Proceedings of the IEEE}, keywords = {Analytical_and_simulation-based_analysis Elasticity Internet_of_Things Performance Resource_management descartes t_journalmagazine myown}, month = {April}, number = 4, pages = {679-694}, title = {{Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud}}, volume = 107, year = 2019 }
%0 Journal Article %1 MeHoBaKoHe2019-PIEEE-IoTTraffic %A Metzger, Florian %A Hoßfeld, Tobias %A Bauer, André %A Kounev, Samuel %A Heegaard, Poul. E. %D 2019 %J Proceedings of the IEEE %N 4 %P 679-694 %T Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud %V 107

Online model learning for self-aware computing infrastructures. Spinner, Simon; Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in Journal of Systems and Software (2019). 147 1–16.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.
@article{SpGrEiKo2019-JSS-ModelLearning, abstract = {Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.}, author = {Spinner, Simon and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, journal = {Journal of Systems and Software}, keywords = {Analytical_and_simulation-based_analysis Application-aware Automated_model_learning Cloud DML LibReDE Meta-models Performance Prediction Simulation Virtualization descartes t_journalmagazine myown}, pages = {1&ndash;16}, title = {Online model learning for self-aware computing infrastructures}, volume = 147, year = 2019 }
%0 Journal Article %1 SpGrEiKo2019-JSS-ModelLearning %A Spinner, Simon %A Grohmann, Johannes %A Eismann, Simon %A Kounev, Samuel %D 2019 %J Journal of Systems and Software %P 1--16 %T Online model learning for self-aware computing infrastructures %U https://doi.org/10.1016/j.jss.2018.09.089 %V 147 %X Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.

2018 [ nach oben ]

CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts. Milenkoski, Aleksandar; Iosup, Alexandru; Kounev, Samuel; Sachs, Kai; Mularz, Diane E.; Curtiss, Jonathan A.; Ding, Jason J.; Rosenberg, Florian; Rygielski, Piotr; in IEEE Cloud Computing (2018). 5(3) 65–76. IEEE.
[ BibTeX ] [ EndNote ]
 
@article{MiIoKoSaMuCuDiRoRy2018-IEEECC-cup, author = {Milenkoski, Aleksandar and Iosup, Alexandru and Kounev, Samuel and Sachs, Kai and Mularz, Diane E. and Curtiss, Jonathan A. and Ding, Jason J. and Rosenberg, Florian and Rygielski, Piotr}, journal = {IEEE Cloud Computing}, keywords = {Cloud Design_of_software_and_systems Virtualization descartes t_journalmagazine myown}, month = 6, number = 3, pages = {65&ndash;76}, publisher = {IEEE}, title = {{CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts}}, volume = 5, year = 2018 }
%0 Journal Article %1 MiIoKoSaMuCuDiRoRy2018-IEEECC-cup %A Milenkoski, Aleksandar %A Iosup, Alexandru %A Kounev, Samuel %A Sachs, Kai %A Mularz, Diane E. %A Curtiss, Jonathan A. %A Ding, Jason J. %A Rosenberg, Florian %A Rygielski, Piotr %D 2018 %I IEEE %J IEEE Cloud Computing %N 3 %P 65--76 %T CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts %V 5

Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges. Herbst, Nikolas; Bauer, André; Kounev, Samuel; Oikonomou, Giorgos; van Eyk, Erwin; Kousiouris, George; Evangelinou, Athanasia; Krebs, Rouven; Brecht, Tim; Abad, Cristina L.; Iosup, Alexandru; in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) (2018). 3(4) 19:1–19:36. ACM, New York, NY, USA.
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@article{HeBaKoOiEyKoEvKrBrAbIo2018-TOMPECS-CloudMetrics, address = {New York, NY, USA}, author = {Herbst, Nikolas and Bauer, Andr{é} and Kounev, Samuel and Oikonomou, Giorgos and van Eyk, Erwin and Kousiouris, George and Evangelinou, Athanasia and Krebs, Rouven and Brecht, Tim and Abad, Cristina L. and Iosup, Alexandru}, journal = {ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)}, keywords = {BUNGEE Cloud Elasticity Isolation Metrics_and_benchmarking_methodologies PRISMA Performance Resource_management SPEC descartes myown t_journalmagazine}, month = 8, number = 4, pages = {19:1&ndash;19:36}, publisher = {ACM}, title = {{Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges}}, volume = 3, year = 2018 }
%0 Journal Article %1 HeBaKoOiEyKoEvKrBrAbIo2018-TOMPECS-CloudMetrics %A Herbst, Nikolas %A Bauer, André %A Kounev, Samuel %A Oikonomou, Giorgos %A van Eyk, Erwin %A Kousiouris, George %A Evangelinou, Athanasia %A Krebs, Rouven %A Brecht, Tim %A Abad, Cristina L. %A Iosup, Alexandru %C New York, NY, USA %D 2018 %I ACM %J ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) %N 4 %P 19:1--19:36 %T Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges %U http://doi.acm.org/10.1145/3236332 %V 3

2017 [ nach oben ]

Modeling and Extracting Load Intensity Profiles. von Kistowski, Jóakim; Herbst, Nikolas; Kounev, Samuel; Groenda, Henning; Stier, Christian; Lehrig, Sebastian; in ACM Transactions on Autonomous and Adaptive Systems (TAAS) (2017). 11(4) 23:1–23:28. ACM, New York, NY, USA.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.
@article{KiHeKo-TAAS17-ModelExtractLoadProfiles, abstract = {Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2\% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.}, address = {New York, NY, USA}, author = {von Kistowski, J{ó}akim and Herbst, Nikolas and Kounev, Samuel and Groenda, Henning and Stier, Christian and Lehrig, Sebastian}, journal = {ACM Transactions on Autonomous and Adaptive Systems (TAAS)}, keywords = {Automated_model_learning BUNGEE Elasticity LIMBO Resource_management SPEC Self-adaptive-systems Self-aware-computing Tool descartes t_journalmagazine myown}, month = {January}, number = 4, pages = {23:1&ndash;23:28}, publisher = {ACM}, title = {{Modeling and Extracting Load Intensity Profiles}}, volume = 11, year = 2017 }
%0 Journal Article %1 KiHeKo-TAAS17-ModelExtractLoadProfiles %A von Kistowski, Jóakim %A Herbst, Nikolas %A Kounev, Samuel %A Groenda, Henning %A Stier, Christian %A Lehrig, Sebastian %C New York, NY, USA %D 2017 %I ACM %J ACM Transactions on Autonomous and Adaptive Systems (TAAS) %N 4 %P 23:1--23:28 %T Modeling and Extracting Load Intensity Profiles %U http://doi.acm.org/10.1145/3019596 %V 11 %X Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.

Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language. Huber, Nikolaus; Brosig, Fabian; Spinner, Simon; Kounev, Samuel; Bähr, Manuel; in IEEE Transactions on Software Engineering (TSE) (2017). 43(5) 432–452. IEEE Computer Society.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.
@article{HuBrSpKoBa2017-TSE-DML, abstract = {Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.}, author = {Huber, Nikolaus and Brosig, Fabian and Spinner, Simon and Kounev, Samuel and B{ä}hr, Manuel}, journal = {IEEE Transactions on Software Engineering (TSE)}, keywords = {Analytical_and_simulation-based_analysis Application-aware Application_quality_of_service_management Cloud DML DQL Design_of_software_and_systems Elasticity Formal_architecture_modeling Meta-models Multi-criteria_optimization Online_monitoring_and_forecasting Optimization Performance Power-energy_efficient_computing Prediction Resource_management Self-adaptive-systems Self-aware-computing Simulation descartes t_journalmagazine myown}, number = 5, pages = {432&ndash;452}, publisher = {IEEE Computer Society}, title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}}, volume = 43, year = 2017 }
%0 Journal Article %1 HuBrSpKoBa2017-TSE-DML %A Huber, Nikolaus %A Brosig, Fabian %A Spinner, Simon %A Kounev, Samuel %A Bähr, Manuel %D 2017 %I IEEE Computer Society %J IEEE Transactions on Software Engineering (TSE) %N 5 %P 432--452 %T Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language %U http://dx.doi.org/10.1109/TSE.2016.2613863 %V 43 %X Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.

2016 [ nach oben ]

A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures. Kounev, Samuel; Huber, Nikolaus; Brosig, Fabian; Zhu, Xiaoyun; in IEEE Computer (2016). 49(7) 53–61. IEEE.
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@article{KoHuBrZh2016-IEEEComputer-Descartes, author = {Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian and Zhu, Xiaoyun}, journal = {IEEE Computer}, keywords = {Analytical_and_simulation-based_analysis Application-aware Application_quality_of_service_management Cloud DML Design_of_software_and_systems Elasticity Formal_architecture_modeling Meta-models Online_monitoring_and_forecasting Optimization Performance Power-energy_efficient_computing Prediction Resource_management Self-adaptive-systems Self-aware-computing Simulation descartes t_journalmagazine myown}, month = {July}, number = 7, pages = {53-61}, publisher = {IEEE}, title = {{A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures}}, volume = 49, year = 2016 }
%0 Journal Article %1 KoHuBrZh2016-IEEEComputer-Descartes %A Kounev, Samuel %A Huber, Nikolaus %A Brosig, Fabian %A Zhu, Xiaoyun %D 2016 %I IEEE %J IEEE Computer %N 7 %P 53-61 %T A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures %U http://dx.doi.org/10.1109/MC.2016.198 %V 49

Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation. Müller, Christoph; Rygielski, Piotr; Spinner, Simon; Kounev, Samuel; in Electronic Notes in Theoretical Computer Science (2016). 327 71–91. Elsevier.
The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Abstract Due to the growing size of modern IT systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.
@article{MuRySpKo2016-PASM-QPN-LQN-Transformation, abstract = {Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.}, author = {M{ü}ller, Christoph and Rygielski, Piotr and Spinner, Simon and Kounev, Samuel}, journal = {Electronic Notes in Theoretical Computer Science}, keywords = {Analytical_and_simulation-based_analysis Performance QPME QPN descartes t_journalmagazine t_workshop myown}, note = {The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016}, pages = {71&ndash;91}, publisher = {Elsevier}, title = {{Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}}, volume = 327, year = 2016 }
%0 Journal Article %1 MuRySpKo2016-PASM-QPN-LQN-Transformation %A Müller, Christoph %A Rygielski, Piotr %A Spinner, Simon %A Kounev, Samuel %D 2016 %I Elsevier %J Electronic Notes in Theoretical Computer Science %P 71--91 %T Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation %U http://www.sciencedirect.com/science/article/pii/S157106611630069X %V 327 %X Abstract Due to the growing size of modern IT systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

2015 [ nach oben ]

Parallel Simulation of Queueing Petri Nets. Walter, Jürgen; Spinner, Simon; Kounev, Samuel; in EAI Endorsed Transactions on Industrial Networks and Intelligent Systems (2015). 16(8) ACM.
[ BibTeX ] [ EndNote ]
 
@article{WaSpKo2016-EAI-parallelSimulation, author = {Walter, J{ü}rgen and Spinner, Simon and Kounev, Samuel}, journal = {EAI Endorsed Transactions on Industrial Networks and Intelligent Systems}, keywords = {descartes se2 t_journalmagazine myown}, month = 8, number = 8, publisher = {ACM}, title = {{Parallel Simulation of Queueing Petri Nets}}, volume = 16, year = 2015 }
%0 Journal Article %1 WaSpKo2016-EAI-parallelSimulation %A Walter, Jürgen %A Spinner, Simon %A Kounev, Samuel %D 2015 %I ACM %J EAI Endorsed Transactions on Industrial Networks and Intelligent Systems %N 8 %T Parallel Simulation of Queueing Petri Nets %V 16

Evaluating Approaches to Resource Demand Estimation. Spinner, Simon; Casale, Giuliano; Brosig, Fabian; Kounev, Samuel; in Performance Evaluation (2015). 92 51–71. Elsevier B.V.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.
@article{SpCaBrKo2015-PEVA-ResDemEstSurvey, abstract = {Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.}, author = {Spinner, Simon and Casale, Giuliano and Brosig, Fabian and Kounev, Samuel}, journal = {Performance Evaluation}, keywords = {Analytical_and_simulation-based_analysis Automated_model_learning Instrumentation_profiling_and_workload_characterization LibReDE Performance Statistical_estimation_and_machine_learning Survey descartes t_journalmagazine myown}, month = {October}, pages = {51&ndash;71}, publisher = {Elsevier B.V.}, title = {{Evaluating Approaches to Resource Demand Estimation}}, volume = 92, year = 2015 }
%0 Journal Article %1 SpCaBrKo2015-PEVA-ResDemEstSurvey %A Spinner, Simon %A Casale, Giuliano %A Brosig, Fabian %A Kounev, Samuel %D 2015 %I Elsevier B.V. %J Performance Evaluation %P 51--71 %T Evaluating Approaches to Resource Demand Estimation %U http://www.sciencedirect.com/science/article/pii/S0166531615000711 %V 92 %X Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.

Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices. Milenkoski, Aleksandar; Vieira, Marco; Kounev, Samuel; Avritzer, Alberto; Payne, Bryan D.; in ACM Computing Surveys (2015). 48(1) 12:1–12:41. ACM, New York, NY, USA.
5-year Impact Factor (2014): 5.949
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.
@article{MiViKoAvPa2015-CSUR-IDSEval, abstract = {{The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.}}, address = {New York, NY, USA}, author = {Milenkoski, Aleksandar and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Payne, Bryan D.}, journal = {{ACM Computing Surveys}}, keywords = {Metrics_and_benchmarking_methodologies Reliability Security Survey Virtualization descartes t_journalmagazine myown}, month = {{September}}, note = {{<b>5-year Impact Factor (2014): 5.949</b>}}, number = 1, pages = {12:1&ndash;12:41}, publisher = {ACM}, title = {{Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}}, volume = 48, year = 2015 }
%0 Journal Article %1 MiViKoAvPa2015-CSUR-IDSEval %A Milenkoski, Aleksandar %A Vieira, Marco %A Kounev, Samuel %A Avritzer, Alberto %A Payne, Bryan D. %C New York, NY, USA %D 2015 %I ACM %J ACM Computing Surveys %N 1 %P 12:1--12:41 %T Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices %U http://dl.acm.org/authorize?N06203 %V 48 %X The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.

Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures. Brosig, Fabian; Meier, Philipp; Becker, Steffen; Koziolek, Anne; Koziolek, Heiko; Kounev, Samuel; in IEEE Transactions on Software Engineering (TSE) (2015). 41(2) 157–175. IEEE.
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@article{BrMeBeKoKoKo2014-TSE-ModelSolutionTradeOffs, author = {Brosig, Fabian and Meier, Philipp and Becker, Steffen and Koziolek, Anne and Koziolek, Heiko and Kounev, Samuel}, journal = {IEEE Transactions on Software Engineering (TSE)}, keywords = {Analytical_and_simulation-based_analysis Formal_architecture_modeling Meta-models Performance Prediction QPME QPME_Bibliography QPN Simulation descartes t_journalmagazine myown}, month = {{February}}, number = 2, pages = {157&ndash;175}, publisher = {IEEE}, title = {{Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures}}, volume = 41, year = 2015 }
%0 Journal Article %1 BrMeBeKoKoKo2014-TSE-ModelSolutionTradeOffs %A Brosig, Fabian %A Meier, Philipp %A Becker, Steffen %A Koziolek, Anne %A Koziolek, Heiko %A Kounev, Samuel %D 2015 %I IEEE %J IEEE Transactions on Software Engineering (TSE) %N 2 %P 157--175 %T Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures %U http://dx.doi.org/10.1109/TSE.2014.2362755 %V 41

2014 [ nach oben ]

Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments. Huber, Nikolaus; van Hoorn, André; Koziolek, Anne; Brosig, Fabian; Kounev, Samuel; in Service Oriented Computing and Applications Journal (SOCA) (2014). 8(1) 73–89. Springer-Verlag.
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{HuHoKoBrKo2014-SOCA-ModelingRuntimeAdaptation, author = {Huber, Nikolaus and van Hoorn, Andr{é} and Koziolek, Anne and Brosig, Fabian and Kounev, Samuel}, journal = {Service Oriented Computing and Applications Journal (SOCA)}, keywords = {Application_quality_of_service_management DML Design_of_software_and_systems Elasticity Formal_architecture_modeling Meta-models Multi-criteria_optimization Optimization Performance Resource_management SOA Self-adaptive-systems Self-aware-computing descartes t_journalmagazine myown}, month = {{March}}, number = 1, pages = {73&ndash;89}, publisher = {Springer-Verlag}, title = {{Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments}}, volume = 8, year = 2014 }
%0 Journal Article %1 HuHoKoBrKo2014-SOCA-ModelingRuntimeAdaptation %A Huber, Nikolaus %A van Hoorn, André %A Koziolek, Anne %A Brosig, Fabian %A Kounev, Samuel %D 2014 %I Springer-Verlag %J Service Oriented Computing and Applications Journal (SOCA) %N 1 %P 73--89 %T Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments %V 8

Architecture-Level Software Performance Abstractions for Online Performance Prediction. Brosig, Fabian; Huber, Nikolaus; Kounev, Samuel; in Elsevier Science of Computer Programming Journal (SciCo) (2014). Vol. 90, Part B 71–92. Elsevier.
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@article{BrHuKo2013-SciCo-SoftwarePerformanceAbstractions, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, journal = {Elsevier Science of Computer Programming Journal (SciCo)}, keywords = {Analytical_and_simulation-based_analysis Application_quality_of_service_management DML Formal_architecture_modeling Meta-models Performance Prediction Self-aware-computing descartes t_journalmagazine myown}, month = {September}, pages = {71&ndash;92}, publisher = {Elsevier}, title = {{Architecture-Level Software Performance Abstractions for Online Performance Prediction}}, volume = {Vol. 90, Part B}, year = 2014 }
%0 Journal Article %1 BrHuKo2013-SciCo-SoftwarePerformanceAbstractions %A Brosig, Fabian %A Huber, Nikolaus %A Kounev, Samuel %D 2014 %I Elsevier %J Elsevier Science of Computer Programming Journal (SciCo) %P 71--92 %T Architecture-Level Software Performance Abstractions for Online Performance Prediction %U http://authors.elsevier.com/sd/article/S0167642313001421 %V Vol. 90, Part B

Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments. Krebs, Rouven; Momm, Christof; Kounev, Samuel; in Elsevier Science of Computer Programming Journal (SciCo) (2014). Vol. 90, Part B 116–134. Elsevier B.V.
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{KrMoKo2013-SciCo-MetricsAndTechniquesForPerformanceIsolation, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, journal = {Elsevier Science of Computer Programming Journal (SciCo)}, keywords = {Application_quality_of_service_management Cloud Isolation Metrics_and_benchmarking_methodologies Multi-tenancy Performance Resource_management Virtualization descartes t_journalmagazine myown}, month = {February}, pages = {116&ndash;134}, publisher = {Elsevier B.V.}, title = {{Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments}}, volume = {Vol. 90, Part B}, year = 2014 }
%0 Journal Article %1 KrMoKo2013-SciCo-MetricsAndTechniquesForPerformanceIsolation %A Krebs, Rouven %A Momm, Christof %A Kounev, Samuel %D 2014 %I Elsevier B.V. %J Elsevier Science of Computer Programming Journal (SciCo) %P 116--134 %T Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments %V Vol. 90, Part B

Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning. Herbst, Nikolas Roman; Huber, Nikolaus; Kounev, Samuel; Amrehn, Erich; in Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd. (2014). 26(12) 2053–2078.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.
@article{HuKoAm2013-CCPE-WorkloadClassificationAndForecasting, abstract = {As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.}, author = {Herbst, Nikolas Roman and Huber, Nikolaus and Kounev, Samuel and Amrehn, Erich}, journal = {Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd.}, keywords = {Application_quality_of_service_management Automated_model_learning Elasticity Instrumentation_profiling_and_workload_characterization Online_monitoring_and_forecasting Performance Prediction Resource_management Self-adaptive-systems Self-aware-computing Statistical_estimation_and_machine_learning Tool WCF descartes t_journalmagazine myown}, month = {March}, number = 12, pages = {2053&ndash;2078}, title = {{Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning}}, volume = 26, year = 2014 }
%0 Journal Article %1 HuKoAm2013-CCPE-WorkloadClassificationAndForecasting %A Herbst, Nikolas Roman %A Huber, Nikolaus %A Kounev, Samuel %A Amrehn, Erich %D 2014 %J Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd. %N 12 %P 2053--2078 %T Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning %U http://dx.doi.org/10.1002/cpe.3224 %V 26 %X As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.

2013 [ nach oben ]

Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey. Rygielski, Piotr; Kounev, Samuel; in PIK --- Praxis der Informationsverarbeitung und Kommunikation (2013). 36(1) 55–64. de Gruyter.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.
@article{RyKo2013, abstract = {The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.}, author = {Rygielski, Piotr and Kounev, Samuel}, journal = {PIK &mdash; Praxis der Informationsverarbeitung und Kommunikation}, keywords = {Application_quality_of_service_management Cloud Networking Performance Resource_management SDN Survey Virtualization descartes t_journalmagazine myown}, month = {February}, number = 1, pages = {55&ndash;64}, publisher = {de Gruyter}, title = {{Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey}}, volume = 36, year = 2013 }
%0 Journal Article %1 RyKo2013 %A Rygielski, Piotr %A Kounev, Samuel %D 2013 %I de Gruyter %J PIK --- Praxis der Informationsverarbeitung und Kommunikation %N 1 %P 55--64 %T Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey %U http://www.degruyter.com/view/j/piko-2013-36-issue-1/pik-2012-0136/pik-2012-0136.xml?format=INT %V 36 %X The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.

Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions. Kounev, Samuel; Rathfelder, Christoph; Klatt, Benjamin; in Electronic Notes in Theoretical Computer Science (ENTCS) (2013). 295 3–9. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.
@article{KoRaKl2012-FESCA-Keynote, abstract = {Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.}, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel and Rathfelder, Christoph and Klatt, Benjamin}, journal = {{Electronic Notes in Theoretical Computer Science (ENTCS)}}, keywords = {Analytical_and_simulation-based_analysis Application_quality_of_service_management Event-based Formal_architecture_modeling Meta-models Networking Performance Prediction Simulation Survey descartes t_journalmagazine myown}, month = {may}, pages = {3&ndash;9}, publisher = {Elsevier Science Publishers B. V.}, title = {{Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions}}, volume = 295, year = 2013 }
%0 Journal Article %1 KoRaKl2012-FESCA-Keynote %A Kounev, Samuel %A Rathfelder, Christoph %A Klatt, Benjamin %C Amsterdam, The Netherlands %D 2013 %I Elsevier Science Publishers B. V. %J Electronic Notes in Theoretical Computer Science (ENTCS) %P 3--9 %T Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions %U http://www.sciencedirect.com/science/article/pii/S1571066113000248 %V 295 %X Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.

Performance modeling and analysis of message-oriented event-driven systems. Sachs, Kai; Kounev, Samuel; Buchmann, Alejandro; in Journal of Software and Systems Modeling (2013). 12(4) 705–729. Springer-Verlag.
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.
@article{SaKoBu2011-SoSyM-PerfModMoEdSys, abstract = {Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.}, author = {Sachs, Kai and Kounev, Samuel and Buchmann, Alejandro}, journal = {Journal of Software and Systems Modeling}, keywords = {Analytical_and_simulation-based_analysis Application_quality_of_service_management Event-based Formal_architecture_modeling Performance Prediction QPME_Bibliography QPN Simulation descartes t_journalmagazine myown}, number = 4, pages = {705&ndash;729}, publisher = {Springer-Verlag}, title = {Performance modeling and analysis of message-oriented event-driven systems}, volume = 12, year = 2013 }
%0 Journal Article %1 SaKoBu2011-SoSyM-PerfModMoEdSys %A Sachs, Kai %A Kounev, Samuel %A Buchmann, Alejandro %D 2013 %I Springer-Verlag %J Journal of Software and Systems Modeling %N 4 %P 705--729 %T Performance modeling and analysis of message-oriented event-driven systems %V 12 %X Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.

Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions. Rathfelder, Christoph; Klatt, Benjamin; Sachs, Kai; Kounev, Samuel; in Journal of Software and Systems Modeling (SoSyM) (2013). 1–27. Springer Verlag.
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@article{rathfelder2013a, author = {Rathfelder, Christoph and Klatt, Benjamin and Sachs, Kai and Kounev, Samuel}, journal = {Journal of Software and Systems Modeling (SoSyM)}, keywords = {Analytical_and_simulation-based_analysis Application_quality_of_service_management Event-based Formal_architecture_modeling Meta-models Performance Prediction descartes t_journalmagazine myown}, month = {March}, pages = {1&ndash;27}, publisher = {Springer Verlag}, title = {Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions}, year = 2013 }
%0 Journal Article %1 rathfelder2013a %A Rathfelder, Christoph %A Klatt, Benjamin %A Sachs, Kai %A Kounev, Samuel %D 2013 %I Springer Verlag %J Journal of Software and Systems Modeling (SoSyM) %P 1--27 %T Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions %U http://dx.doi.org/10.1007/s10270-013-0316-x

2011 [ nach oben ]

Self-Aware Software and Systems Engineering: A Vision and Research Roadmap. Kounev, Samuel; in GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928 (2011). Karlsruhe, Germany.
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@inproceedings{Ko2011-SE-DescartesResearch, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, booktitle = {{GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928}}, keywords = {Application_quality_of_service_management Automated_model_learning Design_of_software_and_systems Formal_architecture_modeling Online_monitoring_and_forecasting Performance Prediction Reliability Resource_management Self-adaptive-systems Self-aware-computing descartes t_journalmagazine myown}, title = {{Self-Aware Software and Systems Engineering: A Vision and Research Roadmap}}, year = 2011 }
%0 Conference Paper %1 Ko2011-SE-DescartesResearch %A Kounev, Samuel %B GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928 %C Karlsruhe, Germany %D 2011 %T Self-Aware Software and Systems Engineering: A Vision and Research Roadmap %U http://pi.informatik.uni-siegen.de/stt/31_4/index.html

2009 [ nach oben ]

Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark. Sachs, Kai; Kounev, Samuel; Bacon, Jean; Buchmann, Alejandro; in Performance Evaluation (2009). 66(8) 410–434. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.
@article{SaKoBaBu2008-PERFEVAL-SPECjms2007, abstract = {Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.}, address = {Amsterdam, The Netherlands}, author = {Sachs, Kai and Kounev, Samuel and Bacon, Jean and Buchmann, Alejandro}, journal = {Performance Evaluation}, keywords = {Event-based Instrumentation_profiling_and_workload_characterization Metrics_and_benchmarking_methodologies Performance SPEC Tool descartes t_journalmagazine myown}, month = {August}, number = 8, pages = {410&ndash;434}, publisher = {Elsevier Science Publishers B. V.}, title = {{Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark}}, volume = 66, year = 2009 }
%0 Journal Article %1 SaKoBaBu2008-PERFEVAL-SPECjms2007 %A Sachs, Kai %A Kounev, Samuel %A Bacon, Jean %A Buchmann, Alejandro %C Amsterdam, The Netherlands %D 2009 %I Elsevier Science Publishers B. V. %J Performance Evaluation %N 8 %P 410--434 %T Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark %U http://www.elsevier.com/wps/find/journaldescription.cws_home/505618/description %V 66 %X Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.

Autonomic QoS control in enterprise Grid environments using online simulation. Nou, Ramon; Kounev, Samuel; Julia, Ferran; Torres, Jordi; in Journal of Systems and Software (2009). 82(3) 486–502. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.
@article{NoKoJuTo2008-JSS-GridAutoQoS, abstract = {As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.}, address = {Amsterdam, The Netherlands}, author = {Nou, Ramon and Kounev, Samuel and Julia, Ferran and Torres, Jordi}, journal = {Journal of Systems and Software}, keywords = {Analytical_and_simulation-based_analysis Application_quality_of_service_management Automated_model_learning Design_of_software_and_systems Formal_architecture_modeling Grid Instrumentation_profiling_and_workload_characterization Performance Prediction QPME_Bibliography QPN Reliability Resource_management SOA Self-adaptive-systems Self-aware-computing Simulation descartes t_journalmagazine myown}, month = {March}, number = 3, pages = {486&ndash;502}, publisher = {Elsevier Science Publishers B. V.}, title = {{Autonomic QoS control in enterprise Grid environments using online simulation}}, volume = 82, year = 2009 }
%0 Journal Article %1 NoKoJuTo2008-JSS-GridAutoQoS %A Nou, Ramon %A Kounev, Samuel %A Julia, Ferran %A Torres, Jordi %C Amsterdam, The Netherlands %D 2009 %I Elsevier Science Publishers B. V. %J Journal of Systems and Software %N 3 %P 486--502 %T Autonomic QoS control in enterprise Grid environments using online simulation %V 82 %X As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.

Benchmarking and Performance Modeling of Event-Based Systems. Kounev, Samuel; Sachs, Kai; in it - Information Technology (2009). 51(5) 262–269. Oldenbourg Wissenschaftsverlag, Munich, Germany.
[ Abstract ] [ BibTeX ] [ EndNote ]
 
Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.
@article{KoSa2009-DEBSPerformance, abstract = {Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.}, address = {Munich, Germany}, author = {Kounev, Samuel and Sachs, Kai}, journal = {it - Information Technology}, keywords = {Analytical_and_simulation-based_analysis Application_quality_of_service_management Event-based Formal_architecture_modeling Metrics_and_benchmarking_methodologies Performance Prediction QPME_Bibliography QPN descartes t_journalmagazine myown}, month = {October}, number = 5, pages = {262&ndash;269}, publisher = {Oldenbourg Wissenschaftsverlag}, title = {{Benchmarking and Performance Modeling of Event-Based Systems}}, volume = 51, year = 2009 }
%0 Journal Article %1 KoSa2009-DEBSPerformance %A Kounev, Samuel %A Sachs, Kai %C Munich, Germany %D 2009 %I Oldenbourg Wissenschaftsverlag %J it - Information Technology %N 5 %P 262--269 %T Benchmarking and Performance Modeling of Event-Based Systems %V 51 %X Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.

QPME - A Performance Modeling Tool Based on Queueing Petri Nets. Kounev, Samuel; Dutz, Christofer; in ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis (2009). 36 46–51. ACM.
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.
@article{KoDu2009-SIGMETRICS_PER-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.}, author = {Kounev, Samuel and Dutz, Christofer}, journal = {ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis}, keywords = {Analytical_and_simulation-based_analysis Formal_architecture_modeling Performance Prediction QPME QPME_Bibliography QPN Simulation Tool descartes t_journalmagazine myown}, month = 3, pages = {46&ndash;51}, publisher = {ACM}, title = {{QPME - A Performance Modeling Tool Based on Queueing Petri Nets}}, volume = 36, year = 2009 }
%0 Journal Article %1 KoDu2009-SIGMETRICS_PER-QPME %A Kounev, Samuel %A Dutz, Christofer %D 2009 %I ACM %J ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis %P 46--51 %T QPME - A Performance Modeling Tool Based on Queueing Petri Nets %V 36 %X Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.

2008 [ nach oben ]

Kaffeekunde - SPECjms misst Message-oriented Middleware. Kounev, Samuel; Sachs, Kai; (2008).
[ BibTeX ] [ EndNote ] [ URL ]
 
@misc{sachs2008a, author = {Kounev, Samuel and Sachs, Kai}, howpublished = {iX Magazin, Heft 02/2008, Heise Zeitschriften Verlag}, keywords = {Event-based Metrics_and_benchmarking_methodologies Performance SPEC Tool descartes t_journalmagazine myown}, title = {{Kaffeekunde - SPECjms misst Message-oriented Middleware}}, year = 2008 }
%0 Generic %1 sachs2008a %A Kounev, Samuel %A Sachs, Kai %D 2008 %T Kaffeekunde - SPECjms misst Message-oriented Middleware %U http://www.heise.de/kiosk/archiv/ix/2008/2/121

2006 [ nach oben ]

SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation. Kounev, Samuel; Buchmann, Alejandro; in Performance Evaluation (2006). 63(4-5) 364–394. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{KoBu2006-PERFEVAL-SimQPN, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel and Buchmann, Alejandro}, journal = {Performance Evaluation}, keywords = {Analytical_and_simulation-based_analysis Formal_architecture_modeling Performance Prediction QPME QPME_Bibliography QPN Simulation Tool descartes t_journalmagazine myown}, month = 5, number = {4-5}, pages = {364&ndash;394}, publisher = {Elsevier Science Publishers B. V.}, title = {{SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation}}, volume = 63, year = 2006 }
%0 Journal Article %1 KoBu2006-PERFEVAL-SimQPN %A Kounev, Samuel %A Buchmann, Alejandro %C Amsterdam, The Netherlands %D 2006 %I Elsevier Science Publishers B. V. %J Performance Evaluation %N 4-5 %P 364--394 %T SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation %V 63

Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets. Kounev, Samuel; in IEEE Transactions on Software Engineering (2006). 32(7) 486–502.
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.
@article{Ko2006-IEEE_TSE-QPN_ModelingMethod, abstract = {Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.}, author = {Kounev, Samuel}, journal = {IEEE Transactions on Software Engineering}, keywords = {QPME_Bibliography descartes t_journalmagazine myown}, month = {July}, number = 7, pages = {486&ndash;502}, title = {{Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets}}, volume = 32, year = 2006 }
%0 Journal Article %1 Ko2006-IEEE_TSE-QPN_ModelingMethod %A Kounev, Samuel %D 2006 %J IEEE Transactions on Software Engineering %N 7 %P 486--502 %T Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets %V 32 %X Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.

2004 [ nach oben ]

Performance Tuning and Optimization of J2EE Applications on the JBoss Platform. Kounev, Samuel; Weis, Börn; Buchmann, Alejandro; in Journal of Computer Resource Management (2004). 113 Computer Measurement Group (CMG).
[ BibTeX ] [ EndNote ] [ Download ]
 
@article{KoWeBu2004-CMG_JCRM-JBoss, author = {Kounev, Samuel and Weis, B{ö}rn and Buchmann, Alejandro}, journal = {Journal of Computer Resource Management}, keywords = {Design_of_software_and_systems Instrumentation_profiling_and_workload_characterization Metrics_and_benchmarking_methodologies Optimization Performance SPEC descartes t_journalmagazine myown}, month = {September}, publisher = {Computer Measurement Group (CMG)}, title = {{Performance Tuning and Optimization of J2EE Applications on the JBoss Platform}}, volume = 113, year = 2004 }
%0 Journal Article %1 KoWeBu2004-CMG_JCRM-JBoss %A Kounev, Samuel %A Weis, Börn %A Buchmann, Alejandro %D 2004 %I Computer Measurement Group (CMG) %J Journal of Computer Resource Management %T Performance Tuning and Optimization of J2EE Applications on the JBoss Platform %V 113

1999 [ nach oben ]

Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server. Nenov, Plamen; Kounev, Samuel; Mihailov, Dimiter; in Journal of Computing and Information (1999).
[ BibTeX ] [ EndNote ]
 
@article{NeKoMi1999-JCI-VideoConf, author = {Nenov, Plamen and Kounev, Samuel and Mihailov, Dimiter}, journal = {Journal of Computing and Information}, keywords = {Design_of_software_and_systems descartes t_journalmagazine myown}, title = {{Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server}}, year = 1999 }
%0 Journal Article %1 NeKoMi1999-JCI-VideoConf %A Nenov, Plamen %A Kounev, Samuel %A Mihailov, Dimiter %D 1999 %J Journal of Computing and Information %T Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server

2022 [ nach oben ]

Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes. Prantl, Thomas; Zeck, Timo; Iffländer, Lukas; Beierlieb, Lukas; Dmitrenko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2022 5th Conference on Cloud and Internet of Things (CIoT) (2022). IEEE.
[ BibTeX ] [ EndNote ]
 
@inproceedings{PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark, author = {Prantl, Thomas and Zeck, Timo and Iffländer, Lukas and Beierlieb, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2022 5th Conference on Cloud and Internet of Things (CIoT)}, keywords = {descartes t_full myown}, organization = {IEEE}, title = {Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes}, year = 2022 }
%0 Conference Paper %1 PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark %A Prantl, Thomas %A Zeck, Timo %A Iffländer, Lukas %A Beierlieb, Lukas %A Dmitrenko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2022 5th Conference on Cloud and Internet of Things (CIoT) %D 2022 %T Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes

Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling. Straesser, Martin; Grohmann, Johannes; von Kistowski, Jóakim; Eismann, Simon; Bauer, André; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 105–115. Association for Computing Machinery, New York, NY, USA.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.
@inproceedings{10.1145/3489525.3511680, abstract = {Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.}, address = {New York, NY, USA}, author = {Straesser, Martin and Grohmann, Johannes and von Kistowski, J{ó}akim and Eismann, Simon and Bauer, Andr{é} and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {sys:relevantfor:se-group autoscaling cloud_computing descartes microservices t_full myown}, pages = {105–115}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling}, year = 2022 }
%0 Conference Paper %1 10.1145/3489525.3511680 %A Straesser, Martin %A Grohmann, Johannes %A von Kistowski, Jóakim %A Eismann, Simon %A Bauer, André %A Kounev, Samuel %B Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering %C New York, NY, USA %D 2022 %I Association for Computing Machinery %P 105–115 %T Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling %U https://doi.org/10.1145/3489525.3511680 %X Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.

2021 [ nach oben ]

A Simulation-based Optimization Framework for Online Adaptation of Networks. Herrnleben, Stefan; Grohmann, Johannes; Rygielski, Pitor; Lesch, Veronika; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools), H. Song, D. Jiang (eds.) (2021). 513–532. Springer International Publishing, Cham.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.
@inproceedings{HeGrRyLeKrKo2020-SIMUtools-Network-Online-Adaptation, abstract = {Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.}, address = {Cham}, author = {Herrnleben, Stefan and Grohmann, Johannes and Rygielski, Pitor and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, editor = {Song, Houbing and Jiang, Dingde}, keywords = {descartes myown networking t_full}, month = {August}, pages = {513&ndash;532}, publisher = {Springer International Publishing}, series = {SIMUtools 2020}, title = {A Simulation-based Optimization Framework for Online Adaptation of Networks}, year = 2021 }
%0 Conference Paper %1 HeGrRyLeKrKo2020-SIMUtools-Network-Online-Adaptation %A Herrnleben, Stefan %A Grohmann, Johannes %A Rygielski, Pitor %A Lesch, Veronika %A Krupitzer, Christian %A Kounev, Samuel %B Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) %C Cham %D 2021 %E Song, Houbing %E Jiang, Dingde %I Springer International Publishing %P 513--532 %T A Simulation-based Optimization Framework for Online Adaptation of Networks %U https://doi.org/10.1007/978-3-030-72792-5_41 %X Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.

ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations. Herrnleben, Stefan; Leidinger, Maximilian; Lesch, Veronika; Prantl, Thomas; Grohmann, Johannes; Krupitzer, Christian; Kounev, Samuel; in Performance Evaluation Methodologies and Tools, Q. Zhao, L. Xia (eds.) (2021). 72–92. Springer International Publishing, Cham.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.
@inproceedings{10.1007/978-3-030-92511-6_5, abstract = {Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Leidinger, Maximilian and Lesch, Veronika and Prantl, Thomas and Grohmann, Johannes and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, editor = {Zhao, Qianchuan and Xia, Li}, keywords = {descartes networking t_full myown}, pages = {72&ndash;92}, publisher = {Springer International Publishing}, title = {ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations}, year = 2021 }
%0 Conference Paper %1 10.1007/978-3-030-92511-6_5 %A Herrnleben, Stefan %A Leidinger, Maximilian %A Lesch, Veronika %A Prantl, Thomas %A Grohmann, Johannes %A Krupitzer, Christian %A Kounev, Samuel %B Performance Evaluation Methodologies and Tools %C Cham %D 2021 %E Zhao, Qianchuan %E Xia, Li %I Springer International Publishing %P 72--92 %T ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations %U https://link.springer.com/chapter/10.1007%2F978-3-030-92511-6_5 %X Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.

Sizeless: Predicting the Optimal Size of Serverless Functions. Eismann, Simon; Bui, Long; Grohmann, Johannes; Abad, Cristina; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 22nd International MIDDLEWARE Conference (2021). 248–259.
Best Student Paper Award, ACM Artifacts Evaluated — Functional
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.
@inproceedings{eismann2021sizeless, abstract = {Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.}, author = {Eismann, Simon and Bui, Long and Grohmann, Johannes and Abad, Cristina and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 22nd International MIDDLEWARE Conference}, keywords = {descartes t_full myown}, note = {Best Student Paper Award, ACM Artifacts Evaluated — Functional}, pages = {248–259}, title = {Sizeless: Predicting the Optimal Size of Serverless Functions}, year = 2021 }
%0 Conference Paper %1 eismann2021sizeless %A Eismann, Simon %A Bui, Long %A Grohmann, Johannes %A Abad, Cristina %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 22nd International MIDDLEWARE Conference %D 2021 %P 248–259 %T Sizeless: Predicting the Optimal Size of Serverless Functions %U https://dl.acm.org/doi/abs/10.1145/3464298.3493398 %X Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.

Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction. Züfle, Marwin; Erhard, Florian; Kounev, Samuel; in 20th IEEE International Conference on Machine Learning and Applications (ICMLA) (2021).
(to appear)
[ BibTeX ] [ EndNote ]
 
@inproceedings{zufle2021machine, author = {Z{ü}fle, Marwin and Erhard, Florian and Kounev, Samuel}, booktitle = {20th IEEE International Conference on Machine Learning and Applications (ICMLA)}, keywords = {descartes online_monitoring_and_forecasting prediction statistical_estimation_and_machine_learning t_full myown}, note = {(to appear)}, title = {Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction}, year = 2021 }
%0 Conference Paper %1 zufle2021machine %A Züfle, Marwin %A Erhard, Florian %A Kounev, Samuel %B 20th IEEE International Conference on Machine Learning and Applications (ICMLA) %D 2021 %T Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction

Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT. Prantl, Thomas; Prantl, Dominik; Bauer, André; Iffländer, Lukas; Dmitrenko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
[ BibTeX ] [ EndNote ]
 
@inproceedings{PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption, author = {Prantl, Thomas and Prantl, Dominik and Bauer, André and Iffländer, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {descartes sss-group t_full myown}, title = {Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT}, year = 2021 }
%0 Conference Paper %1 PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption %A Prantl, Thomas %A Prantl, Dominik %A Bauer, André %A Iffländer, Lukas %A Dmitrenko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) %D 2021 %T Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT

Performance Evaluation of a Post-Quantum Public-Key Cryptosystem. Prantl, Thomas; Prantl, Dominik; Beierlieb, Lukas; Iffländer, Lukas; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021). IEEE.
[ BibTeX ] [ EndNote ]
 
@inproceedings{PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem, author = {Prantl, Thomas and Prantl, Dominik and Beierlieb, Lukas and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {sys:relevantfor:se-group descartes sss-group t_full t_workshop myown}, organization = {IEEE}, title = {Performance Evaluation of a Post-Quantum Public-Key Cryptosystem}, year = 2021 }
%0 Conference Paper %1 PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem %A Prantl, Thomas %A Prantl, Dominik %A Beierlieb, Lukas %A Iffländer, Lukas %A Dmitrienko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) %D 2021 %T Performance Evaluation of a Post-Quantum Public-Key Cryptosystem

Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies. Lesch, Veronika; Noack, Tanja; Hefter, Johannes; Kounev, Samuel; Krupitzer, Christian; in Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021 (2021). IEEE.
Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%
[ BibTeX ] [ EndNote ] [ Download ]
 
@inproceedings{lesch2021towards, author = {Lesch, Veronika and Noack, Tanja and Hefter, Johannes and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021}, keywords = {cyber-physical_systems descartes internet_of_things t_full myown}, note = {Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%}, publisher = {IEEE}, title = {Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies}, year = 2021 }
%0 Conference Paper %1 lesch2021towards %A Lesch, Veronika %A Noack, Tanja %A Hefter, Johannes %A Kounev, Samuel %A Krupitzer, Christian %B Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021 %D 2021 %I IEEE %T Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies

Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping. Finke, Nils; Mohr, Marisa; Lontke, Alexander; Züfle, Marwin; Kounev, Samuel; Möller, Ralf; in Research Challenges in Information Science (RCIS) (2021). 189–204. Springer.
[ BibTeX ] [ EndNote ]
 
@inproceedings{finke2021recommendations, author = {Finke, Nils and Mohr, Marisa and Lontke, Alexander and Z{ü}fle, Marwin and Kounev, Samuel and M{ö}ller, Ralf}, booktitle = {Research Challenges in Information Science (RCIS)}, keywords = {descartes prediction statistical_estimation_and_machine_learning t_full t_interdisciplinary myown}, organization = {Springer}, pages = {189&ndash;204}, title = {Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping}, year = 2021 }
%0 Conference Paper %1 finke2021recommendations %A Finke, Nils %A Mohr, Marisa %A Lontke, Alexander %A Züfle, Marwin %A Kounev, Samuel %A Möller, Ralf %B Research Challenges in Information Science (RCIS) %D 2021 %P 189--204 %T Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping

A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0. Züfle, Marwin; Agne, Joachim; Grohmann, Johannes; Dörtoluk, Ibrahim; Kounev, Samuel; in Proceedings of the 21st IEEE IES International Conference on Industrial Informatics (2021). IEEE.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.
@inproceedings{ZuAgGrDoKo-INDIN2021-PdM, abstract = {Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.}, author = {Züfle, Marwin and Agne, Joachim and Grohmann, Johannes and Dörtoluk, Ibrahim and Kounev, Samuel}, booktitle = {Proceedings of the 21st IEEE IES International Conference on Industrial Informatics}, keywords = {Cyber-Physical_Systems Internet_of_Things descartes prediction statistical_estimation_and_machine_learning t_full t_interdisciplinary myown}, month = {July}, publisher = {IEEE}, series = {INDIN'21}, title = {A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0}, year = 2021 }
%0 Conference Paper %1 ZuAgGrDoKo-INDIN2021-PdM %A Züfle, Marwin %A Agne, Joachim %A Grohmann, Johannes %A Dörtoluk, Ibrahim %A Kounev, Samuel %B Proceedings of the 21st IEEE IES International Conference on Industrial Informatics %D 2021 %I IEEE %T A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0 %U https://ieeexplore.ieee.org/abstract/document/9557387 %X Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.

SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications. Grohmann, Johannes; Straesser, Martin; Chalbani, Avi; Eismann, Simon; Arian, Yair; Herbst, Nikolas; Peretz, Noam; Kounev, Samuel; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021). ACM, New York, NY, USA.
Acceptance Rate: 29%
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.
@inproceedings{GrStChEiArHePeKo2021-ICPE, abstract = {Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing&mdash;a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90\%.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Straesser, Martin and Chalbani, Avi and Eismann, Simon and Arian, Yair and Herbst, Nikolas and Peretz, Noam and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {descartes performance prediction statistical_estimation_and_machine_learning t_full myown}, month = {April}, note = {Acceptance Rate: 29%}, organization = {ACM}, title = {SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications}, year = 2021 }
%0 Conference Paper %1 GrStChEiArHePeKo2021-ICPE %A Grohmann, Johannes %A Straesser, Martin %A Chalbani, Avi %A Eismann, Simon %A Arian, Yair %A Herbst, Nikolas %A Peretz, Noam %A Kounev, Samuel %B Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) %C New York, NY, USA %D 2021 %T SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications %U https://doi.org/10.1145/3427921.3450248 %X Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.

Libra: A Benchmark for Time Series Forecasting Methods. Bauer, André; Züfle, Marwin; Eismann, Simon; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021). ACM, New York, NY, USA.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.
@inproceedings{bauer2021benchmark, abstract = {In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.}, address = {New York, NY, USA}, author = {Bauer, Andr{é} and Z{ü}fle, Marwin and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {benchmark descartes forecasting metrics_and_benchmarking_methodologies t_full timeseries myown}, month = {April}, organization = {ACM}, title = {Libra: A Benchmark for Time Series Forecasting Methods}, year = 2021 }
%0 Conference Paper %1 bauer2021benchmark %A Bauer, André %A Züfle, Marwin %A Eismann, Simon %A Grohmann, Johannes %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) %C New York, NY, USA %D 2021 %T Libra: A Benchmark for Time Series Forecasting Methods %U https://dl.acm.org/doi/10.1145/3427921.3450241 %X In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.

2020 [ nach oben ]

Predicting the Costs of Serverless Workflows. Eismann, Simon; Grohmann, Johannes; van Eyk, Erwin; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) (2020). 265–276. Association for Computing Machinery (ACM), New York, NY, USA.
Acceptance Rate: 23.4% (15/64)
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.
@inproceedings{EiGrEyHeKo2020-ICPE-ServerlessWorkflows, abstract = {Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.}, address = {New York, NY, USA}, author = {Eismann, Simon and Grohmann, Johannes and van Eyk, Erwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {descartes se2 t_full myown}, month = {April}, note = {{Acceptance Rate: 23.4% (15/64)}}, pages = {265&ndash;276}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {Predicting the Costs of Serverless Workflows}, year = 2020 }
%0 Conference Paper %1 EiGrEyHeKo2020-ICPE-ServerlessWorkflows %A Eismann, Simon %A Grohmann, Johannes %A van Eyk, Erwin %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) %C New York, NY, USA %D 2020 %I Association for Computing Machinery (ACM) %P 265--276 %T Predicting the Costs of Serverless Workflows %U https://doi.org/10.1145/3358960.3379133 %X Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.

To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows. Züfle, Marwin; Krupitzer, Christian; Erhard, Florian; Grohmann, Johannes; Kounev, Samuel; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020). 19–36. Springer, Cham.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.
@inproceedings{ZuKrErGrKo-MMB2020-ToFailOrNotToFail, abstract = {Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.}, address = {Cham}, author = {Z{ü}fle, Marwin and Krupitzer, Christian and Erhard, Florian and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {Cyber-Physical_Systems Internet_of_Things PRISMA Performance Prediction Reliability Statistical_estimation_and_machine_learning descartes t_full myown}, month = {March}, pages = {19&ndash;36}, publisher = {Springer}, series = {MMB 2020}, title = {{To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows}}, year = 2020 }
%0 Conference Paper %1 ZuKrErGrKo-MMB2020-ToFailOrNotToFail %A Züfle, Marwin %A Krupitzer, Christian %A Erhard, Florian %A Grohmann, Johannes %A Kounev, Samuel %B Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems %C Cham %D 2020 %I Springer %P 19--36 %T To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows %U https://doi.org/10.1007/978-3-030-43024-5_2 %X Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.

Model-based Performance Predictions for SDN-based Networks: A Case Study. Herrnleben, Stefan; Rygielski, Piotr; Grohmann, Johannes; Eismann, Simon; Hossfeld, Tobias; Kounev, Samuel; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020). Springer, Cham.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.
@inproceedings{HeRyGrEiHoKo-MMB2020-Model-based-SDN-Performance, abstract = {Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Rygielski, Piotr and Grohmann, Johannes and Eismann, Simon and Hossfeld, Tobias and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {SDN descartes t_full myown}, month = {March}, publisher = {Springer}, series = {MMB 2020}, title = {{Model-based Performance Predictions for SDN-based Networks: A Case Study}}, year = 2020 }
%0 Conference Paper %1 HeRyGrEiHoKo-MMB2020-Model-based-SDN-Performance %A Herrnleben, Stefan %A Rygielski, Piotr %A Grohmann, Johannes %A Eismann, Simon %A Hossfeld, Tobias %A Kounev, Samuel %B Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems %C Cham %D 2020 %I Springer %T Model-based Performance Predictions for SDN-based Networks: A Case Study %U https://doi.org/10.1007/978-3-030-43024-5_6 %X Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.

An IoT Network Emulator for Analyzing the Influence of Varying Network Quality. Herrnleben, Stefan; Ailabouni, Rudy; Grohmann, Johannes; Prantl, Thomas; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) (2020).
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.
@inproceedings{HeAiGrPrKrKo2020-SIMUtools-IoT-Network-Emulator, abstract = {IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.}, author = {Herrnleben, Stefan and Ailabouni, Rudy and Grohmann, Johannes and Prantl, Thomas and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, keywords = {descartes internet_of_things myown networking t_full}, month = {August}, series = {SIMUtools 2020}, title = {An IoT Network Emulator for Analyzing the Influence of Varying Network Quality}, year = 2020 }
%0 Conference Paper %1 HeAiGrPrKrKo2020-SIMUtools-IoT-Network-Emulator %A Herrnleben, Stefan %A Ailabouni, Rudy %A Grohmann, Johannes %A Prantl, Thomas %A Krupitzer, Christian %A Kounev, Samuel %B Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) %D 2020 %T An IoT Network Emulator for Analyzing the Influence of Varying Network Quality %U https://doi.org/10.1007/978-3-030-72795-6_47 %X IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.

Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario. Prantl, Thomas; Ten, Peter; Iffländer, Lukas; Dmitrenko, Alexandra; Kounev, Samuel; Krupitzer, Christian; in 2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020).
Acceptance Rate: 27%
[ Abstract ] [ BibTeX ] [ EndNote ]
 
New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.
@inproceedings{PrTeIfDmKo2020-MASCOTS-GroupEncryption, abstract = {New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.}, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Group_Encryption Internet_of_Things t_full Security myown Performance_Evaluation descartes sss-group}, month = {November}, note = {Acceptance Rate: 27\%}, series = {MASCOTS '20}, title = {Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario}, year = 2020 }
%0 Conference Paper %1 PrTeIfDmKo2020-MASCOTS-GroupEncryption %A Prantl, Thomas %A Ten, Peter %A Iffländer, Lukas %A Dmitrenko, Alexandra %A Kounev, Samuel %A Krupitzer, Christian %B 2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) %D 2020 %T Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario %X New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.

Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS. Grohmann, Johannes; Seybold, Daniel; Eismann, Simon; Leznik, Mark; Kounev, Samuel; Domaschka, Jörg; in 2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020). 1–8. IEEE.
Acceptance Rate: 27%
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.
@inproceedings{GrSeEiLeKoDo2020-MASCOTS-DBMSPerformance, abstract = {Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.}, author = {Grohmann, Johannes and Seybold, Daniel and Eismann, Simon and Leznik, Mark and Kounev, Samuel and Domaschka, Jörg}, booktitle = {2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {automated_model_learning descartes myown performance performance_modeling prediction statistical_estimation_and_machine_learning t_full}, month = {November}, note = {Acceptance Rate: 27%}, pages = {1&ndash;8}, publisher = {IEEE}, series = {MASCOTS '20}, title = {Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS}, year = 2020 }
%0 Conference Paper %1 GrSeEiLeKoDo2020-MASCOTS-DBMSPerformance %A Grohmann, Johannes %A Seybold, Daniel %A Eismann, Simon %A Leznik, Mark %A Kounev, Samuel %A Domaschka, Jörg %B 2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) %D 2020 %I IEEE %P 1--8 %T Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS %U https://doi.org/10.1109/MASCOTS50786.2020.9285960 %X Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.

Quantifying measurement quality and load distribution in Tor. Greubel, Andre; Pohl, Steffen; Kounev, Samuel; in Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020) (2020).
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.
@article{greubel2020quantifying, abstract = {Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80\%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21\% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.}, author = {Greubel, Andre and Pohl, Steffen and Kounev, Samuel}, journal = {Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020)}, keywords = {TLBM t_full tor balancing descartes myown}, title = {Quantifying measurement quality and load distribution in Tor}, year = 2020 }
%0 Journal Article %1 greubel2020quantifying %A Greubel, Andre %A Pohl, Steffen %A Kounev, Samuel %D 2020 %J Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020) %T Quantifying measurement quality and load distribution in Tor %X Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.

2019 [ nach oben ]

Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning. Grohmann, Johannes; Nicholson, Patrick K.; Iglesias, Jesus Omana; Kounev, Samuel; Lugones, Diego; in Proceedings of the 20th International Middleware Conference (2019). 149–162. Association for Computing Machinery (ACM), New York, NY, USA.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics -- while still fulfilling application KPIs -- to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.
@inproceedings{GrNiIgKoLu-MIDDLEWARE2019-Monitorless, abstract = {Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics &ndash; while still fulfilling application KPIs &ndash; to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Nicholson, Patrick K. and Iglesias, Jesus Omana and Kounev, Samuel and Lugones, Diego}, booktitle = {Proceedings of the 20th International Middleware Conference}, keywords = {Application_quality_of_service_management Cloud Instrumentation_profiling_and_workload_characterization Multi-tenancy Performance Prediction Statistical_estimation_and_machine_learning Virtualization descartes t_full myown}, pages = {149&ndash;162}, publisher = {Association for Computing Machinery (ACM)}, series = {Middleware '19}, title = {Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning}, year = 2019 }
%0 Conference Paper %1 GrNiIgKoLu-MIDDLEWARE2019-Monitorless %A Grohmann, Johannes %A Nicholson, Patrick K. %A Iglesias, Jesus Omana %A Kounev, Samuel %A Lugones, Diego %B Proceedings of the 20th International Middleware Conference %C New York, NY, USA %D 2019 %I Association for Computing Machinery (ACM) %P 149--162 %T Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning %U https://doi.org/10.1145/3361525.3361543 %X Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics -- while still fulfilling application KPIs -- to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.

Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques. Grohmann, Johannes; Eismann, Simon; Elflein, Sven; Mazkatli, Manar; von Kistowski, Jóakim; Kounev, Samuel; in 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2019). 309–322. IEEE Computer Society.
Acceptance Rate: 23.8% (29/122)
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.
@inproceedings{GrEiElMaKiKo2019-MASCOTS-DependencyIdentification, abstract = {Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.}, author = {Grohmann, Johannes and Eismann, Simon and Elflein, Sven and Mazkatli, Manar and von Kistowski, J{ó}akim and Kounev, Samuel}, booktitle = {2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Automated_model_learning DML PRISMA Performance Prediction Self-aware-computing Statistical_estimation_and_machine_learning descartes t_full myown}, month = {October}, note = {{Acceptance Rate: 23.8% (29/122)}}, pages = {309&ndash;322}, publisher = {IEEE Computer Society}, series = {MASCOTS '19}, title = {{Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques}}, year = 2019 }
%0 Conference Paper %1 GrEiElMaKiKo2019-MASCOTS-DependencyIdentification %A Grohmann, Johannes %A Eismann, Simon %A Elflein, Sven %A Mazkatli, Manar %A von Kistowski, Jóakim %A Kounev, Samuel %B 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) %D 2019 %I IEEE Computer Society %P 309--322 %T Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques %U https://doi.org/10.1109/MASCOTS.2019.00042 %X Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.

Online Power Consumption Estimation for Functions in Cloud Applications. Schmitt, Norbert; Iffländer, Lukas; Bauer, André; Kounev, Samuel; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019). IEEE.
[ BibTeX ] [ EndNote ] [ Download ]
 
@inproceedings{ScIfBaKo2019-ICAC-OnlinePowerEstimation, author = {Schmitt, Norbert and Iffländer, Lukas and Bauer, André and Kounev, Samuel}, booktitle = {Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC)}, keywords = {Cloud Power Power-energy_efficient_computing descartes myown t_full}, month = {June}, publisher = {IEEE}, title = {Online Power Consumption Estimation for Functions in Cloud Applications}, year = 2019 }
%0 Conference Paper %1 ScIfBaKo2019-ICAC-OnlinePowerEstimation %A Schmitt, Norbert %A Iffländer, Lukas %A Bauer, André %A Kounev, Samuel %B Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) %D 2019 %I IEEE %T Online Power Consumption Estimation for Functions in Cloud Applications

Autonomic Forecasting Method Selection: Examination and Ways Ahead. Züfle, Marwin; Bauer, André; Lesch, Veronika; Krupitzer, Christian; Herbst, Nikolas; Kounev, Samuel; Curtef, Valentin; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019). IEEE.
[ BibTeX ] [ EndNote ] [ Download ]
 
@inproceedings{ZuBaLeKrHeKoCu2019-ICAC-Autonomic, author = {Z{ü}fle, Marwin and Bauer, Andr{é} and Lesch, Veronika and Krupitzer, Christian and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 16th {IEEE} International Conference on Autonomic Computing (ICAC)}, keywords = {Internet_of_Things Online_monitoring_and_forecasting PRISMA Performance Prediction Statistical_estimation_and_machine_learning descartes t_full myown}, month = {June}, publisher = {IEEE}, title = {{Autonomic Forecasting Method Selection: Examination and Ways Ahead}}, year = 2019 }
%0 Conference Paper %1 ZuBaLeKrHeKoCu2019-ICAC-Autonomic %A Züfle, Marwin %A Bauer, André %A Lesch, Veronika %A Krupitzer, Christian %A Herbst, Nikolas %A Kounev, Samuel %A Curtef, Valentin %B Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) %D 2019 %I IEEE %T Autonomic Forecasting Method Selection: Examination and Ways Ahead

Chamulteon: Coordinated Auto-Scaling of Micro-Services. Bauer, André; Lesch, Veronika; Versluis, Laurens; Ilyushkin, Alexey; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS) (2019).
[ BibTeX ] [ EndNote ] [ Download ]
 
@inproceedings{BaLeVeIlHeKo2019-ICDCS-Chamulteon, author = {Bauer, Andr{é} and Lesch, Veronika and Versluis, Laurens and Ilyushkin, Alexey and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS)}, keywords = {Chameleon Cloud Elasticity Metrics_and_benchmarking_methodologies Online_monitoring_and_forecasting Resource_management SPEC Self-aware-computing descartes t_full myown}, month = {July}, title = {Chamulteon: Coordinated Auto-Scaling of Micro-Services}, year = 2019 }
%0 Conference Paper %1 BaLeVeIlHeKo2019-ICDCS-Chamulteon %A Bauer, André %A Lesch, Veronika %A Versluis, Laurens %A Ilyushkin, Alexey %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS) %D 2019 %T Chamulteon: Coordinated Auto-Scaling of Micro-Services

Integrating Statistical Response Time Models in Architectural Performance Models. Eismann, Simon; Grohmann, Johannes; Walter, Jürgen; von Kistowski, Jóakim; Kounev, Samuel; in Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) (2019). 71–80. IEEE.
Acceptance Rate: 21,9% (21/96)
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.
@inproceedings{EiGrWaKiKo2019-ICSA-Integrating, abstract = {Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.}, author = {Eismann, Simon and Grohmann, Johannes and Walter, J{ü}rgen and von Kistowski, J{ó}akim and Kounev, Samuel}, booktitle = {Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA)}, keywords = {Analytical_and_simulation-based_analysis DML Formal_architecture_modeling Meta-models PRISMA Performance Prediction QPME QPN Simulation Statistical_estimation_and_machine_learning descartes t_full myown}, month = {March}, note = {Acceptance Rate: 21,9\% (21/96)}, pages = {71&ndash;80}, publisher = {IEEE}, title = {{Integrating Statistical Response Time Models in Architectural Performance Models}}, year = 2019 }
%0 Conference Paper %1 EiGrWaKiKo2019-ICSA-Integrating %A Eismann, Simon %A Grohmann, Johannes %A Walter, Jürgen %A von Kistowski, Jóakim %A Kounev, Samuel %B Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) %D 2019 %I IEEE %P 71--80 %T Integrating Statistical Response Time Models in Architectural Performance Models %U https://doi.org/10.1109/ICSA.2019.00016 %X Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.

Measuring the Energy Efficiency of Transactional Loads on GPGPU. von Kistowski, Jóakim; Pais, Johann; Wahl, Tobias; Lange, Klaus-Dieter; Block, Hansfried; Beckett, John; Kounev, Samuel; in Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering (2019). ACM, New York, NY, USA.
Best Industry Paper Award
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4% regarding power consumption.
@inproceedings{KiPaWaLaBlBeKo2019-ICPE-GPGPUPower, abstract = {{General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4\% regarding power consumption.}}, address = {New York, NY, USA}, author = {von Kistowski, J{ó}akim and Pais, Johann and Wahl, Tobias and Lange, Klaus-Dieter and Block, Hansfried and Beckett, John and Kounev, Samuel}, booktitle = {Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering}, keywords = {Award Cloud Metrics_and_benchmarking_methodologies Performance Power Power-energy_efficient_computing SPEC descartes t_full myown}, note = {{<b>Best Industry Paper Award</b>}}, publisher = {ACM}, series = {ICPE '19}, title = {{Measuring the Energy Efficiency of Transactional Loads on GPGPU}}, year = 2019 }
%0 Conference Paper %1 KiPaWaLaBlBeKo2019-ICPE-GPGPUPower %A von Kistowski, Jóakim %A Pais, Johann %A Wahl, Tobias %A Lange, Klaus-Dieter %A Block, Hansfried %A Beckett, John %A Kounev, Samuel %B Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2019 %I ACM %T Measuring the Energy Efficiency of Transactional Loads on GPGPU %X General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4% regarding power consumption.

Predicting Server Power Consumption from Standard Rating Results. von Kistowski, Jóakim; Grohmann, Johannes; Schmitt, Norbert; Kounev, Samuel; in Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering (2019). 301–312. Association for Computing Machinery (ACM), New York, NY, USA.
Full Paper Acceptance Rate: 18.6% (13/70)
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.
@inproceedings{KiGrScKo2019-ICPE-PowerPrediction, abstract = {Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.}, address = {New York, NY, USA}, author = {von Kistowski, J{ó}akim and Grohmann, Johannes and Schmitt, Norbert and Kounev, Samuel}, booktitle = {Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering}, keywords = {Metrics_and_benchmarking_methodologies PRISMA Power Prediction SPEC Statistical_estimation_and_machine_learning descartes t_full myown}, note = {{Full Paper Acceptance Rate: 18.6\% (13/70)}}, pages = {301&ndash;312}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '19}, title = {{Predicting Server Power Consumption from Standard Rating Results}}, year = 2019 }
%0 Conference Paper %1 KiGrScKo2019-ICPE-PowerPrediction %A von Kistowski, Jóakim %A Grohmann, Johannes %A Schmitt, Norbert %A Kounev, Samuel %B Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2019 %I Association for Computing Machinery (ACM) %P 301--312 %T Predicting Server Power Consumption from Standard Rating Results %U https://doi.org/10.1145/3297663.3310298 %X Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.

2018 [ nach oben ]

TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research. von Kistowski, Jóakim; Eismann, Simon; Schmitt, Norbert; Bauer, André; Grohmann, Johannes; Kounev, Samuel; in Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (2018). 223–236. IEEE Computer Society.
Acceptance Rate: 29.5% (23/78)
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.
@inproceedings{KiEiScBaGrKo2018-MASCOTS-TeaStore, abstract = {Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.}, author = {von Kistowski, J{ó}akim and Eismann, Simon and Schmitt, Norbert and Bauer, Andr{é} and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems}, keywords = {Cloud DML Design_of_software_and_systems Elasticity Metrics_and_benchmarking_methodologies Performance Power Prediction Resource_management Tool descartes t_full myown}, month = {September}, note = {{Acceptance Rate: 29.5\% (23/78)}}, pages = {223&ndash;236}, publisher = {IEEE Computer Society}, series = {MASCOTS '18}, title = {{TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research}}, year = 2018 }
%0 Conference Paper %1 KiEiScBaGrKo2018-MASCOTS-TeaStore %A von Kistowski, Jóakim %A Eismann, Simon %A Schmitt, Norbert %A Bauer, André %A Grohmann, Johannes %A Kounev, Samuel %B Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems %D 2018 %I IEEE Computer Society %P 223--236 %T TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research %U https://doi.org/10.1109/MASCOTS.2018.00030 %X Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.

SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network. Greubel, Andre; Dmitrienko, Alexandra; Kounev, Samuel; in Proceedings of the 34th Annual Computer Security Applications Conference (2018). ACM, New York, NY, USA.
[ BibTeX ] [ EndNote ] [ Download ]
 
@inproceedings{GrDmKo2018-ACSAC-SmarTor, address = {New York, NY, USA}, author = {Greubel, Andre and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {Proceedings of the 34th Annual Computer Security Applications Conference}, keywords = {Metrics_and_benchmarking_methodologies t_full Security descartes Networking sss-group myown}, month = {December}, organization = {ACM}, series = {ACSAC '18}, title = {{SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network}}, year = 2018 }
%0 Conference Paper %1 GrDmKo2018-ACSAC-SmarTor %A Greubel, Andre %A Dmitrienko, Alexandra %A Kounev, Samuel %B Proceedings of the 34th Annual Computer Security Applications Conference %C New York, NY, USA %D 2018 %T SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network

Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time. Eismann, Simon; Walter, Jürgen; von Kistowski, Jóakim; Kounev, Samuel; in 2018 IEEE International Conference on Software Architecture (ICSA) (2018). 135–144.
Acceptance Rate: 25,6% (22/86)
[ Abstract ] [ BibTeX ] [ EndNote ] [ Download ]
 
Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.
@inproceedings{EiWaKiKo2018-ParametricDependencies, abstract = {Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.}, author = {Eismann, Simon and Walter, J{ü}rgen and von Kistowski, J{ó}akim and Kounev, Samuel}, booktitle = {2018 IEEE International Conference on Software Architecture (ICSA)}, keywords = {PRISMA Performance Resource_management Meta-models descartes Formal_architecture_modeling t_full myown Prediction DML}, month = {April}, note = {Acceptance Rate: 25,6\% (22/86)}, pages = {135-144}, title = {{Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time}}, year = 2018 }
%0 Conference Paper %1 EiWaKiKo2018-ParametricDependencies %A Eismann, Simon %A Walter, Jürgen %A von Kistowski, Jóakim %A Kounev, Samuel %B 2018 IEEE International Conference on Software Architecture (ICSA) %D 2018 %P 135-144 %T Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time %X Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.

On the Value of Service Demand Estimation for Auto-Scaling. Bauer, André; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018) (2018). (Vol. 10740) 142–156. Springer, Cham.
[ Abstract ] [ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.
@inproceedings{BaGrHeKo2018-MMB-ServiceDemand, abstract = {In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.}, address = {Cham}, author = {Bauer, Andr{é} and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018)}, keywords = {Application-aware Application_quality_of_service_management Automated_model_learning BUNGEE Chameleon Cloud Elasticity GoogleResearchAward Instrumentation_profiling_and_workload_characterization LibReDE Metrics_and_benchmarking_methodologies PRISMA Prediction Resource_management Tool Virtualization descartes t_full myown}, month = {February}, pages = {142&ndash;156}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{On the Value of Service Demand Estimation for Auto-Scaling}}, volume = 10740, year = 2018 }
%0 Conference Paper %1 BaGrHeKo2018-MMB-ServiceDemand %A Bauer, André %A Grohmann, Johannes %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018) %C Cham %D 2018 %I Springer %P 142--156 %T On the Value of Service Demand Estimation for Auto-Scaling %U https://doi.org/10.1007/978-3-319-74947-1_10 %V 10740 %X In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.

FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds. Lesch, Veronika; Bauer, André; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) (2018). ACM, New York, NY, USA.
Full Paper Acceptance Ratio: 23,7%
[ BibTeX ] [ EndNote ] [ URL ] [ Download ]
 
@inproceedings{LeBaHeKo-ICPE18-FOX-AS-COST, address = {New York, NY, USA}, author = {Lesch, Veronika and Bauer, Andr{é} and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018)}, keywords = {Online_monitoring_and_forecasting Optimization Performance Resource_management descartes Elasticity Cloud t_full BUNGEE myown Self-aware-computing Tool Prediction}, month = {April}, note = {Full Paper Acceptance Ratio: 23,7%}, publisher = {ACM}, title = {{FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds}}, year = 2018 }
%0 Conference Paper %1 LeBaHeKo-ICPE18-FOX-AS-COST %A Lesch, Veronika %A Bauer, André %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) %C New York, NY, USA %D 2018 %I ACM %T FOX: Cost-Awareness for Autonomic Resource Management in