Publications

Peer Reviewed Journal and Magazine Articles

Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis. Stenger, Michael; Bauer, Andr{é}; Prantl, Thomas; Leppich, Robert; Hudson, Nathaniel; Chard, Kyle; Foster, Ian; Kounev, Samuel; in J. Data and Information Quality (2024). 16(2) Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.

@article{10.1145/3666006, abstract = {Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.}, address = {New York, NY, USA}, author = {Stenger, Michael and Bauer, Andr\'{e} and Prantl, Thomas and Leppich, Robert and Hudson, Nathaniel and Chard, Kyle and Foster, Ian and Kounev, Samuel}, journal = {J. Data and Information Quality}, keywords = {t_journalmagazine}, month = {06}, number = 2, publisher = {Association for Computing Machinery}, title = {Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis}, volume = 16, year = 2024 }
Benchmarking of Secure Group Communication schemes with Focus on IoT. Prantl, Thomas; Bauer, André; Engel, Simon; Horn, Lukas; Krupitzer, Christian; Iffl"{a}nder, Lukas; Kounev, Samuel; in Discover Data (2024). 2(1) Springer Science and Business Media LLC.
- [ BibTeX ]
- [ URL ]
@article{PrBaEnHoKrIfKo_Discover_Data_2024, author = {Prantl, Thomas and Bauer, André and Engel, Simon and Horn, Lukas and Krupitzer, Christian and Iffl"{a}nder, Lukas and Kounev, Samuel}, journal = {Discover Data}, keywords = {t_journalmagazine}, month = {05}, number = 1, publisher = {Springer Science and Business Media LLC}, title = {Benchmarking of Secure Group Communication schemes with Focus on IoT}, volume = 2, year = 2024 }
Evaluation is key: a survey on evaluation measures for synthetic time series. Stenger, Michael; Leppich, Robert; Foster, Ian; Kounev, Samuel; Bauer, Andr{é}; in Journal of Big Data (2024). 11 66.
- [ BibTeX ]
@article{Stenger2024, author = {Stenger, Michael and Leppich, Robert and Foster, Ian and Kounev, Samuel and Bauer, Andr{\'e}}, journal = {Journal of Big Data}, keywords = {t_journalmagazine}, month = {05}, pages = 66, title = {Evaluation is key: a survey on evaluation measures for synthetic time series}, volume = 11, year = 2024 }
Comprehensive Exploration of Synthetic Data Generation: A Survey. Bauer, André; and Trapp, Simon; and Stenger, Michael; and Leppich, Robert; and Kounev, Samuel; and Leznik, Mark; and Chard, Kyle; and Foster, Ian; in arXiv preprint (2024). 103.

https://doi.org/10.48550/arXiv.2401.02524
- [ BibTeX ]
@article{bauer2024comprehensive, author = {Bauer, André and and Trapp, Simon and and Stenger, Michael and and Leppich, Robert and and Kounev, Samuel and and Leznik, Mark and and Chard, Kyle and and Foster, Ian}, journal = {arXiv preprint}, keywords = {t_journalmagazine}, note = {https://doi.org/10.48550/arXiv.2401.02524}, pages = 103, title = {Comprehensive Exploration of Synthetic Data Generation: A Survey}, year = 2024 }
De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem. Prantl, Thomas; Horn, Lukas; Engel, Simon; Iffländer, Lukas; Beierlieb, Lukas; Krupitzer, Christian; Bauer, André; Sakarvadia, Mansi; Foster, Ian; Kounev, Samuel; in International Journal of Information Security (2023).
- [ BibTeX ]
- [ DOI ]
@article{PrHoEnIfBeKrBaSaFoKo_IJoIS_SGC_Pareto, author = {Prantl, Thomas and Horn, Lukas and Engel, Simon and Iffl\"ander, Lukas and Beierlieb, Lukas and Krupitzer, Christian and Bauer, Andr\'e and Sakarvadia, Mansi and Foster, Ian and Kounev, Samuel}, journal = {International Journal of Information Security}, keywords = {t_journalmagazine}, month = 11, title = {De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem}, year = 2023 }
Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption. Prantl, Thomas; Zeck, Timo; Horn, Lukas; Iffländer, Lukas; Bauer, André; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in Journal of Surveillance, Security and Safety (2023).
- [ BibTeX ]
- [ DOI ]
@article{PrZeHoIfBaDmKrKo_ABESurvey, author = {Prantl, Thomas and Zeck, Timo and Horn, Lukas and Iffl\"ander, Lukas and Bauer, Andr\'e and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, journal = {Journal of Surveillance, Security and Safety}, keywords = {t_journalmagazine}, title = {Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption}, year = 2023 }
Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field. Bauer, André; Leznik, Mark; Stenger, Michael; Leppich, Robert; Herbst, Nikolas; Kounev, Samuel; Foster, Ian; (2023).
- [ BibTeX ]
- [ Download ]
@misc{bauer2023telescope, author = {Bauer, André and Leznik, Mark and Stenger, Michael and Leppich, Robert and Herbst, Nikolas and Kounev, Samuel and Foster, Ian}, keywords = {t_journalmagazine}, title = {Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field}, year = 2023 }
Recommendation of Secure Group Communication Schemes using Multi Objective Optimization. Prantl, Thomas; Bauer, André; Iffländer, Lukas; Krupitzer, Christian; Kounev, Samuel; in International Journal of Information Security (2023).
- [ BibTeX ]
@article{PrBaIfKrKo_IJoIS_SGC_Pareto, author = {Prantl, Thomas and Bauer, André and Iffländer, Lukas and Krupitzer, Christian and Kounev, Samuel}, journal = {International Journal of Information Security}, keywords = {t_journalmagazine}, title = {Recommendation of Secure Group Communication Schemes using Multi Objective Optimization}, year = 2023 }
A literature review of IoT and CPS—What they are, and what they are not. Lesch, Veronika; Z{ü}fle, Marwin; Bauer, Andr{é}; Iffl{ä}nder, Lukas; Krupitzer, Christian; Kounev, Samuel; in Journal of Systems and Software (2023). 111631. Elsevier.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.

@article{lesch2023literature, abstract = {Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.}, author = {Lesch, Veronika and Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Iffl{\"a}nder, Lukas and Krupitzer, Christian and Kounev, Samuel}, journal = {Journal of Systems and Software}, keywords = {t_journalmagazine}, pages = 111631, publisher = {Elsevier}, title = {A literature review of IoT and CPS—What they are, and what they are not}, year = 2023 }
A Survey on Secure Group Communication Schemes with Focus on IoT Communication. Prantl, Thomas; Zeck, Timo; Bauer, André; Ten, Peter; Prantl, Dominik; Ben Yahya, Ala Eddine; Iffländer, Lukas; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in IEEE Access (2022). IEEE.

Accepted but not yet published
- [ BibTeX ]
@article{PrZeBaTePrYaIfDmKrKo_IEEEAccess_SGCSurvey, author = {Prantl, Thomas and Zeck, Timo and Bauer, André and Ten, Peter and Prantl, Dominik and Ben Yahya, Ala Eddine and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, journal = {IEEE Access}, keywords = {t_journalmagazine}, note = {Accepted but not yet published}, publisher = {IEEE}, title = {A Survey on Secure Group Communication Schemes with Focus on IoT Communication}, year = 2022 }
Towards Forecasting Future Snow Cover Dynamics in the European Alps - The Potential of Long Optical Remote-Sensing Time Series. Koehler, Jonas; Bauer, André; Dietz, Andreas J.; Kuenzer, Claudia; in Remote Sensing (2022). 14(18)
- [ BibTeX ]
- [ URL ]
@article{rs14184461, author = {Koehler, Jonas and Bauer, André and Dietz, Andreas J. and Kuenzer, Claudia}, journal = {Remote Sensing}, keywords = {t_journalmagazine}, number = 18, title = {Towards Forecasting Future Snow Cover Dynamics in the European Alps - The Potential of Long Optical Remote-Sensing Time Series}, volume = 14, year = 2022 }
Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery. Kalisnik, Jurij Matija; Bauer, Andr{é}; Vogt, Ferdinand Aurel; Stickl, Franziska Josephine; Zibert, Janez; Fittkau, Matthias; Bertsch, Thomas; Kounev, Samuel; Fischlein, Theodor; in European Journal of Cardio-Thoracic Surgery (2022).

Joint first authorship; To be published soon
- [ Abstract ]
- [ BibTeX ]
Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.

@article{kalisnik2022artificial, abstract = {Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.}, author = {Kalisnik, Jurij Matija and Bauer, Andr{é} and Vogt, Ferdinand Aurel and Stickl, Franziska Josephine and Zibert, Janez and Fittkau, Matthias and Bertsch, Thomas and Kounev, Samuel and Fischlein, Theodor}, journal = {European Journal of Cardio-Thoracic Surgery}, keywords = {t_journalmagazine}, note = {Joint first authorship; To be published soon}, title = {Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery}, year = 2022 }
SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation. Grohmann, Johannes; Eismann, Simon; Bauer, Andr{é}; Spinner, Simon; Blum, Johannes; Herbst, Nikolas; Kounev, Samuel; in ACM Transactions on Autonomous and Adaptive Systems (2021). 15(2) Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

@article{GrEiBaSpBlHeKo2021-TaaS-SARDE, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andr\'{e} and Spinner, Simon and Blum, Johannes and Herbst, Nikolas and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {t_journalmagazine}, month = {06}, number = 2, publisher = {Association for Computing Machinery}, title = {SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation}, volume = 15, year = 2021 }
{Time Series Forecasting for Self-Aware Systems}. Bauer, Andr{é}; Z{ü}fle, Marwin; Herbst, Nikolas; Zehe, Albin; Hotho, Andreas; Kounev, Samuel; in Proceedings of the IEEE (2020). 108(7) 1068–1093.
- [ BibTeX ]
- [ DOI ]
- [ Download ]
@article{BaZuHeZeHoKo-PIEEE-Forecasting, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Zehe, Albin and Hotho, Andreas and Kounev, Samuel}, journal = {Proceedings of the IEEE}, keywords = {t_journalmagazine}, month = {07}, number = 7, pages = {1068 - 1093}, title = {{Time Series Forecasting for Self-Aware Systems}}, volume = 108, year = 2020 }
The SPEC-RG Reference Architecture for FaaS: From Microservices and Containers to Serverless Platforms. van Eyk, Erwin; Grohmann, Johannes; Eismann, Simon; Bauer, Andr{é}; Versluis, Laurens; Toader, Lucian; Schmitt, Norbert; Herbst, Nikolas; Abad, Cristina L.; Iosup, Alexandru; in IEEE Internet Computing (2019). 23(6) 7–18. IEEE.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Microservices, containers, and serverless computing belong to a trend toward applications composed of many small, self-contained, and automatically managed components. Core to serverless computing, Function-as-a-Service (FaaS) platforms employ state-of-the-art container technology and microservices-based architectures to enable users to manage complex applications without the need for systems-level expertise. Victim of its own success, and partially due to proprietary technology, currently the community has a limited overview of these platforms. To address this, we propose a reference architecture and ecosystem for FaaS platforms. Based on a year-long survey of real-world platforms conducted within the SPEC-RG Cloud Group, we highlight specific components and identify common operational patterns.

@article{vEGrEiBaVeToScHeAbIo-IC-FaaS, abstract = {Microservices, containers, and serverless computing belong to a trend toward applications composed of many small, self-contained, and automatically managed components. Core to serverless computing, Function-as-a-Service (FaaS) platforms employ state-of-the-art container technology and microservices-based architectures to enable users to manage complex applications without the need for systems-level expertise. Victim of its own success, and partially due to proprietary technology, currently the community has a limited overview of these platforms. To address this, we propose a reference architecture and ecosystem for FaaS platforms. Based on a year-long survey of real-world platforms conducted within the SPEC-RG Cloud Group, we highlight specific components and identify common operational patterns.}, author = {van Eyk, Erwin and Grohmann, Johannes and Eismann, Simon and Bauer, Andr{\'e} and Versluis, Laurens and Toader, Lucian and Schmitt, Norbert and Herbst, Nikolas and Abad, Cristina L. and Iosup, Alexandru}, journal = {IEEE Internet Computing}, keywords = {SPEC}, month = 11, number = 6, pages = {7–18}, publisher = {IEEE}, title = {The SPEC-RG Reference Architecture for FaaS: From Microservices and Containers to Serverless Platforms}, volume = 23, year = 2019 }
Methodological Principles for Reproducible Performance Evaluation in Cloud Computing. Papadopoulos, Alessandro Vittorio; Versluis, Laurens; Bauer, Andr{é}; Herbst, Nikolas; von Kistowski, J{ó}akim; Ali-Eldin, Ahmed; Abad, Cristina; Amaral, J. Nelson; Tuma, Petr; Iosup, Alexandru; in IEEE Transactions on Software Engineering (2019). 47(8) 1528–1543.
- [ BibTeX ]
- [ DOI ]
- [ Download ]
@article{PaVeBaHeKiAEAbAmTuIo2019-TSE-MethodologicalPrinciples, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{\'e} and Herbst, Nikolas and von Kistowski, J{\'o}akim and Ali-Eldin, Ahmed and Abad, Cristina and Amaral, J. Nelson and Tuma, Petr and Iosup, Alexandru}, journal = {IEEE Transactions on Software Engineering}, keywords = {SPEC}, month = {07}, number = 8, pages = {1528-1543}, title = {Methodological Principles for Reproducible Performance Evaluation in Cloud Computing}, volume = 47, year = 2019 }
{Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud}. Metzger, Florian; Hoßfeld, Tobias; Bauer, Andr{é}; Kounev, Samuel; Heegaard, Poul. E.; in Proceedings of the IEEE (2019). 107(4) 679–694.
- [ BibTeX ]
- [ Download ]
@article{MeHoBaKoHe2019-PIEEE-IoTTraffic, author = {Metzger, Florian and Ho{\ss}feld, Tobias and Bauer, Andr{\'e} and Kounev, Samuel and Heegaard, Poul. E.}, journal = {Proceedings of the IEEE}, keywords = {t_journalmagazine}, month = {04}, number = 4, pages = {679-694}, title = {{Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud}}, volume = 107, year = 2019 }
{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}. Bauer, Andr{é}; Herbst, Nikolas; Spinner, Simon; Ali-Eldin, Ahmed; Kounev, Samuel; in IEEE Transactions on Parallel and Distributed Systems (2019). 30(4) 800–813. IEEE.
- [ BibTeX ]
- [ Download ]
@article{BaHeSpAEKo2019-TPDS-ChameleonAutoscaler, author = {Bauer, Andr{\'e} and Herbst, Nikolas and Spinner, Simon and Ali-Eldin, Ahmed and Kounev, Samuel}, journal = {IEEE Transactions on Parallel and Distributed Systems}, keywords = {Self-adaptive-systems}, month = {04}, number = 4, pages = {800–813}, publisher = {IEEE}, title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}}, volume = 30, year = 2019 }
{Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges}. Herbst, Nikolas; Bauer, Andr{é}; Kounev, Samuel; Oikonomou, Giorgos; van Eyk, Erwin; Kousiouris, George; Evangelinou, Athanasia; Krebs, Rouven; Brecht, Tim; Abad, Cristina L.; Iosup, Alexandru; in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) (2018). 3(4) 19:1–19:36. ACM, New York, NY, USA.
- [ BibTeX ]
- [ URL ]
- [ Download ]
@article{HeBaKoOiEyKoEvKrBrAbIo2018-TOMPECS-CloudMetrics, address = {New York, NY, USA}, author = {Herbst, Nikolas and Bauer, Andr{\'e} and Kounev, Samuel and Oikonomou, Giorgos and van Eyk, Erwin and Kousiouris, George and Evangelinou, Athanasia and Krebs, Rouven and Brecht, Tim and Abad, Cristina L. and Iosup, Alexandru}, journal = {ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)}, keywords = {SPEC}, month = {08}, number = 4, pages = {19:1–19:36}, publisher = {ACM}, title = {{Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges}}, volume = 3, year = 2018 }
An Experimental Performance Evaluation of Autoscalers for Complex Workflows. Ilyushkin, Alexey; Ali-Eldin, Ahmed; Herbst, Nikolas; Bauer, Andr{é}; Papadopoulos, Alessandro V.; Epema, Dick; Iosup, Alexandru; in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) (2018). 3(2) 8:1–8:32. ACM, New York, NY, USA.
- [ BibTeX ]
- [ URL ]
- [ Download ]
@article{IlAEHeBaPaEpIo2018-TOMPECS-Workflow, address = {New York, NY, USA}, author = {Ilyushkin, Alexey and Ali-Eldin, Ahmed and Herbst, Nikolas and Bauer, Andr{\'e} and Papadopoulos, Alessandro V. and Epema, Dick and Iosup, Alexandru}, journal = {ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)}, keywords = {SPEC}, month = {04}, number = 2, pages = {8:1–8:32}, publisher = {ACM}, title = {An Experimental Performance Evaluation of Autoscalers for Complex Workflows}, volume = 3, year = 2018 }

Peer Reviewed Full Conference Papers

Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme. Prantl, Thomas; Lauer, Marco; Horn, Lukas; Engel, Simon; Dingel, David; Kounev, Samuel; Bauer, André; Krupitzer, Christian; in Proceedings of the 19th International Conference on Availability, Reliability and Security (2024). ACM.
- [ BibTeX ]
@inproceedings{ARES_2024_ABE, author = {Prantl, Thomas and Lauer, Marco and Horn, Lukas and Engel, Simon and Dingel, David and Kounev, Samuel and Bauer, André and Krupitzer, Christian}, booktitle = {Proceedings of the 19th International Conference on Availability, Reliability and Security}, keywords = {descartes}, publisher = {ACM}, series = {ARES 2024}, title = {Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme}, year = 2024 }
An Empirical Investigation of Container Building Strategies and Warm Times to Reduce Cold Starts in Scientific Computing Serverless Functions. Bauer, André; Gonthier, Maxime; Chard, Ryan; Pan, Haochen; Grzenda, Daniel; Straesser, Martin; Pauloski, J. Gregory; Kamatar, Alok; Baughman, Matthew E.; Hudson, Nathaniel; Foster, Ian; Chard, Kyle; in 2024 IEEE 20th International Conference on e-Science (e-Science) (2024).
- [ BibTeX ]
@inproceedings{bauer2024coldstarts, author = {Bauer, André and Gonthier, Maxime and Chard, Ryan and Pan, Haochen and Grzenda, Daniel and Straesser, Martin and Pauloski, J. Gregory and Kamatar, Alok and Baughman, Matthew E. and Hudson, Nathaniel and Foster, Ian and Chard, Kyle}, booktitle = {2024 IEEE 20th International Conference on e-Science (e-Science)}, keywords = {Scientific_Computing}, title = {An Empirical Investigation of Container Building Strategies and Warm Times to Reduce Cold Starts in Scientific Computing Serverless Functions}, year = 2024 }
An Empirical Study of Container Image Configurations and Their Impact on Start Times. Straesser, Martin; Bauer, André; Leppich, Robert; Herbst, Nikolas; Chard, Kyle; Foster, Ian; Kounev, Samuel; in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid) (2023). 94–105. IEEE.
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
@inproceedings{straesser2023starttime, author = {Straesser, Martin and Bauer, André and Leppich, Robert and Herbst, Nikolas and Chard, Kyle and Foster, Ian and Kounev, Samuel}, booktitle = {2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)}, keywords = {empirical_study}, pages = {94-105}, publisher = {IEEE}, title = {An Empirical Study of Container Image Configurations and Their Impact on Start Times}, year = 2023 }
Autoscaler Evaluation and Configuration: A Practitioner’s Guideline. Straesser, Martin; Eismann, Simon; von Kistowski, J{ó}akim; Bauer, Andr{é}; Kounev, Samuel; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 31–41. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.

@inproceedings{straesser2023guideline, abstract = {Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.}, address = {New York, NY, USA}, author = {Straesser, Martin and Eismann, Simon and von Kistowski, J\'{o}akim and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {methodology}, pages = {31-41}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {Autoscaler Evaluation and Configuration: A Practitioner's Guideline}, year = 2023 }
A Systematic Approach for Benchmarking of Container Orchestration Frameworks. Straesser, Martin; Mathiasch, Jonas; Bauer, Andr{é}; Kounev, Samuel; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 187–198. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.

@inproceedings{straesser2023orchestration, abstract = {Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.}, address = {New York, NY, USA}, author = {Straesser, Martin and Mathiasch, Jonas and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {container_orchestration}, pages = {187-198}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {A Systematic Approach for Benchmarking of Container Orchestration Frameworks}, year = 2023 }
Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling. Straesser, Martin; Grohmann, Johannes; von Kistowski, J{ó}akim; Eismann, Simon; Bauer, Andr{é}; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 105–115. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.

@inproceedings{10.1145/3489525.3511680, abstract = {Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.}, address = {New York, NY, USA}, author = {Straesser, Martin and Grohmann, Johannes and von Kistowski, J\'{o}akim and Eismann, Simon and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {microservices}, pages = {105–115}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling}, year = 2022 }
Libra: A Benchmark for Time Series Forecasting Methods. Bauer, Andr{é}; Z{ü}fle, Marwin; Eismann, Simon; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021). ACM, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.

@inproceedings{bauer2021benchmark, abstract = {In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.}, address = {New York, NY, USA}, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {benchmark}, month = {04}, organization = {ACM}, title = {Libra: A Benchmark for Time Series Forecasting Methods}, year = 2021 }
Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT. Prantl, Thomas; Prantl, Dominik; Bauer, André; Iffländer, Lukas; Dmitrenko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
- [ BibTeX ]
@inproceedings{PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption, author = {Prantl, Thomas and Prantl, Dominik and Bauer, André and Iffländer, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {sss-group}, title = {Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT}, year = 2021 }
Kaa: Evaluating Elasticity of Cloud-hosted DBMS. Seybold, Daniel; Volpert, Simon; Wesner, Stefan; Bauer, Andr{é}; Herbst, Nikolas; Domaschka, J{ö}rg; in Proceedings of the 11th IEEE International Conference on Cloud Computing (CloudCom) (2019).
- [ BibTeX ]
- [ Download ]
@inproceedings{SeVoWeBaHeDo-CloudCom-Kaa, author = {Seybold, Daniel and Volpert, Simon and Wesner, Stefan and Bauer, Andr{\'e} and Herbst, Nikolas and Domaschka, J{\"o}rg}, booktitle = {Proceedings of the 11th IEEE International Conference on Cloud Computing (CloudCom)}, keywords = {Elasticity}, month = 12, title = {Kaa: Evaluating Elasticity of Cloud-hosted DBMS}, year = 2019 }
Chamulteon: Coordinated Auto-Scaling of Micro-Services. Bauer, Andr{é}; Lesch, Veronika; Versluis, Laurens; Ilyushkin, Alexey; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS) (2019).
- [ BibTeX ]
- [ Download ]
@inproceedings{BaLeVeIlHeKo2019-ICDCS-Chamulteon, author = {Bauer, Andr{\'e} and Lesch, Veronika and Versluis, Laurens and Ilyushkin, Alexey and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS)}, keywords = {SPEC}, month = {07}, title = {Chamulteon: Coordinated Auto-Scaling of Micro-Services}, year = 2019 }
Online Power Consumption Estimation for Functions in Cloud Applications. Schmitt, Norbert; Iffländer, Lukas; Bauer, André; Kounev, Samuel; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019). IEEE.
- [ BibTeX ]
- [ Download ]
@inproceedings{ScIfBaKo2019-ICAC-OnlinePowerEstimation, author = {Schmitt, Norbert and Iffländer, Lukas and Bauer, André and Kounev, Samuel}, booktitle = {Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC)}, keywords = {Power}, month = {06}, publisher = {IEEE}, title = {Online Power Consumption Estimation for Functions in Cloud Applications}, year = 2019 }
{Autonomic Forecasting Method Selection: Examination and Ways Ahead}. Z{ü}fle, Marwin; Bauer, Andr{é}; Lesch, Veronika; Krupitzer, Christian; Herbst, Nikolas; Kounev, Samuel; Curtef, Valentin; in Proceedings of the 16th {IEEE} International Conference on Autonomic Computing (ICAC) (2019). IEEE.
- [ BibTeX ]
- [ Download ]
@inproceedings{ZuBaLeKrHeKoCu2019-ICAC-Autonomic, author = {Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Lesch, Veronika and Krupitzer, Christian and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 16th {IEEE} International Conference on Autonomic Computing (ICAC)}, keywords = {PRISMA}, month = {06}, publisher = {IEEE}, title = {{Autonomic Forecasting Method Selection: Examination and Ways Ahead}}, year = 2019 }
{TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research}. von Kistowski, J{ó}akim; Eismann, Simon; Schmitt, Norbert; Bauer, Andr{é}; Grohmann, Johannes; Kounev, Samuel; in Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (2018). 223–236. IEEE Computer Society.

{Acceptance Rate: 29.5\% (23/78)}
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.

@inproceedings{KiEiScBaGrKo2018-MASCOTS-TeaStore, abstract = {Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.}, author = {von Kistowski, J{\'o}akim and Eismann, Simon and Schmitt, Norbert and Bauer, Andr{\'e} and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems}, keywords = {Power}, month = {09}, note = {{Acceptance Rate: 29.5\% (23/78)}}, pages = {223–236}, publisher = {IEEE Computer Society}, series = {MASCOTS '18}, title = {{TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research}}, year = 2018 }
{FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds}. Lesch, Veronika; Bauer, Andr{é}; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) (2018). ACM, New York, NY, USA.

Full Paper Acceptance Ratio: 23,7%
- [ BibTeX ]
- [ URL ]
- [ Download ]
@inproceedings{LeBaHeKo-ICPE18-FOX-AS-COST, address = {New York, NY, USA}, author = {Lesch, Veronika and Bauer, Andr{\'e} and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018)}, keywords = {BUNGEE}, month = {04}, note = {Full Paper Acceptance Ratio: 23,7%}, publisher = {ACM}, title = {{FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds}}, year = 2018 }
{On the Value of Service Demand Estimation for Auto-Scaling}. Bauer, Andr{é}; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018) (2018). (Vol. 10740) 142–156. Springer, Cham.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.

@inproceedings{BaGrHeKo2018-MMB-ServiceDemand, abstract = {In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.}, address = {Cham}, author = {Bauer, Andr{\'e} and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018)}, keywords = {Virtualization}, month = {02}, pages = {142–156}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{On the Value of Service Demand Estimation for Auto-Scaling}}, volume = 10740, year = 2018 }

Peer Reviewed Short Conference Papers

Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis. Bauer, André; Dittus, Timo; Straesser, Martin; Kamatar, Alok; Baughman, Matt; Beierlieb, Lukas; Hadry, Marius; Grillmeyer, Daniel; Lubas, Yannik; Kounev, Samuel; Foster, Ian; Chard, Kyle; in Companion of the 15th ACM/SPEC International Conference on Performance Engineering (2024). 72–76. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

@inproceedings{10.1145/3629527.3651843, abstract = {As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.}, address = {New York, NY, USA}, author = {Bauer, André and Dittus, Timo and Straesser, Martin and Kamatar, Alok and Baughman, Matt and Beierlieb, Lukas and Hadry, Marius and Grillmeyer, Daniel and Lubas, Yannik and Kounev, Samuel and Foster, Ian and Chard, Kyle}, booktitle = {Companion of the 15th ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {72–76}, publisher = {Association for Computing Machinery}, series = {ICPE '24 Companion}, title = {Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis}, year = 2024 }
Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example. Prantl, Thomas; Engel, Simon; Horn, Lukas; Kaiser, Dennis; Iffländer, Lukas; Bauer, André; Krupitzer, Christian; Kounev, Samuel; in Information Security Practice and Experience, W. Meng, Z. Yan, V. Piuri (eds.) (2023). 284–298. Springer Nature Singapore, Singapore.
- [ Abstract ]
- [ BibTeX ]
In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority.

@inproceedings{PrEnHoDeIfBaKrKo_ISPEC_De_Bello, abstract = {In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority.}, address = {Singapore}, author = {Prantl, Thomas and Engel, Simon and Horn, Lukas and Kaiser, Dennis and Iffl\"ander, Lukas and Bauer, Andr\'e and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Information Security Practice and Experience}, editor = {Meng, Weizhi and Yan, Zheng and Piuri, Vincenzo}, keywords = {descartes}, pages = {284–298}, publisher = {Springer Nature Singapore}, title = {Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example}, year = 2023 }
Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs. Bauer, Andr{é}; Straesser, Martin; Leznik, Mark; Beierlieb, Lukas; Hadry, Marius; Hudson, Nathaniel; Chard, Kyle; Kounev, Samuel; Foster, Ian; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 95–99. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.

@inproceedings{10.1145/3578245.3584693, abstract = {Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.}, address = {New York, NY, USA}, author = {Bauer, Andr\'{e} and Straesser, Martin and Leznik, Mark and Beierlieb, Lukas and Hadry, Marius and Hudson, Nathaniel and Chard, Kyle and Kounev, Samuel and Foster, Ian}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {distance_measures}, pages = {95–99}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs}, year = 2023 }
Change Point Detection for MongoDB Time Series Performance Regression. Leznik, Mark; Iqbal, Md Shahriar; Trubin, Igor A.; Lochner, Arne; Jamshidi, Pooyan; Bauer, André; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). ACM.
- [ BibTeX ]
@conference{leznik2022change, author = {Leznik, Mark and Iqbal, Md Shahriar and Trubin, Igor A. and Lochner, Arne and Jamshidi, Pooyan and Bauer, André}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {performance}, month = {04}, publisher = {ACM}, title = {Change Point Detection for MongoDB Time Series Performance Regression}, year = 2022 }
Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning. Bauer, André; Straesser, Martin; Beierlieb, Lukas; Meissner, Maximilian; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). ACM.
- [ BibTeX ]
- [ Download ]
@conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {machine_learning}, month = {04}, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }
Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity. Leznik, Mark; Grohmann, Johannes; Kliche, Nina; Bauer, Andr{é}; Seybold, Daniel; Eismann, Simon; Kounev, Samuel; Domaschka, J{ö}rg; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 89–96. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.

@inproceedings{leznik2022dissimilar, abstract = {Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.}, address = {New York, NY, USA}, author = {Leznik, Mark and Grohmann, Johannes and Kliche, Nina and Bauer, Andr\'{e} and Seybold, Daniel and Eismann, Simon and Kounev, Samuel and Domaschka, J\"{o}rg}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {analysis}, pages = {89–96}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity}, year = 2022 }
Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field. Bauer, Andr{é}; Z{ü}fle, Marwin; Herbst, Nikolas; Kounev, Samuel; Curtef, Valentin; in Proceedings of the 36th International Conference on Data Engineering (ICDE) (2020).
- [ BibTeX ]
- [ Download ]
@inproceedings{BaZuHeKoCu-ICDE-Telescope, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 36th International Conference on Data Engineering (ICDE)}, keywords = {Prediction}, month = {04}, title = {Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field}, year = 2020 }
{An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series}. Bauer, Andr{é}; Z{ü}fle, Marwin; Grohmann, Johannes; Schmitt, Norbert; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2020). 48–55. Association for Computing Machinery (ACM), New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.

@inproceedings{BaZuGrScHeKo-ICPE20-Seasonal-Forecast, abstract = {Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.}, address = {New York, NY, USA}, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Grohmann, Johannes and Schmitt, Norbert and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {PRISMA}, month = {04}, pages = {48–55}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {{An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series}}, year = 2020 }

PhD Thesis

Automated Hybrid Time Series Forecasting: Design, Benchmarking, and Use Cases. Bauer, Andr{é}; (2021). Universit{ä}t W{ü}rzburg.
- [ BibTeX ]
@phdthesis{Bauer2021-phdthesis, author = {Bauer, Andr{\'e}}, keywords = {descartes}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Automated Hybrid Time Series Forecasting: Design, Benchmarking, and Use Cases}, type = {doctoralthesis}, year = 2021 }
Automated Hybrid Time Series Forecasting: Design, Benchmarking, and Use Cases. Bauer, Andr{é}; (2020, December). University of W{ü}rzburg, Germany.

{Distinction, SPEC Kaivalya Dixit Distinguished Dissertation Award 2021}
- [ BibTeX ]
- [ DOI ]
- [ Download ]
@phdthesis{Bauer2020-phd, author = {Bauer, Andr{\'e}}, keywords = {telescope}, month = 12, note = {<b>{Distinction, SPEC Kaivalya Dixit Distinguished Dissertation Award 2021}</b>}, school = {University of W{ü}rzburg, Germany}, title = {Automated Hybrid Time Series Forecasting: Design, Benchmarking, and Use Cases}, year = 2020 }

Book Chapters

Automatisierte Hybride Zeitreihenprognose. Bauer, André; in Ausgezeichnete Informatikdissertationen 2020, S. Hölldobler (ed.) (2021). 29–38. Gesellschaft für Informatik e.V., Bonn.
- [ BibTeX ]
@inproceedings{mci/Bauer2021, address = {Bonn}, author = {Bauer, André}, booktitle = {Ausgezeichnete Informatikdissertationen 2020}, editor = {Hölldobler, Steffen}, keywords = {telescope}, pages = {29-38}, publisher = {Gesellschaft für Informatik e.V.}, title = {Automatisierte Hybride Zeitreihenprognose}, year = 2021 }
{Elasticity of Cloud Platforms}. Herbst, Nikolas; Bauer, Andr{é}; Kounev, Samuel; in {Systems Benchmarking: For Scientists and Engineers}, S. Kounev, K.-D. Lange, J. von Kistowski (eds.) (2020). 319–340. Springer.
- [ BibTeX ]
@inproceedings{bauer2020bookelasticity, author = {Herbst, Nikolas and Bauer, Andr{\'e} and Kounev, Samuel}, booktitle = {{Systems Benchmarking: For Scientists and Engineers}}, editor = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, J{\'o}akim}, keywords = {t_bookchapter}, month = {01}, organization = {Springer}, pages = {319-340}, title = {{Elasticity of Cloud Platforms}}, year = 2020 }
{Challenges and Approaches: Forecasting for Autonomic Computing}. Bauer, Andr{é}; in {Organic Computing: Doctoral Dissertation Colloquium 2018}, S. Tomforde, B. Sick (eds.) (2019). kassel university press GmbH.
- [ BibTeX ]
- [ Download ]
@incollection{Ba2019-OCDDC-Forecatsing, author = {Bauer, Andr{\'e}}, booktitle = {{Organic Computing: Doctoral Dissertation Colloquium 2018}}, editor = {Tomforde, Sven and Sick, Bernhard}, keywords = {Self-adaptive-systems}, month = {04}, publisher = {kassel university press GmbH}, title = {{Challenges and Approaches: Forecasting for Autonomic Computing}}, year = 2019 }

Peer Reviewed Workshop Papers

An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case. Prantl, Thomas; Engel, Simon; Bauer, Andre; Ben Yahya, Ala Eddine; Herrnleben, Stefan; Iffländer, Lukas; Dmitrienko, Alexandra; Kounev, Samuel; in 2022 IEEE 95th Vehicular Technology Conference (VTC2022) (2022). IEEE.
- [ BibTeX ]
@inproceedings{PrEnBaAlHeIfDmKo2022-IoT/CPS-DistributedGroupEncryption, author = {Prantl, Thomas and Engel, Simon and Bauer, Andre and Ben Yahya, Ala Eddine and Herrnleben, Stefan and Iffländer, Lukas and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {2022 IEEE 95th Vehicular Technology Conference (VTC2022)}, keywords = {t_workshop}, organization = {IEEE}, title = {An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case}, year = 2022 }
{Systematic Search for Optimal Resource Configurations of Distributed Applications}. Bauer, Andr{é}; Eismann, Simon; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019). 120–125. IEEE Computer Society, Los Alamitos, CA, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.

@inproceedings{BaEiGrHeKo-ICAC-BUNGEE, abstract = {With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.}, address = {Los Alamitos, CA, USA}, author = {Bauer, Andr{\'e} and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}, keywords = {Optimization}, month = {06}, pages = {120–125}, publisher = {IEEE Computer Society}, title = {{Systematic Search for Optimal Resource Configurations of Distributed Applications}}, year = 2019 }
{Utilizing Clustering to Optimize Resource Demand Estimation Approaches}. Grohmann, Johannes; Eismann, Simon; Bauer, Andre; Zuefle, Marwin; Herbst, Nikolas; Kounev, Samuel; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019). 134–139. IEEE.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.

@inproceedings{GrEiBaZuHeKo2019-ICAC-RDEClustering, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andre and Zuefle, Marwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}, keywords = {Optimization}, month = {06}, pages = {134–139}, publisher = {IEEE}, title = {{Utilizing Clustering to Optimize Resource Demand Estimation Approaches}}, year = 2019 }

Peer Reviewed Extended Abstracts

{Towards Splitting Monolithic Workflows into Serverless Functions and Estimating Their Run-Time in the Earth Observation Domain}. Kaiser, Dennis; Dovhan, Bohdan; Bauer, Andr{é}; Kounev, Samuel; in Proceedings of the 12th Symposium on Software Performance (SSP) (2021).

Extended Abstract
- [ BibTeX ]
@inproceedings{kaiser2021split, author = {Kaiser, Dennis and Dovhan, Bohdan and Bauer, Andr{\'e} and Kounev, Samuel}, booktitle = {Proceedings of the 12th Symposium on Software Performance (SSP)}, keywords = {severless}, month = 11, note = {Extended Abstract}, title = {{Towards Splitting Monolithic Workflows into Serverless Functions and Estimating Their Run-Time in the Earth Observation Domain}}, year = 2021 }
{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing}. Papadopoulos, Alessandro Vittorio; Versluis, Laurens; Bauer, Andr{{é}}; Herbst, Nikolas; von Kistowski, J{{ó}}akim; Ali{-}Eldin, Ahmed; Abad, Cristina L.; Amaral, Jos{{é}} Nelson; Tuma, Petr; Iosup, Alexandru; in Software Engineering 2020, Fachtagung des GI-Fachbereichs Softwaretechnik, 24.-28. Februar 2020, Innsbruck, Austria, M. Felderer, W. Hasselbring, R. Rabiser, R. Jung (eds.) (2020). (Vol. {P-300}) 93–94. Gesellschaft f{{ü}}r Informatik e.V.
- [ BibTeX ]
- [ URL ]
- [ Download ]
@inproceedings{DBLP:conf/se/PapadopoulosVBH20, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{{\'e}} and Herbst, Nikolas and von Kistowski, J{{\'o}}akim and Ali{-}Eldin, Ahmed and Abad, Cristina L. and Amaral, Jos{{\'e}} Nelson and Tuma, Petr and Iosup, Alexandru}, booktitle = {Software Engineering 2020, Fachtagung des GI-Fachbereichs Softwaretechnik, 24.-28. Februar 2020, Innsbruck, Austria}, editor = {Felderer, Michael and Hasselbring, Wilhelm and Rabiser, Rick and Jung, Reiner}, keywords = {se2}, pages = {93–94}, publisher = {Gesellschaft f{{\"u}}r Informatik e.V.}, series = {{LNI}}, title = {{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing}}, volume = {{P-300}}, year = 2020 }
{Telescope: A Hybrid Forecast Method for Univariate Time Series}. Z{ü}fle, Marwin; Bauer, Andr{é}; Herbst, Nikolas; Curtef, Valentin; Kounev, Samuel; in {Proceedings of the International work-conference on Time Series (ITISE 2017)} (2017).
- [ Abstract ]
- [ BibTeX ]
- [ Download ]
Forecasting is an important part of the decision-making process and used in many fields like business, economics, finance, science, and engineering. According to the No-Free-Lunch-Theorem from 1997, there is no general forecasting method, that performs best for all time series. Instead, expert knowledge is needed to decide which forecasting method to choose for a specific time series with its own characteristics. Since a trial and error approach is very inefficient and expert knowledge is useful but a time-consuming task that cannot be fully automated, we present a new hybrid multi-step-ahead forecasting approach based on time series decomposition. Initial evaluations show that this hybrid approach improves the forecast accuracy compared to six existing forecasting methods while maintaining a short runtime.

@inproceedings{ZBBHCK2017-ITISE-Telescope, abstract = {Forecasting is an important part of the decision-making process and used in many fields like business, economics, finance, science, and engineering. According to the No-Free-Lunch-Theorem from 1997, there is no general forecasting method, that performs best for all time series. Instead, expert knowledge is needed to decide which forecasting method to choose for a specific time series with its own characteristics. Since a trial and error approach is very inefficient and expert knowledge is useful but a time-consuming task that cannot be fully automated, we present a new hybrid multi-step-ahead forecasting approach based on time series decomposition. Initial evaluations show that this hybrid approach improves the forecast accuracy compared to six existing forecasting methods while maintaining a short runtime.}, author = {Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Herbst, Nikolas and Curtef, Valentin and Kounev, Samuel}, booktitle = {{Proceedings of the International work-conference on Time Series (ITISE 2017)}}, keywords = {Optimization}, month = {09}, title = {{Telescope: A Hybrid Forecast Method for Univariate Time Series}}, year = 2017 }

Peer Reviewed Poster Papers

SPEC Research - Introducing the Predictive Data Analytics Working Group. Bauer, André; Leznik, Mark; Seybold, Daniel; Trubin, Igor; Erb, Benjamin; Domaschka, Jörg; Jamshidi, Pooyan; in Companion of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
- [ BibTeX ]
- [ Download ]
@conference{noauthororeditor, author = {Bauer, André and Leznik, Mark and Seybold, Daniel and Trubin, Igor and Erb, Benjamin and Domaschka, Jörg and Jamshidi, Pooyan}, booktitle = {Companion of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {spec}, month = {04}, title = {SPEC Research - Introducing the Predictive Data Analytics Working Group}, year = 2022 }
SPEC Research - Introducing the Predictive Data Analytics Working Group. Bauer, André; Leznik, Mark; Seybold, Daniel; Trubin, Igor; Erb, Benjamin; Domaschka, Jörg; Jamshidi, Pooyan; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022).
- [ BibTeX ]
- [ Download ]
@conference{bauer2022research, author = {Bauer, André and Leznik, Mark and Seybold, Daniel and Trubin, Igor and Erb, Benjamin and Domaschka, Jörg and Jamshidi, Pooyan}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {spec}, month = {04}, series = {ICPE '22}, title = {SPEC Research - Introducing the Predictive Data Analytics Working Group}, year = 2022 }
TeaStore: A Micro-Service Reference Application for Cloud Researchers. Eismann, Simon; v. Kistowski, J{ó}akim; Grohmann, Johannes; Bauer, Andr{é}; Schmitt, Norbert; Herbst, Nikolas; Kounev, Samuel; in Proceedings of 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion) (2018). 11–12. IEEE.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed cloud applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. Further, most cloud reference applications are difficult to deploy and run. In this paper, we present the TeaStore, a micro-service-based test and reference cloud application. TeaStore offers services with various performance characteristics and a high degree of freedom regarding its deployment and configuration to be used as a cloud reference application for researchers. The TeaStore is designed for the evaluation of performance modeling and resource management techniques. We invite cloud researchers to use the TeaStore and provide it open-source, extendable, easily deployable and monitorable.

@inproceedings{EiKiGrBaScHeKo2018-UCC-TeastorePoster, abstract = {Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed cloud applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. Further, most cloud reference applications are difficult to deploy and run. In this paper, we present the TeaStore, a micro-service-based test and reference cloud application. TeaStore offers services with various performance characteristics and a high degree of freedom regarding its deployment and configuration to be used as a cloud reference application for researchers. The TeaStore is designed for the evaluation of performance modeling and resource management techniques. We invite cloud researchers to use the TeaStore and provide it open-source, extendable, easily deployable and monitorable.}, author = {Eismann, Simon and v. Kistowski, J{\'o}akim and Grohmann, Johannes and Bauer, Andr{\'e} and Schmitt, Norbert and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion)}, keywords = {t_poster}, month = 12, pages = {11–12}, publisher = {IEEE}, title = {TeaStore: A Micro-Service Reference Application for Cloud Researchers}, year = 2018 }

Peer Reviewed Tutorial Papers

{Best Practices for Time Series Forecasting}. Bauer, Andr{é}; Z{ü}fle, Marwin; Herbst, Nikolas; Kounev, Samuel; in {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)} (2019).
- [ BibTeX ]
- [ Download ]
@inproceedings{BaZuHeKo2019-ICAC-ForecastingTutorial, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {{2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}}, keywords = {t_tutorial}, month = {06}, title = {{Best Practices for Time Series Forecasting}}, year = 2019 }
{Design and Evaluation of a Proactive, Application-Aware Auto-Scaler}. Bauer, Andr{é}; Herbst, Nikolas; Kounev, Samuel; in {Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)} (2017).
- [ BibTeX ]
- [ Download ]
@inproceedings{BaHeKo2017-ICPE-Auto-Scaler-Tutorial, author = {Bauer, Andr{\'e} and Herbst, Nikolas and Kounev, Samuel}, booktitle = {{Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}}, keywords = {Application-aware}, month = {04}, title = {{Design and Evaluation of a Proactive, Application-Aware Auto-Scaler}}, year = 2017 }

Techreports

{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing - A SPEC Research Technical Report}. (SPEC-RG-2019-04), Papadopoulos, Alessandro Vittorio; Versluis, Laurens; Bauer, Andr{é}; Herbst, Nikolas; von Kistowski, J{ó}akim; Ali-Eldin, Ahmed; Abad, Cristina; Amaral, J. Nelson; Tuma, Petr; Iosup, Alexandru; (2019). SPEC Research Group --- Cloud Working Group, Standard Performance Evaluation Corporation (SPEC).
- [ BibTeX ]
- [ URL ]
@techreport{PaVeBaHeKiEAAbAmTuIo2019-SPEC-CloudPerformance, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{\'e} and Herbst, Nikolas and von Kistowski, J{\'o}akim and Ali-Eldin, Ahmed and Abad, Cristina and Amaral, J. Nelson and Tuma, Petr and Iosup, Alexandru}, institution = {SPEC Research Group — Cloud Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {SPEC}, month = {04}, number = {SPEC-RG-2019-04}, title = {{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing - A SPEC Research Technical Report}}, year = 2019 }
Performance-feedback autoscaling with budget constraints for cloud-based workloads of workflows. Ilyushkin, Alexey; Bauer, Andr{é}; Papadopoulos, Alessandro V; Deelman, Ewa; Iosup, Alexandru; in arXiv preprint arXiv:1905.10270 (2019).
- [ BibTeX ]
- [ Download ]
@article{IlBaPaDeIo-Tech-BudgetAS, author = {Ilyushkin, Alexey and Bauer, Andr{\'e} and Papadopoulos, Alessandro V and Deelman, Ewa and Iosup, Alexandru}, journal = {arXiv preprint arXiv:1905.10270}, keywords = {budget}, title = {Performance-feedback autoscaling with budget constraints for cloud-based workloads of workflows}, year = 2019 }

Newsletter

SPEC Research Group Newsletter, Issue 12. Kounev, Samuel; van Hoorn, Andre; Straesser, Martin; Bauer, Andr{é}; (2023, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-12, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 12}, year = 2023 }
SPEC Research Group Newsletter, Issue 11. Kounev, Samuel; van Hoorn, Andre; Straesser, Martin; Bauer, Andr{é}; (2022, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-11, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 11}, year = 2022 }
{SPEC Research Group Newsletter, vol. 3 no. 2}. Kounev, Samuel; van Hoorn, Andre; Bauer, Andr{é}; (2021, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-10, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{\'e}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 3 no. 2}}, year = 2021 }
{SPEC Research Group Newsletter, vol. 3 no. 1}. Kounev, Samuel; van Hoorn, Andre; Bauer, Andr{é}; Herbst, Nikolas; (2020, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{kounev2020research, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{\'e} and Herbst, Nikolas}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 3 no. 1}}, year = 2020 }
{SPEC Research Group Newsletter, vol. 2 no. 4}. Kounev, Samuel; van Hoorn, Andre; Bauer, Andr{é}; Herbst, Nikolas; (2019, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-8, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{\'e} and Herbst, Nikolas}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 4}}, year = 2019 }
{SPEC Research Group Newsletter, vol. 2 no. 3}. Kounev, Samuel; van Hoorn, Andre; Herbst, Nikolas; Bauer, Andr{é}; (2018, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-7, author = {Kounev, Samuel and van Hoorn, Andre and Herbst, Nikolas and Bauer, Andr{\'e}}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 3}}, year = 2018 }
{SPEC Research Group Newsletter, vol. 2 no. 2}. Kounev, Samuel; van Hoorn, Andre; Rygielski, Piotr; Herbst, Nikolas; Bauer, Andr{é}; (2017, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-6, author = {Kounev, Samuel and van Hoorn, Andre and Rygielski, Piotr and Herbst, Nikolas and Bauer, Andr{\'e}}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 2}}, year = 2017 }

Talks

{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}. Bauer, Andr{é}; (2019, June).
- [ BibTeX ]
- [ Download ]
@misc{talk-Bauer-HotCloudPerf2019, author = {Bauer, Andr{\'e}}, howpublished = {Talk at HotCloudperf 2019}, keywords = {Chameleon}, month = {06}, title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}}, year = 2019 }
{Extending BUNGEE Elasticity Benchmark for Multi-Tier Cloud Applications}. Bauer, Andr{é}; (2018, April).
- [ BibTeX ]
- [ Download ]
@misc{talk-Bauer-HotCloudPerf2018, author = {Bauer, Andr{\'e}}, howpublished = {Talk at HotCloudperf 2018}, keywords = {BUNGEE}, month = {04}, title = {{Extending BUNGEE Elasticity Benchmark for Multi-Tier Cloud Applications}}, year = 2018 }
{Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism}. Bauer, Andr{é}; (2016, November).
- [ BibTeX ]
- [ Download ]
@misc{talk-Bauer-SSP2016, author = {Bauer, Andr{\'e}}, howpublished = {Experience Talk at SSP 2016}, keywords = {BUNGEE}, month = 11, title = {{Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism}}, year = 2016 }
{Chameleon: Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism}. Bauer, Andr{é}; (2016, October).
- [ BibTeX ]
@misc{talk-Bauer-InvSpec16, author = {Bauer, Andr{\'e}}, howpublished = {Invited Talk, Spec Research - Cloud Working Group}, keywords = {Application-aware}, month = 10, title = {{Chameleon: Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism}}, year = 2016 }

Peer Reviewed Journal and Magazine Articles

Peer Reviewed Full Conference Papers

Peer Reviewed Short Conference Papers

PhD Thesis

Book Chapters

Peer Reviewed Workshop Papers

Peer Reviewed Extended Abstracts

Peer Reviewed Poster Papers

Peer Reviewed Tutorial Papers

Techreports

Newsletter

Talks

Data privacy protection

Data privacy protection

Picture credits