Publications

Journal and Magazine Articles

Trust your local scaler: A continuous, decentralized approach to autoscaling. Straesser, Martin; Geissler, Stefan; Lange, Stanislav; Schumann, Lukas Kilian; Hossfeld, Tobias; Kounev, Samuel; in Performance Evaluation (2025). 167 102452.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.

@article{STRAESSER2025102452, abstract = {Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.}, author = {Straesser, Martin and Geissler, Stefan and Lange, Stanislav and Schumann, Lukas Kilian and Hossfeld, Tobias and Kounev, Samuel}, journal = {Performance Evaluation}, keywords = {t_journalmagazine}, pages = 102452, title = {Trust your local scaler: A continuous, decentralized approach to autoscaling}, volume = 167, year = 2025 }
A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions. Henrichs, Elia; Lesch, Veronika; Straesser, Martin; Kounev, Samuel; Krupitzer, Christian; in Information and Software Technology (2022). 149 106940.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.

@article{HENRICHS2022106940, abstract = {Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.}, author = {Henrichs, Elia and Lesch, Veronika and Straesser, Martin and Kounev, Samuel and Krupitzer, Christian}, journal = {Information and Software Technology}, keywords = {sys:relevantfor:se-group}, pages = 106940, title = {A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions}, volume = 149, year = 2022 }

Full Papers

An Empirical Investigation of Container Building Strategies and Warm Times to Reduce Cold Starts in Scientific Computing Serverless Functions. Bauer, André; Gonthier, Maxime; Chard, Ryan; Pan, Haochen; Grzenda, Daniel; Straesser, Martin; Pauloski, J. Gregory; Kamatar, Alok; Baughman, Matthew E.; Hudson, Nathaniel; Foster, Ian; Chard, Kyle; in 2024 IEEE 20th International Conference on e-Science (e-Science) (2024).
- [ BibTeX ]
@inproceedings{bauer2024coldstarts, author = {Bauer, André and Gonthier, Maxime and Chard, Ryan and Pan, Haochen and Grzenda, Daniel and Straesser, Martin and Pauloski, J. Gregory and Kamatar, Alok and Baughman, Matthew E. and Hudson, Nathaniel and Foster, Ian and Chard, Kyle}, booktitle = {2024 IEEE 20th International Conference on e-Science (e-Science)}, keywords = {Scientific_Computing}, title = {An Empirical Investigation of Container Building Strategies and Warm Times to Reduce Cold Starts in Scientific Computing Serverless Functions}, year = 2024 }
Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration. Straesser, Martin; Haas, Patrick; Frank, Sebastian; Hakamian, Alireza; Van Hoorn, André; Kounev, Samuel; in Performance Evaluation Methodologies and Tools (2023).

In print.
- [ Abstract ]
- [ BibTeX ]
- [ Download ]
Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.

@inproceedings{straesser2023kubernetesintheloop, abstract = {Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.}, author = {Straesser, Martin and Haas, Patrick and Frank, Sebastian and Hakamian, Alireza and Van Hoorn, André and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, keywords = {discrete_event_simulation}, note = {In print.}, title = {Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration}, year = 2023 }
An Empirical Study of Container Image Configurations and Their Impact on Start Times. Straesser, Martin; Bauer, André; Leppich, Robert; Herbst, Nikolas; Chard, Kyle; Foster, Ian; Kounev, Samuel; in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid) (2023). 94–105. IEEE.
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
@inproceedings{straesser2023starttime, author = {Straesser, Martin and Bauer, André and Leppich, Robert and Herbst, Nikolas and Chard, Kyle and Foster, Ian and Kounev, Samuel}, booktitle = {2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)}, keywords = {empirical_study}, pages = {94-105}, publisher = {IEEE}, title = {An Empirical Study of Container Image Configurations and Their Impact on Start Times}, year = 2023 }
Autoscaler Evaluation and Configuration: A Practitioner’s Guideline. Straesser, Martin; Eismann, Simon; von Kistowski, Jóakim; Bauer, André; Kounev, Samuel; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 31–41. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.

@inproceedings{straesser2023guideline, abstract = {Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.}, address = {New York, NY, USA}, author = {Straesser, Martin and Eismann, Simon and von Kistowski, J\'{o}akim and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {methodology}, pages = {31-41}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {Autoscaler Evaluation and Configuration: A Practitioner's Guideline}, year = 2023 }
A Systematic Approach for Benchmarking of Container Orchestration Frameworks. Straesser, Martin; Mathiasch, Jonas; Bauer, André; Kounev, Samuel; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 187–198. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.

@inproceedings{straesser2023orchestration, abstract = {Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.}, address = {New York, NY, USA}, author = {Straesser, Martin and Mathiasch, Jonas and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {container_orchestration}, pages = {187-198}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {A Systematic Approach for Benchmarking of Container Orchestration Frameworks}, year = 2023 }
Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling. Straesser, Martin; Grohmann, Johannes; von Kistowski, Jóakim; Eismann, Simon; Bauer, André; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 105–115. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.

@inproceedings{10.1145/3489525.3511680, abstract = {Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.}, address = {New York, NY, USA}, author = {Straesser, Martin and Grohmann, Johannes and von Kistowski, J\'{o}akim and Eismann, Simon and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {microservices}, pages = {105–115}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling}, year = 2022 }
MiSim: A Simulator for Resilience Assessment of Microservice-based Architectures. Frank, Sebastian; Wagner, Lion; Hakamian, Alireza; Straesser, Martin; van Hoorn, Andre; in 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) (2022). 1014–1025. IEEE.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Increased resilience compared to monolithic architectures is both one of the key promises of microservice-based architectures and a big challenge, e.g., due to the systems’ distributed nature. Resilience assessment through simulation requires fewer resources than the measurement-based techniques used in practice. However, there is no existing simulation approach that is suitable for a holistic resilience assessment of microservices comprised of (i) representative fault injections, (ii) common resilience mechanisms, and (iii) time-varying workloads. This paper presents MiSim — an extensible simulator for resilience assessment of microservice-based architectures. It overcomes the stated limitations of related work. MiSim fits resilience engineering practices by supporting scenario-based experiments and requiring only lightweight input models. We demonstrate how MiSim simulates (1) common resilience mechanisms — i.e., circuit breaker, connection limiter, retry, load balancer, and autoscaler — and (2) fault injections — i.e., instance/service killing and latency injections. In addition, we use TeaStore, a reference microservice-based architecture, aiming to reproduce scaling behavior from an experiment by using simulation. Our results show that MiSim allows for quantitative insights into microservice-based systems’ complex transient behavior by providing up to 25 metrics.

@inproceedings{FrankWagnerHakamianStraesserVanHoorn2022MiSim, abstract = {Increased resilience compared to monolithic architectures is both one of the key promises of microservice-based architectures and a big challenge, e.g., due to the systems’ distributed nature. Resilience assessment through simulation requires fewer resources than the measurement-based techniques used in practice. However, there is no existing simulation approach that is suitable for a holistic resilience assessment of microservices comprised of (i) representative fault injections, (ii) common resilience mechanisms, and (iii) time-varying workloads. This paper presents MiSim — an extensible simulator for resilience assessment of microservice-based architectures. It overcomes the stated limitations of related work. MiSim fits resilience engineering practices by supporting scenario-based experiments and requiring only lightweight input models. We demonstrate how MiSim simulates (1) common resilience mechanisms — i.e., circuit breaker, connection limiter, retry, load balancer, and autoscaler — and (2) fault injections — i.e., instance/service killing and latency injections. In addition, we use TeaStore, a reference microservice-based architecture, aiming to reproduce scaling behavior from an experiment by using simulation. Our results show that MiSim allows for quantitative insights into microservice-based systems’ complex transient behavior by providing up to 25 metrics.}, author = {Frank, Sebastian and Wagner, Lion and Hakamian, Alireza and Straesser, Martin and van Hoorn, Andre}, booktitle = {2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)}, keywords = {resilience}, pages = {1014-1025}, publisher = {IEEE}, title = {MiSim: A Simulator for Resilience Assessment of Microservice-based Architectures}, year = 2022 }
SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications. Grohmann, Johannes; Straesser, Martin; Chalbani, Avi; Eismann, Simon; Arian, Yair; Herbst, Nikolas; Peretz, Noam; Kounev, Samuel; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021). ACM, New York, NY, USA.

Acceptance Rate: 29%
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.

@inproceedings{GrStChEiArHePeKo2021-ICPE, abstract = {Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing—a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90\%.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Straesser, Martin and Chalbani, Avi and Eismann, Simon and Arian, Yair and Herbst, Nikolas and Peretz, Noam and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {performance}, month = {04}, note = {Acceptance Rate: 29%}, organization = {ACM}, title = {SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications}, year = 2021 }

Short Papers

Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis. Bauer, André; Dittus, Timo; Straesser, Martin; Kamatar, Alok; Baughman, Matt; Beierlieb, Lukas; Hadry, Marius; Grillmeyer, Daniel; Lubas, Yannik; Kounev, Samuel; Foster, Ian; Chard, Kyle; in Companion of the 15th ACM/SPEC International Conference on Performance Engineering (2024). 72–76. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

@inproceedings{10.1145/3629527.3651843, abstract = {As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.}, address = {New York, NY, USA}, author = {Bauer, André and Dittus, Timo and Straesser, Martin and Kamatar, Alok and Baughman, Matt and Beierlieb, Lukas and Hadry, Marius and Grillmeyer, Daniel and Lubas, Yannik and Kounev, Samuel and Foster, Ian and Chard, Kyle}, booktitle = {Companion of the 15th ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {72–76}, publisher = {Association for Computing Machinery}, series = {ICPE '24 Companion}, title = {Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis}, year = 2024 }
Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs. Bauer, André; Straesser, Martin; Leznik, Mark; Beierlieb, Lukas; Hadry, Marius; Hudson, Nathaniel; Chard, Kyle; Kounev, Samuel; Foster, Ian; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 95–99. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.

@inproceedings{10.1145/3578245.3584693, abstract = {Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.}, address = {New York, NY, USA}, author = {Bauer, Andr\'{e} and Straesser, Martin and Leznik, Mark and Beierlieb, Lukas and Hadry, Marius and Hudson, Nathaniel and Chard, Kyle and Kounev, Samuel and Foster, Ian}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {distance_measures}, pages = {95–99}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs}, year = 2023 }
Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning. Bauer, André; Straesser, Martin; Beierlieb, Lukas; Meissner, Maximilian; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). ACM.
- [ BibTeX ]
- [ Download ]
@conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {machine_learning}, month = {04}, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }

Workshop Papers

An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times. Straesser, Martin; Erhard, Nicholas; Kounev, Samuel; (2024). Gesellschaft für Informatik e.V.

In print.
- [ BibTeX ]
- [ Download ]
@misc{StraesserSSP2024, author = {Straesser, Martin and Erhard, Nicholas and Kounev, Samuel}, booktitle = {Softwaretechnik-Trends Band 44, Heft 4}, keywords = {t_workshop}, note = {In print.}, publisher = {Gesellschaft für Informatik e.V.}, series = {Softwaretechnik-Trends}, title = {An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times}, year = 2024 }

Poster Papers

Power to the Applications: The Vision of Continuous Decentralized Autoscaling. Straesser, Martin; Geißler, Stefan; Hoßfeld, Tobias; Kounev, Samuel; in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW) (2023). 281–283. IEEE.
- [ BibTeX ]
- [ URL ]
- [ DOI ]
- [ Download ]
@inproceedings{straesser2023decentral, author = {Straesser, Martin and Geißler, Stefan and Hoßfeld, Tobias and Kounev, Samuel}, booktitle = {2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW)}, keywords = {t_poster}, pages = {281-283}, publisher = {IEEE}, title = {Power to the Applications: The Vision of Continuous Decentralized Autoscaling}, year = 2023 }

Extended Abstracts

Simulating Microservice-based Architectures for Resilience Assessment Enriched by Authentic Container Orchestration. Frank, Sebastian; Straesser, Martin; Wagner, Lion; Haas, Patrick; Hakamian, Alireza; Kounev, Samuel; Van Hoorn, André; in Software Engineering 2024 (2024). Gesellschaft für Informatik e.V., Bonn.
- [ BibTeX ]
- [ Download ]
@incollection{Frank2024Simulating, address = {Bonn}, author = {Frank, Sebastian and Straesser, Martin and Wagner, Lion and Haas, Patrick and Hakamian, Alireza and Kounev, Samuel and Van Hoorn, André}, booktitle = {Software Engineering 2024}, keywords = {container_orchestration}, publisher = {Gesellschaft für Informatik e.V.}, title = {Simulating Microservice-based Architectures for Resilience Assessment Enriched by Authentic Container Orchestration}, year = 2024 }

Talks

Autoscaling Rethought: A Continuous, Decentralized Approach Straesser, Martin; (2024).
- [ BibTeX ]
- [ Download ]
@presentation{straesser2024rethought, author = {Straesser, Martin}, keywords = {microservices}, title = {Autoscaling Rethought: A Continuous, Decentralized Approach}, year = 2024 }
Towards Explainable, Coordinated and Proactive Autoscaling for Microservices and Function Chains Straesser, Martin; (2024).
- [ BibTeX ]
- [ Download ]
@presentation{straesser2024towards, author = {Straesser, Martin}, keywords = {microservices}, title = {Towards Explainable, Coordinated and Proactive Autoscaling for Microservices and Function Chains}, year = 2024 }
Generating Microservice Applications for Performance Benchmarking Lubas, Yannik; Straesser, Martin; Kounev, Samuel; (2023).
- [ BibTeX ]
- [ Download ]
@presentation{lubas2023generating, author = {Lubas, Yannik and Straesser, Martin and Kounev, Samuel}, keywords = {microservices}, title = {Generating Microservice Applications for Performance Benchmarking}, year = 2023 }
Towards Simulation-Driven Optimization of Container Orchestration Mechanisms Dittus, Timo; Straesser, Martin; Kounev, Samuel; (2023).
- [ BibTeX ]
- [ Download ]
@presentation{dittus2023simdrivenopt, author = {Dittus, Timo and Straesser, Martin and Kounev, Samuel}, keywords = {container_orchestration}, title = {Towards Simulation-Driven Optimization of Container Orchestration Mechanisms}, year = 2023 }
Leveraging Kubernetes Source Code for Performance Simulation Straesser, Martin; Haas, Patrick; (2022).
- [ BibTeX ]
- [ URL ]
- [ Download ]
@presentation{straesser2022leveraging, author = {Straesser, Martin and Haas, Patrick}, keywords = {container_orchestration}, title = {Leveraging Kubernetes Source Code for Performance Simulation}, year = 2022 }
Container Start Times: Empirical Analysis and Predictability Straesser, Martin; (2021).
- [ BibTeX ]
- [ URL ]
- [ Download ]
@presentation{straesser2021container, author = {Straesser, Martin}, keywords = {empirical_study}, title = {Container Start Times: Empirical Analysis and Predictability}, year = 2021 }
Predicting Performance Degradations of Black-box Microservice Applications Straesser, Martin; (2020).
- [ BibTeX ]
- [ URL ]
- [ Download ]
@presentation{straesser2020talk, author = {Straesser, Martin}, keywords = {sys:relevantfor:se-group}, title = {Predicting Performance Degradations of Black-box Microservice Applications}, year = 2020 }

Master Thesis

Predicting Performance Degradations of Microservice Applications. Straesser, Martin; (2020, May). University of Wuerzburg.

Wuerzburg Software Engineering Award 2020: Best Master Thesis
- [ BibTeX ]
- [ Download ]
@mastersthesis{straesser2020predicting, author = {Straesser, Martin}, keywords = {performance_prediction}, month = {05}, note = {Wuerzburg Software Engineering Award 2020: Best Master Thesis}, school = {University of Wuerzburg}, title = {Predicting Performance Degradations of Microservice Applications}, year = 2020 }

Miscellaneous

SPEC Research Group Newsletter, Issue 12. Kounev, Samuel; van Hoorn, Andre; Straesser, Martin; Bauer, André; (2023, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-12, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 12}, year = 2023 }
SPEC Research Group Newsletter, Issue 11. Kounev, Samuel; van Hoorn, Andre; Straesser, Martin; Bauer, André; (2022, April).

Published by Standard Performance Evaluation Corporation (SPEC)
- [ BibTeX ]
- [ Download ]
@misc{SPEC-RG-NEWSLETTER-11, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 11}, year = 2022 }

Press Coverage

Universität Würzburg (Press Release): Damit der Webshop rundläuft. August 2021. [ Link ]

Journal and Magazine Articles

Full Papers

Short Papers

Workshop Papers

Poster Papers

Extended Abstracts

Talks

Master Thesis

Miscellaneous

Press Coverage

Hinweis zum Datenschutz

Hinweis zum Datenschutz

Bildnachweise