Publications

• [ 2020 ]
• [ 2017 ]
• [ 2015 ]
• [ 2013 ]
• [ 2011 ]
• [ 2008 ]
• [ 2005 ]

• 
  Systems Benchmarking Kounev, Samuel; Lange, Klaus-Dieter; von Kistowski, Jóakim; (2020). (1st ed.) Springer International Publishing.
  • [ BibTeX ]
  • [ EndNote ]
  @book{KoLaKi-2020-SystemsBenchmarking, author = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, Jóakim}, edition = 1, keywords = {t_book}, publisher = {Springer International Publishing}, title = {{Systems Benchmarking}}, year = 2020 }

  %0 Book %1 KoLaKi-2020-SystemsBenchmarking %A Kounev, Samuel %A Lange, Klaus-Dieter %A von Kistowski, Jóakim %D 2020 %I Springer International Publishing %T Systems Benchmarking %7 1

• 
  Self-Aware Computing Systems Kounev, Samuel; Kephart, Jeffrey O.; Milenkoski, Aleksandar; Zhu, Xiaoyun; (2017). Springer Verlag, Berlin Heidelberg, Germany.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  @book{KoKeMiZh2017-self, address = {{Berlin Heidelberg, Germany}}, author = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Self-Aware Computing Systems}}, year = 2017 }

  %0 Book %1 KoKeMiZh2017-self %A Kounev, Samuel %A Kephart, Jeffrey O. %A Milenkoski, Aleksandar %A Zhu, Xiaoyun %C Berlin Heidelberg, Germany %D 2017 %I Springer Verlag %R 10.1007/978-3-319-47474-8 %T Self-Aware Computing Systems %U http://www.springer.com/de/book/9783319474724 %@ 978-3-319-47474-8

• 
  Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8--10, 2015 Kounev, Samuel; (P. Lalanda; A. Diaconescu; L. Cherkasova; S. Kounev, eds.) (2015). IEEE Computer Society Conference Publishing Services (CPS), Los Alamitos, California.
  • [ BibTeX ]
  • [ EndNote ]
  @proceedings{proceedings-icac-2015, address = {Los Alamitos, California}, author = {Kounev, Samuel}, editor = {Lalanda, Philippe and Diaconescu, Ada and Cherkasova, Lucy and Kounev, Samuel}, keywords = {se2}, month = {07}, publisher = {IEEE Computer Society Conference Publishing Services (CPS)}, title = {Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8–10, 2015}, year = 2015 }

  %0 Conference Proceedings %1 proceedings-icac-2015 %A Kounev, Samuel %C Los Alamitos, California %D 2015 %E Lalanda, Philippe %E Diaconescu, Ada %E Cherkasova, Lucy %E Kounev, Samuel %I IEEE Computer Society Conference Publishing Services (CPS) %T Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8--10, 2015
• 
  Model-driven Algorithms and Architectures for Self-Aware Computing Systems Kounev, Samuel; in Dagstuhl Reports, (S. Kounev; X. Zhu; J. O. Kephart; M. Kwiatkowska; A. M. [Assistant Editor], eds.) (2015). Dagstuhl, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.

  @proceedings{KoZhKeKw2015, abstract = {{This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.}}, address = {Dagstuhl, Germany}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Zhu, Xiaoyun and Kephart, Jeffrey O. and Kwiatkowska, Marta and Editor], Aleksandar Milenkoski [Assistant}, keywords = {Self-adaptive-systems}, month = {01}, series = {Dagstuhl Reports}, title = {{Model-driven Algorithms and Architectures for Self-Aware Computing Systems}}, year = 2015 }

  %0 Conference Proceedings %1 KoZhKeKw2015 %A Kounev, Samuel %B Dagstuhl Reports %C Dagstuhl, Germany %D 2015 %E Kounev, Samuel %E Zhu, Xiaoyun %E Kephart, Jeffrey O. %E Kwiatkowska, Marta %E Editor], Aleksandar Milenkoski [Assistant %T Model-driven Algorithms and Architectures for Self-Aware Computing Systems %U http://drops.dagstuhl.de/opus/volltexte/2015/5038/ %X This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.

• 
  Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) Kounev, Samuel; (S. Kounev; S. Zschaler; K. Sachs, eds.) (2013). ACM.
  • [ BibTeX ]
  • [ EndNote ]
  @proceedings{KoZsSa2013-HOTTOPICS-Proceedings, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Zschaler, Steffen and Sachs, Kai}, keywords = {t_proceedings}, month = {04}, publisher = {ACM}, title = {Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)}, year = 2013 }

  %0 Conference Proceedings %1 KoZsSa2013-HOTTOPICS-Proceedings %A Kounev, Samuel %D 2013 %E Kounev, Samuel %E Zschaler, Steffen %E Sachs, Kai %I ACM %T Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)

• 
  ICPE’11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14--16, 2011 Kounev, Samuel; (S. Kounev; V. Cortellessa; R. Mirandola; D. J. Lilja, eds.) (2011). ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  @proceedings{DBLP-conf-wosp-2011, address = {New York, NY, USA}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Cortellessa, Vittorio and Mirandola, Raffaela and Lilja, David J.}, keywords = {Power}, month = {03}, publisher = {ACM}, title = {ICPE'11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14–16, 2011}, year = 2011 }

  %0 Conference Proceedings %1 DBLP-conf-wosp-2011 %A Kounev, Samuel %C New York, NY, USA %D 2011 %E Kounev, Samuel %E Cortellessa, Vittorio %E Mirandola, Raffaela %E Lilja, David J. %I ACM %T ICPE'11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14--16, 2011

• 
  Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28 Kounev, Samuel; in Lecture Notes in Computer Science (LNCS), (S. Kounev; I. Gorton; K. Sachs, eds.) (2008). (Vol. 5119) Springer, Heidelberg, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.

  @proceedings{kounev2008c, abstract = {This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.}, address = {Heidelberg, Germany}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Gorton, Ian and Sachs, Kai}, keywords = {Power}, month = {06}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28}}, volume = 5119, year = 2008 }

  %0 Conference Proceedings %1 kounev2008c %A Kounev, Samuel %B Lecture Notes in Computer Science (LNCS) %C Heidelberg, Germany %D 2008 %E Kounev, Samuel %E Gorton, Ian %E Sachs, Kai %I Springer %T Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28 %U http://www.springer.com/computer/programming/book/978-3-540-69813-5 %V 5119 %X This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.

• 
  Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction Kounev, Samuel; (2005). Shaker Verlag, Ph.D. Thesis, Technische Universität Darmstadt, Germany.

  Distinguished Dissertation Award
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @book{Ko2005-TUD-PhD_Thesis, author = {Kounev, Samuel}, keywords = {QPME_Bibliography}, month = 12, note = {<b>Distinguished Dissertation Award</b>}, publisher = {Shaker Verlag, Ph.D. Thesis, Technische Universit{\"a}t Darmstadt, Germany}, title = {Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction}, year = 2005 }

  %0 Book %1 Ko2005-TUD-PhD_Thesis %A Kounev, Samuel %D 2005 %I Shaker Verlag, Ph.D. Thesis, Technische Universität Darmstadt, Germany %T Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction

• [ 2025 ]
• [ 2024 ]
• [ 2023 ]
• [ 2022 ]
• [ 2021 ]
• [ 2020 ]
• [ 2019 ]
• [ 2018 ]
• [ 2017 ]
• [ 2016 ]
• [ 2015 ]
• [ 2014 ]
• [ 2013 ]
• [ 2011 ]
• [ 2009 ]
• [ 2008 ]
• [ 2006 ]
• [ 2004 ]
• [ 1999 ]

• 
  Continuous Earth Observation of Forest Dynamics and Biodiversity. Herbst, Nikolas; Kortmann, Mareike; Jaggy, Niklas; Thonfeld, Frank; Abad, Christina; Künzer, Claudia; Müller, Jörg; Kounev, Samuel; in Communications of the ACM (2025).

  to appear
  • [ BibTeX ]
  • [ EndNote ]
  @article{herbst2025continuous, author = {Herbst, Nikolas and Kortmann, Mareike and Jaggy, Niklas and Thonfeld, Frank and Abad, Christina and Künzer, Claudia and Müller, Jörg and Kounev, Samuel}, journal = {Communications of the ACM}, keywords = {t_journalmagazine}, note = {to appear}, title = {Continuous Earth Observation of Forest Dynamics and Biodiversity}, year = 2025 }

  %0 Journal Article %1 herbst2025continuous %A Herbst, Nikolas %A Kortmann, Mareike %A Jaggy, Niklas %A Thonfeld, Frank %A Abad, Christina %A Künzer, Claudia %A Müller, Jörg %A Kounev, Samuel %D 2025 %J Communications of the ACM %T Continuous Earth Observation of Forest Dynamics and Biodiversity
• 
  Telling Fortunes? Evaluation of Traffic Forecasting Models using Traffic and Context Features. Hadry, Marius; Bauer, André; Leppich, Robert; Lesch, Veronika; Kounev, Samuel; in Applied intelligence (2025). Springer.

  just accepted
  • [ BibTeX ]
  • [ EndNote ]
  @article{hadry2025telling, author = {Hadry, Marius and Bauer, André and Leppich, Robert and Lesch, Veronika and Kounev, Samuel}, journal = {Applied intelligence}, keywords = {t_journalmagazine}, note = {just accepted}, publisher = {Springer}, title = {Telling Fortunes? Evaluation of Traffic Forecasting Models using Traffic and Context Features}, year = 2025 }

  %0 Journal Article %1 hadry2025telling %A Hadry, Marius %A Bauer, André %A Leppich, Robert %A Lesch, Veronika %A Kounev, Samuel %D 2025 %I Springer %J Applied intelligence %T Telling Fortunes? Evaluation of Traffic Forecasting Models using Traffic and Context Features
• 
  Trust your local scaler: A continuous, decentralized approach to autoscaling. Straesser, Martin; Geissler, Stefan; Lange, Stanislav; Schumann, Lukas Kilian; Hossfeld, Tobias; Kounev, Samuel; in Performance Evaluation (2025). 167 102452.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.

  @article{STRAESSER2025102452, abstract = {Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.}, author = {Straesser, Martin and Geissler, Stefan and Lange, Stanislav and Schumann, Lukas Kilian and Hossfeld, Tobias and Kounev, Samuel}, journal = {Performance Evaluation}, keywords = {t_journalmagazine}, pages = 102452, title = {Trust your local scaler: A continuous, decentralized approach to autoscaling}, volume = 167, year = 2025 }

  %0 Journal Article %1 STRAESSER2025102452 %A Straesser, Martin %A Geissler, Stefan %A Lange, Stanislav %A Schumann, Lukas Kilian %A Hossfeld, Tobias %A Kounev, Samuel %D 2025 %J Performance Evaluation %P 102452 %R https://doi.org/10.1016/j.peva.2024.102452 %T Trust your local scaler: A continuous, decentralized approach to autoscaling %U https://www.sciencedirect.com/science/article/pii/S0166531624000579 %V 167 %X Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.

• 
  Challenges in the evaluation of earth observation products: Accuracy assessment case study using convolutional neural networks. Prantl, Thomas; Barthel, Til; Kaiser, Dennis; Schwinger, Maximilian; Bauer, André; Kounev, Samuel; in Remote Sensing Applications: Society and Environment (2024). Elsevier BV.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  @article{PrBaKaScBaKo_2024_EO_Challenges, author = {Prantl, Thomas and Barthel, Til and Kaiser, Dennis and Schwinger, Maximilian and Bauer, André and Kounev, Samuel}, journal = {Remote Sensing Applications: Society and Environment}, keywords = {t_journalmagazine}, month = 12, publisher = {Elsevier BV}, title = {Challenges in the evaluation of earth observation products: Accuracy assessment case study using convolutional neural networks}, year = 2024 }

  %0 Journal Article %1 PrBaKaScBaKo_2024_EO_Challenges %A Prantl, Thomas %A Barthel, Til %A Kaiser, Dennis %A Schwinger, Maximilian %A Bauer, André %A Kounev, Samuel %D 2024 %I Elsevier BV %J Remote Sensing Applications: Society and Environment %R 10.1016/j.rsase.2024.101420 %T Challenges in the evaluation of earth observation products: Accuracy assessment case study using convolutional neural networks %U http://dx.doi.org/10.1016/j.rsase.2024.101420
• 
  Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis. Stenger, Michael; Bauer, André; Prantl, Thomas; Leppich, Robert; Hudson, Nathaniel; Chard, Kyle; Foster, Ian; Kounev, Samuel; in ACM J. Data and Information Quality (2024). 16(2) Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.

  @article{10.1145/3666006, abstract = {Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.}, address = {New York, NY, USA}, author = {Stenger, Michael and Bauer, Andr\'{e} and Prantl, Thomas and Leppich, Robert and Hudson, Nathaniel and Chard, Kyle and Foster, Ian and Kounev, Samuel}, journal = {ACM J. Data and Information Quality}, keywords = {t_journalmagazine}, month = {06}, number = 2, publisher = {Association for Computing Machinery}, title = {Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis}, volume = 16, year = 2024 }

  %0 Journal Article %1 10.1145/3666006 %A Stenger, Michael %A Bauer, André %A Prantl, Thomas %A Leppich, Robert %A Hudson, Nathaniel %A Chard, Kyle %A Foster, Ian %A Kounev, Samuel %C New York, NY, USA %D 2024 %I Association for Computing Machinery %J ACM J. Data and Information Quality %N 2 %T Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis %U https://doi.org/10.1145/3666006 %V 16 %X Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.
• 
  Benchmarking of Secure Group Communication schemes with Focus on IoT. Prantl, Thomas; Bauer, André; Engel, Simon; Horn, Lukas; Krupitzer, Christian; Iffl"ander, Lukas; Kounev, Samuel; in Discover Data (2024). 2(1) Springer Science and Business Media LLC.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  @article{PrBaEnHoKrIfKo_Discover_Data_2024, author = {Prantl, Thomas and Bauer, André and Engel, Simon and Horn, Lukas and Krupitzer, Christian and Iffl"{a}nder, Lukas and Kounev, Samuel}, journal = {Discover Data}, keywords = {t_journalmagazine}, month = {05}, number = 1, publisher = {Springer Science and Business Media LLC}, title = {Benchmarking of Secure Group Communication schemes with Focus on IoT}, volume = 2, year = 2024 }

  %0 Journal Article %1 PrBaEnHoKrIfKo_Discover_Data_2024 %A Prantl, Thomas %A Bauer, André %A Engel, Simon %A Horn, Lukas %A Krupitzer, Christian %A Iffl"ander, Lukas %A Kounev, Samuel %D 2024 %I Springer Science and Business Media LLC %J Discover Data %N 1 %T Benchmarking of Secure Group Communication schemes with Focus on IoT %U http://dx.doi.org/10.1007/s44248-024-00010-6 %V 2
• 
  Evaluation is key: a survey on evaluation measures for synthetic time series. Stenger, Michael; Leppich, Robert; Foster, Ian; Kounev, Samuel; Bauer, André; in Journal of Big Data (2024). 11 66.
  • [ BibTeX ]
  • [ EndNote ]
  @article{Stenger2024, author = {Stenger, Michael and Leppich, Robert and Foster, Ian and Kounev, Samuel and Bauer, Andr{\'e}}, journal = {Journal of Big Data}, keywords = {t_journalmagazine}, month = {05}, pages = 66, title = {Evaluation is key: a survey on evaluation measures for synthetic time series}, volume = 11, year = 2024 }

  %0 Journal Article %1 Stenger2024 %A Stenger, Michael %A Leppich, Robert %A Foster, Ian %A Kounev, Samuel %A Bauer, André %D 2024 %J Journal of Big Data %P 66 %T Evaluation is key: a survey on evaluation measures for synthetic time series %V 11
• 
  A Data-Driven Model for Range Sensors. Spiess, Florian; Strobel, Norbert; Kaupp, Tobias; Kounev, Samuel; in International Journal of Semantic Computing (2024). 1–18.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  In this paper, an analysis of the precision of range sensors is presented. Light Detection and Ranging (LIDAR) data from three different sensors (HLS-LFCD-LDS, SICK TIM561, and Kinect V2), stereo data from the Realsense D435, and structured light data from both Kinect V1 for Xbox and the Xtion PRO Live were analyzed regarding the influence of range, incident angle to the surface, and surface material. In addition, a comparison with standard deviation models based on vendor-provided specifications was performed. We found that the vendor specifications are in general not specific enough to facilitate accurate sensor simulations. Therefore, we developed a data-driven model for range precision. Our model can be used to create realistic sensor simulations and to develop robot navigation algorithms weighing sensor range readings based on the precision.

  @article{doi:10.1142/S1793351X24300012, abstract = {In this paper, an analysis of the precision of range sensors is presented. Light Detection and Ranging (LIDAR) data from three different sensors (HLS-LFCD-LDS, SICK TIM561, and Kinect V2), stereo data from the Realsense D435, and structured light data from both Kinect V1 for Xbox and the Xtion PRO Live were analyzed regarding the influence of range, incident angle to the surface, and surface material. In addition, a comparison with standard deviation models based on vendor-provided specifications was performed. We found that the vendor specifications are in general not specific enough to facilitate accurate sensor simulations. Therefore, we developed a data-driven model for range precision. Our model can be used to create realistic sensor simulations and to develop robot navigation algorithms weighing sensor range readings based on the precision.}, author = {Spiess, Florian and Strobel, Norbert and Kaupp, Tobias and Kounev, Samuel}, journal = {International Journal of Semantic Computing}, keywords = {t_journalmagazine}, month = {02}, pages = {1-18}, title = {A Data-Driven Model for Range Sensors}, year = 2024 }

  %0 Journal Article %1 doi:10.1142/S1793351X24300012 %A Spiess, Florian %A Strobel, Norbert %A Kaupp, Tobias %A Kounev, Samuel %D 2024 %J International Journal of Semantic Computing %P 1-18 %T A Data-Driven Model for Range Sensors %U https://doi.org/10.1142/S1793351X24300012 %X In this paper, an analysis of the precision of range sensors is presented. Light Detection and Ranging (LIDAR) data from three different sensors (HLS-LFCD-LDS, SICK TIM561, and Kinect V2), stereo data from the Realsense D435, and structured light data from both Kinect V1 for Xbox and the Xtion PRO Live were analyzed regarding the influence of range, incident angle to the surface, and surface material. In addition, a comparison with standard deviation models based on vendor-provided specifications was performed. We found that the vendor specifications are in general not specific enough to facilitate accurate sensor simulations. Therefore, we developed a data-driven model for range precision. Our model can be used to create realistic sensor simulations and to develop robot navigation algorithms weighing sensor range readings based on the precision.
• 
  Comprehensive Exploration of Synthetic Data Generation: A Survey. Bauer, André; and Trapp, Simon; and Stenger, Michael; and Leppich, Robert; and Kounev, Samuel; and Leznik, Mark; and Chard, Kyle; and Foster, Ian; in arXiv preprint (2024). 103.

  https://doi.org/10.48550/arXiv.2401.02524
  • [ BibTeX ]
  • [ EndNote ]
  @article{bauer2024comprehensive, author = {Bauer, André and and Trapp, Simon and and Stenger, Michael and and Leppich, Robert and and Kounev, Samuel and and Leznik, Mark and and Chard, Kyle and and Foster, Ian}, journal = {arXiv preprint}, keywords = {t_journalmagazine}, note = {https://doi.org/10.48550/arXiv.2401.02524}, pages = 103, title = {Comprehensive Exploration of Synthetic Data Generation: A Survey}, year = 2024 }

  %0 Journal Article %1 bauer2024comprehensive %A Bauer, André %A and Trapp, Simon %A and Stenger, Michael %A and Leppich, Robert %A and Kounev, Samuel %A and Leznik, Mark %A and Chard, Kyle %A and Foster, Ian %D 2024 %J arXiv preprint %P 103 %T Comprehensive Exploration of Synthetic Data Generation: A Survey
• 
  Attack-Aware Security Function Chaining. Iffländer, Lukas; Beierlieb, Lukas; Kounev, Samuel; in Electronics (2024). 13(17)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  Cyberattacks have become more frequent and more violent in recent years. To date, defensive infrastructure has been relatively static, and security functions are usually placed in a common order that does not depend on the current situation. We propose the concept of attack-aware Security Service Function Chain reordering. The idea is to change the order of security functions depending on the malicious traffic observed. We present the basic idea, evaluate the impact of the function chain order, and introduce a framework for function chain reordering. Our evaluation shows that the order often has a significant impact on the performance of the security function chain and that there is no single order that outperforms all other orders in every situation. The proposed proof-of-concept framework successfully validates the feasibility of attack-aware security function chain reordering, and we propose additional extensions to eliminate the remaining deficiencies.

  @article{electronics13173357, abstract = {Cyberattacks have become more frequent and more violent in recent years. To date, defensive infrastructure has been relatively static, and security functions are usually placed in a common order that does not depend on the current situation. We propose the concept of attack-aware Security Service Function Chain reordering. The idea is to change the order of security functions depending on the malicious traffic observed. We present the basic idea, evaluate the impact of the function chain order, and introduce a framework for function chain reordering. Our evaluation shows that the order often has a significant impact on the performance of the security function chain and that there is no single order that outperforms all other orders in every situation. The proposed proof-of-concept framework successfully validates the feasibility of attack-aware security function chain reordering, and we propose additional extensions to eliminate the remaining deficiencies.}, author = {Iffländer, Lukas and Beierlieb, Lukas and Kounev, Samuel}, journal = {Electronics}, keywords = {t_journalmagazine}, number = 17, title = {Attack-Aware Security Function Chaining}, volume = 13, year = 2024 }

  %0 Journal Article %1 electronics13173357 %A Iffländer, Lukas %A Beierlieb, Lukas %A Kounev, Samuel %D 2024 %J Electronics %N 17 %R 10.3390/electronics13173357 %T Attack-Aware Security Function Chaining %U https://www.mdpi.com/2079-9292/13/17/3357 %V 13 %X Cyberattacks have become more frequent and more violent in recent years. To date, defensive infrastructure has been relatively static, and security functions are usually placed in a common order that does not depend on the current situation. We propose the concept of attack-aware Security Service Function Chain reordering. The idea is to change the order of security functions depending on the malicious traffic observed. We present the basic idea, evaluate the impact of the function chain order, and introduce a framework for function chain reordering. Our evaluation shows that the order often has a significant impact on the performance of the security function chain and that there is no single order that outperforms all other orders in every situation. The proposed proof-of-concept framework successfully validates the feasibility of attack-aware security function chain reordering, and we propose additional extensions to eliminate the remaining deficiencies.
• 
  Beyond-local neural information processing in neuronal networks. Balkenhol, Johannes; Händel, Barbara; Biswas, Sounak; Grohmann, Johannes; v. Kistowski, Jóakim; Prada, Juan; Bosman, Conrado A.; Ehrenreich, Hannelore; Wojcik, Sonja M.; Kounev, Samuel; Blum, Robert; Dandekar, Thomas; in Computational and Structural Biotechnology Journal (2024).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  While there is much knowledge about local neuronal circuitry, considerably less is known about how neuronal input is integrated and combined across neuronal networks to encode higher order brain functions. One challenge lies in the large number of complex neural interactions. Neural networks use oscillating activity for information exchange between distributed nodes. To better understand building principles underlying the observation of synchronized oscillatory activity in a large-scale network, we developed a reductionistic neuronal network model. Fundamental building principles are laterally and temporally interconnected virtual nodes (microcircuits), wherein each node was modeled as a local oscillator. By this building principle, the neuronal network model can integrate information in time and space. The simulation gives rise to a wave interference pattern that spreads over all simulated columns in form of a travelling wave. The model design stabilizes states of efficient information processing across all participating neuronal equivalents. Model-specific oscillatory patterns, generated by complex input stimuli, were similar to electrophysiological high-frequency signals that we could confirm in the primate visual cortex during a visual perception task. Important oscillatory model pre-runners, limitations and strength of our reductionistic model are discussed. Our simple scalable model shows unique integration properties and successfully reproduces a variety of biological phenomena such as harmonics, coherence patterns, frequency-speed relationships, and oscillatory activities. We suggest that our scalable model simulates aspects of a basic building principle underlying oscillatory, large-scale integration of information in small and large brains.

  @article{BALKENHOL2024, abstract = {While there is much knowledge about local neuronal circuitry, considerably less is known about how neuronal input is integrated and combined across neuronal networks to encode higher order brain functions. One challenge lies in the large number of complex neural interactions. Neural networks use oscillating activity for information exchange between distributed nodes. To better understand building principles underlying the observation of synchronized oscillatory activity in a large-scale network, we developed a reductionistic neuronal network model. Fundamental building principles are laterally and temporally interconnected virtual nodes (microcircuits), wherein each node was modeled as a local oscillator. By this building principle, the neuronal network model can integrate information in time and space. The simulation gives rise to a wave interference pattern that spreads over all simulated columns in form of a travelling wave. The model design stabilizes states of efficient information processing across all participating neuronal equivalents. Model-specific oscillatory patterns, generated by complex input stimuli, were similar to electrophysiological high-frequency signals that we could confirm in the primate visual cortex during a visual perception task. Important oscillatory model pre-runners, limitations and strength of our reductionistic model are discussed. Our simple scalable model shows unique integration properties and successfully reproduces a variety of biological phenomena such as harmonics, coherence patterns, frequency-speed relationships, and oscillatory activities. We suggest that our scalable model simulates aspects of a basic building principle underlying oscillatory, large-scale integration of information in small and large brains.}, author = {Balkenhol, Johannes and Händel, Barbara and Biswas, Sounak and Grohmann, Johannes and v. Kistowski, Jóakim and Prada, Juan and Bosman, Conrado A. and Ehrenreich, Hannelore and Wojcik, Sonja M. and Kounev, Samuel and Blum, Robert and Dandekar, Thomas}, journal = {Computational and Structural Biotechnology Journal}, keywords = {t_journalmagazine}, title = {Beyond-local neural information processing in neuronal networks}, year = 2024 }

  %0 Journal Article %1 BALKENHOL2024 %A Balkenhol, Johannes %A Händel, Barbara %A Biswas, Sounak %A Grohmann, Johannes %A v. Kistowski, Jóakim %A Prada, Juan %A Bosman, Conrado A. %A Ehrenreich, Hannelore %A Wojcik, Sonja M. %A Kounev, Samuel %A Blum, Robert %A Dandekar, Thomas %D 2024 %J Computational and Structural Biotechnology Journal %R https://doi.org/10.1016/j.csbj.2024.10.040 %T Beyond-local neural information processing in neuronal networks %U https://www.sciencedirect.com/science/article/pii/S200103702400360X %X While there is much knowledge about local neuronal circuitry, considerably less is known about how neuronal input is integrated and combined across neuronal networks to encode higher order brain functions. One challenge lies in the large number of complex neural interactions. Neural networks use oscillating activity for information exchange between distributed nodes. To better understand building principles underlying the observation of synchronized oscillatory activity in a large-scale network, we developed a reductionistic neuronal network model. Fundamental building principles are laterally and temporally interconnected virtual nodes (microcircuits), wherein each node was modeled as a local oscillator. By this building principle, the neuronal network model can integrate information in time and space. The simulation gives rise to a wave interference pattern that spreads over all simulated columns in form of a travelling wave. The model design stabilizes states of efficient information processing across all participating neuronal equivalents. Model-specific oscillatory patterns, generated by complex input stimuli, were similar to electrophysiological high-frequency signals that we could confirm in the primate visual cortex during a visual perception task. Important oscillatory model pre-runners, limitations and strength of our reductionistic model are discussed. Our simple scalable model shows unique integration properties and successfully reproduces a variety of biological phenomena such as harmonics, coherence patterns, frequency-speed relationships, and oscillatory activities. We suggest that our scalable model simulates aspects of a basic building principle underlying oscillatory, large-scale integration of information in small and large brains.

• 
  Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption. Prantl, Thomas; Zeck, Timo; Horn, Lukas; Iffländer, Lukas; Bauer, André; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in Journal of Surveillance, Security and Safety (2023).
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  @article{PrZeHoIfBaDmKrKo_ABESurvey, author = {Prantl, Thomas and Zeck, Timo and Horn, Lukas and Iffl\"ander, Lukas and Bauer, Andr\'e and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, journal = {Journal of Surveillance, Security and Safety}, keywords = {t_journalmagazine}, title = {Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption}, year = 2023 }

  %0 Journal Article %1 PrZeHoIfBaDmKrKo_ABESurvey %A Prantl, Thomas %A Zeck, Timo %A Horn, Lukas %A Iffländer, Lukas %A Bauer, André %A Dmitrienko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %D 2023 %J Journal of Surveillance, Security and Safety %R http://dx.doi.org/10.20517/jsss.2023.30 %T Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption
• 
  A literature review of IoT and CPS—What they are, and what they are not. Lesch, Veronika; Züfle, Marwin; Bauer, André; Iffländer, Lukas; Krupitzer, Christian; Kounev, Samuel; in Journal of Systems and Software (2023). 111631. Elsevier.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.

  @article{lesch2023literature, abstract = {Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.}, author = {Lesch, Veronika and Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Iffl{\"a}nder, Lukas and Krupitzer, Christian and Kounev, Samuel}, journal = {Journal of Systems and Software}, keywords = {t_journalmagazine}, pages = 111631, publisher = {Elsevier}, title = {A literature review of IoT and CPS—What they are, and what they are not}, year = 2023 }

  %0 Journal Article %1 lesch2023literature %A Lesch, Veronika %A Züfle, Marwin %A Bauer, André %A Iffländer, Lukas %A Krupitzer, Christian %A Kounev, Samuel %D 2023 %I Elsevier %J Journal of Systems and Software %P 111631 %T A literature review of IoT and CPS—What they are, and what they are not %U https://www.sciencedirect.com/science/article/pii/S0164121223000262 %X Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.
• 
  Recommendation of Secure Group Communication Schemes using Multi Objective Optimization. Prantl, Thomas; Bauer, André; Iffländer, Lukas; Krupitzer, Christian; Kounev, Samuel; in International Journal of Information Security (2023).
  • [ BibTeX ]
  • [ EndNote ]
  @article{PrBaIfKrKo_IJoIS_SGC_Pareto, author = {Prantl, Thomas and Bauer, André and Iffländer, Lukas and Krupitzer, Christian and Kounev, Samuel}, journal = {International Journal of Information Security}, keywords = {t_journalmagazine}, title = {Recommendation of Secure Group Communication Schemes using Multi Objective Optimization}, year = 2023 }

  %0 Journal Article %1 PrBaIfKrKo_IJoIS_SGC_Pareto %A Prantl, Thomas %A Bauer, André %A Iffländer, Lukas %A Krupitzer, Christian %A Kounev, Samuel %D 2023 %J International Journal of Information Security %T Recommendation of Secure Group Communication Schemes using Multi Objective Optimization
• 
  Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field. Bauer, André; Leznik, Mark; Stenger, Michael; Leppich, Robert; Herbst, Nikolas; Kounev, Samuel; Foster, Ian; (2023).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @misc{bauer2023telescope, author = {Bauer, André and Leznik, Mark and Stenger, Michael and Leppich, Robert and Herbst, Nikolas and Kounev, Samuel and Foster, Ian}, keywords = {t_journalmagazine}, title = {Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field}, year = 2023 }

  %0 Generic %1 bauer2023telescope %A Bauer, André %A Leznik, Mark %A Stenger, Michael %A Leppich, Robert %A Herbst, Nikolas %A Kounev, Samuel %A Foster, Ian %D 2023 %T Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field
• 
  Deep Neural Network Regression for Normalized Digital Surface Model Generation with Sentinel-2 Imagery. Müller, Konstantin; Leppich, Robert; Geiß, Christian; Borst, Vanessa; Pelizari, Patrick Aravena; Kounev, Samuel; in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (2023). 1–14.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  In recent history, normalized digital surface models (nDSMs) have been constantly gaining importance as a means to solve large-scale geographic problems. High-resolution surface models are precious, as they can provide detailed information for a specific area. However, measurements with a high resolution are time-consuming and costly. Only a few approaches exist to create high-resolution normalized digital surface models for extensive areas. This paper explores approaches to extract high-resolution nDSMs from low-resolution Sentinel-2 data, allowing us to derive large-scale models. We thereby utilize the advantages of Sentinel 2 being open access, having global coverage, and providing steady updates through a high repetition rate. Several deep-learning models are trained to overcome the gap in producing high-resolution surface maps from low-resolution input data. With U-Net as a base architecture, we extend the capabilities of our model by integrating tailored multiscale encoders with differently sized kernels in the convolution as well as conformed self-attention inside the skip connection gates. Using pixel-wise regression, our U-Net base models can achieve a mean height error of approximately two meters. Moreover, through our enhancements to the model architecture, we reduce the model error by more than seven percent.

  @article{10189905, abstract = {In recent history, normalized digital surface models (nDSMs) have been constantly gaining importance as a means to solve large-scale geographic problems. High-resolution surface models are precious, as they can provide detailed information for a specific area. However, measurements with a high resolution are time-consuming and costly. Only a few approaches exist to create high-resolution normalized digital surface models for extensive areas. This paper explores approaches to extract high-resolution nDSMs from low-resolution Sentinel-2 data, allowing us to derive large-scale models. We thereby utilize the advantages of Sentinel 2 being open access, having global coverage, and providing steady updates through a high repetition rate. Several deep-learning models are trained to overcome the gap in producing high-resolution surface maps from low-resolution input data. With U-Net as a base architecture, we extend the capabilities of our model by integrating tailored multiscale encoders with differently sized kernels in the convolution as well as conformed self-attention inside the skip connection gates. Using pixel-wise regression, our U-Net base models can achieve a mean height error of approximately two meters. Moreover, through our enhancements to the model architecture, we reduce the model error by more than seven percent.}, author = {Müller, Konstantin and Leppich, Robert and Geiß, Christian and Borst, Vanessa and Pelizari, Patrick Aravena and Kounev, Samuel}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, keywords = {t_journalmagazine}, pages = {1-14}, title = {Deep Neural Network Regression for Normalized Digital Surface Model Generation with Sentinel-2 Imagery}, year = 2023 }

  %0 Journal Article %1 10189905 %A Müller, Konstantin %A Leppich, Robert %A Geiß, Christian %A Borst, Vanessa %A Pelizari, Patrick Aravena %A Kounev, Samuel %D 2023 %J IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing %P 1-14 %T Deep Neural Network Regression for Normalized Digital Surface Model Generation with Sentinel-2 Imagery %X In recent history, normalized digital surface models (nDSMs) have been constantly gaining importance as a means to solve large-scale geographic problems. High-resolution surface models are precious, as they can provide detailed information for a specific area. However, measurements with a high resolution are time-consuming and costly. Only a few approaches exist to create high-resolution normalized digital surface models for extensive areas. This paper explores approaches to extract high-resolution nDSMs from low-resolution Sentinel-2 data, allowing us to derive large-scale models. We thereby utilize the advantages of Sentinel 2 being open access, having global coverage, and providing steady updates through a high repetition rate. Several deep-learning models are trained to overcome the gap in producing high-resolution surface maps from low-resolution input data. With U-Net as a base architecture, we extend the capabilities of our model by integrating tailored multiscale encoders with differently sized kernels in the convolution as well as conformed self-attention inside the skip connection gates. Using pixel-wise regression, our U-Net base models can achieve a mean height error of approximately two meters. Moreover, through our enhancements to the model architecture, we reduce the model error by more than seven percent.
• 
  Optimizing storage assignment, order picking, and their interaction in mezzanine warehouses. Lesch, Veronika; Müller, Patrick B.M.; Krämer, Moritz; Hadry, Marius; Kounev, Samuel; Krupitzer, Christian; in Applied intelligence (2023). 25.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  @article{lesch2023optimizing, author = {Lesch, Veronika and Müller, Patrick B.M. and Krämer, Moritz and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, journal = {Applied intelligence}, keywords = {t_journalmagazine}, month = {02}, pages = 25, title = {Optimizing storage assignment, order picking, and their interaction in mezzanine warehouses}, year = 2023 }

  %0 Journal Article %1 lesch2023optimizing %A Lesch, Veronika %A Müller, Patrick B.M. %A Krämer, Moritz %A Hadry, Marius %A Kounev, Samuel %A Krupitzer, Christian %D 2023 %J Applied intelligence %P 25 %R 10.1007/s10489-022-04443-x %T Optimizing storage assignment, order picking, and their interaction in mezzanine warehouses %U https://doi.org/10.1007/s10489-022-04443-x
• 
  MVNOCoreSim: A Digital Twin for Virtualized IoT-centric Mobile Core Networks. Geissler, Stefan; Wamser, Florian; Bauer, Wolfgang; Gebert, Steffen; Kounev, Samuel; Hossfeld, Tobias; in IEEE Internet of Things (2023). 10(15) 13974–13987.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  The rapid growth of connected devices has led to the implementation of IoT-centric platforms designed to provide connectivity for machine-to-machine type communication. Specifically, mobile virtual network operators (MVNOs) provide global service for IoT use cases by leveraging roaming in readily deployed physical networks. This global deployment of connected devices poses a significant challenge regarding the operation of centralized core networks with respect to dimensioning, scaling as well as system survivability in case of signaling incidents. Overload control mechanisms for IoT mobile networks offer a proactive solution for mitigating excessive signaling traffic from IoT devices in mobile core networks. However, global scaling and a heterogeneous composition of devices present network operators with additional challenges. To address these challenges, this work presents a detailed, protocol level simulation framework of a real-world IoT MVNO core network. We present both models for the expected IoT signaling load and the processing of signaling messages based on measurements in a live, production environment as well as a dedicated testbed. Finally, we present a case study on various overload mechanisms and identify critical performance characteristics to compare their performance. The results of this study categorize overload control mechanisms and shed light on the necessity for MVNOs to deal with the upcoming IoT traffic by defining appropriate overload control mechanisms in mobile core networks to ensure network survivability under unforeseen conditions.

  @article{geissler2023mvnocoresim, abstract = {The rapid growth of connected devices has led to the implementation of IoT-centric platforms designed to provide connectivity for machine-to-machine type communication. Specifically, mobile virtual network operators (MVNOs) provide global service for IoT use cases by leveraging roaming in readily deployed physical networks. This global deployment of connected devices poses a significant challenge regarding the operation of centralized core networks with respect to dimensioning, scaling as well as system survivability in case of signaling incidents. Overload control mechanisms for IoT mobile networks offer a proactive solution for mitigating excessive signaling traffic from IoT devices in mobile core networks. However, global scaling and a heterogeneous composition of devices present network operators with additional challenges. To address these challenges, this work presents a detailed, protocol level simulation framework of a real-world IoT MVNO core network. We present both models for the expected IoT signaling load and the processing of signaling messages based on measurements in a live, production environment as well as a dedicated testbed. Finally, we present a case study on various overload mechanisms and identify critical performance characteristics to compare their performance. The results of this study categorize overload control mechanisms and shed light on the necessity for MVNOs to deal with the upcoming IoT traffic by defining appropriate overload control mechanisms in mobile core networks to ensure network survivability under unforeseen conditions.}, author = {Geissler, Stefan and Wamser, Florian and Bauer, Wolfgang and Gebert, Steffen and Kounev, Samuel and Hossfeld, Tobias}, journal = {IEEE Internet of Things}, keywords = {t_journalmagazine}, month = {08}, number = 15, pages = {13974-13987}, title = {MVNOCoreSim: A Digital Twin for Virtualized IoT-centric Mobile Core Networks}, volume = 10, year = 2023 }

  %0 Journal Article %1 geissler2023mvnocoresim %A Geissler, Stefan %A Wamser, Florian %A Bauer, Wolfgang %A Gebert, Steffen %A Kounev, Samuel %A Hossfeld, Tobias %D 2023 %J IEEE Internet of Things %N 15 %P 13974-13987 %R 10.1109/JIOT.2023.3263258 %T MVNOCoreSim: A Digital Twin for Virtualized IoT-centric Mobile Core Networks %U https://ieeexplore.ieee.org/document/10089151 %V 10 %X The rapid growth of connected devices has led to the implementation of IoT-centric platforms designed to provide connectivity for machine-to-machine type communication. Specifically, mobile virtual network operators (MVNOs) provide global service for IoT use cases by leveraging roaming in readily deployed physical networks. This global deployment of connected devices poses a significant challenge regarding the operation of centralized core networks with respect to dimensioning, scaling as well as system survivability in case of signaling incidents. Overload control mechanisms for IoT mobile networks offer a proactive solution for mitigating excessive signaling traffic from IoT devices in mobile core networks. However, global scaling and a heterogeneous composition of devices present network operators with additional challenges. To address these challenges, this work presents a detailed, protocol level simulation framework of a real-world IoT MVNO core network. We present both models for the expected IoT signaling load and the processing of signaling messages based on measurements in a live, production environment as well as a dedicated testbed. Finally, we present a case study on various overload mechanisms and identify critical performance characteristics to compare their performance. The results of this study categorize overload control mechanisms and shed light on the necessity for MVNOs to deal with the upcoming IoT traffic by defining appropriate overload control mechanisms in mobile core networks to ensure network survivability under unforeseen conditions.
• 
  Self-Aware Optimization of Adaptation Planning Strategies. Lesch, Veronika; Hadry, Marius; Krupitzer, Christian; Kounev, Samuel; in ACM Transactions on Autonomous and Adaptive Systems (2023). 18(3)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  In today’s world, circumstances, processes, and requirements for software systems are becoming increasingly complex. In order to operate properly in such dynamic environments, software systems must adapt to these changes, which has led to the research area of Self-Adaptive Systems (SAS). Platooning is one example of adaptive systems in Intelligent Transportation Systems, which is the ability of vehicles to travel with close inter-vehicle distances. This technology leads to an increase in road throughput and safety, which directly addresses the increased infrastructure needs due to increased traffic on the roads. However, the No-Free-Lunch theorem states that the performance of one adaptation planning strategy is not necessarily transferable to other problems. Moreover, especially in the field of SAS, the selection of the most appropriate strategy depends on the current situation of the system. In this paper, we address the problem of self-aware optimization of adaptation planning strategies by designing a framework that includes situation detection, strategy selection, and parameter optimization of the selected strategies. We apply our approach on the case study platooning coordination and evaluate the performance of the proposed framework.

  @article{lesch2022selfaware, abstract = {In today’s world, circumstances, processes, and requirements for software systems are becoming increasingly complex. In order to operate properly in such dynamic environments, software systems must adapt to these changes, which has led to the research area of Self-Adaptive Systems (SAS). Platooning is one example of adaptive systems in Intelligent Transportation Systems, which is the ability of vehicles to travel with close inter-vehicle distances. This technology leads to an increase in road throughput and safety, which directly addresses the increased infrastructure needs due to increased traffic on the roads. However, the No-Free-Lunch theorem states that the performance of one adaptation planning strategy is not necessarily transferable to other problems. Moreover, especially in the field of SAS, the selection of the most appropriate strategy depends on the current situation of the system. In this paper, we address the problem of self-aware optimization of adaptation planning strategies by designing a framework that includes situation detection, strategy selection, and parameter optimization of the selected strategies. We apply our approach on the case study platooning coordination and evaluate the performance of the proposed framework.}, author = {Lesch, Veronika and Hadry, Marius and Krupitzer, Christian and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {self-awareness}, month = {09}, number = 3, title = {Self-Aware Optimization of Adaptation Planning Strategies}, volume = 18, year = 2023 }

  %0 Journal Article %1 lesch2022selfaware %A Lesch, Veronika %A Hadry, Marius %A Krupitzer, Christian %A Kounev, Samuel %D 2023 %J ACM Transactions on Autonomous and Adaptive Systems %N 3 %R 10.1145/3568680 %T Self-Aware Optimization of Adaptation Planning Strategies %U https://doi.org/10.1145/3568680 %V 18 %X In today’s world, circumstances, processes, and requirements for software systems are becoming increasingly complex. In order to operate properly in such dynamic environments, software systems must adapt to these changes, which has led to the research area of Self-Adaptive Systems (SAS). Platooning is one example of adaptive systems in Intelligent Transportation Systems, which is the ability of vehicles to travel with close inter-vehicle distances. This technology leads to an increase in road throughput and safety, which directly addresses the increased infrastructure needs due to increased traffic on the roads. However, the No-Free-Lunch theorem states that the performance of one adaptation planning strategy is not necessarily transferable to other problems. Moreover, especially in the field of SAS, the selection of the most appropriate strategy depends on the current situation of the system. In this paper, we address the problem of self-aware optimization of adaptation planning strategies by designing a framework that includes situation detection, strategy selection, and parameter optimization of the selected strategies. We apply our approach on the case study platooning coordination and evaluate the performance of the proposed framework.
• 
  De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem. Prantl, Thomas; Horn, Lukas; Engel, Simon; Iffländer, Lukas; Beierlieb, Lukas; Krupitzer, Christian; Bauer, André; Sakarvadia, Mansi; Foster, Ian; Kounev, Samuel; in International Journal of Information Security (2023).
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  @article{PrHoEnIfBeKrBaSaFoKo_IJoIS_SGC_Pareto, author = {Prantl, Thomas and Horn, Lukas and Engel, Simon and Iffl\"ander, Lukas and Beierlieb, Lukas and Krupitzer, Christian and Bauer, Andr\'e and Sakarvadia, Mansi and Foster, Ian and Kounev, Samuel}, journal = {International Journal of Information Security}, keywords = {t_journalmagazine}, month = 11, title = {De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem}, year = 2023 }

  %0 Journal Article %1 PrHoEnIfBeKrBaSaFoKo_IJoIS_SGC_Pareto %A Prantl, Thomas %A Horn, Lukas %A Engel, Simon %A Iffländer, Lukas %A Beierlieb, Lukas %A Krupitzer, Christian %A Bauer, André %A Sakarvadia, Mansi %A Foster, Ian %A Kounev, Samuel %D 2023 %J International Journal of Information Security %R https://doi.org/10.1007/s10207-023-00781-0 %T De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem
• 
  Serverless Computing: What It Is, and What It Is Not?. Kounev, Samuel; Herbst, Nikolas; Abad, Cristina L.; Iosup, Alexandru; Foster, Ian; Shenoy, Prashant; Rana, Omer; Chien, Andrew A.; in Commun. ACM (2023). 66(9) 80–92. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Dispelling the confusion around serverless computing by capturing its essential and conceptual characteristics.

  @article{10.1145/3587249, abstract = {Dispelling the confusion around serverless computing by capturing its essential and conceptual characteristics.}, address = {New York, NY, USA}, author = {Kounev, Samuel and Herbst, Nikolas and Abad, Cristina L. and Iosup, Alexandru and Foster, Ian and Shenoy, Prashant and Rana, Omer and Chien, Andrew A.}, journal = {Commun. ACM}, keywords = {t_journalmagazine}, month = {08}, number = 9, pages = {80–92}, publisher = {Association for Computing Machinery}, title = {Serverless Computing: What It Is, and What It Is Not?}, volume = 66, year = 2023 }

  %0 Journal Article %1 10.1145/3587249 %A Kounev, Samuel %A Herbst, Nikolas %A Abad, Cristina L. %A Iosup, Alexandru %A Foster, Ian %A Shenoy, Prashant %A Rana, Omer %A Chien, Andrew A. %C New York, NY, USA %D 2023 %I Association for Computing Machinery %J Commun. ACM %N 9 %P 80–92 %T Serverless Computing: What It Is, and What It Is Not? %U https://doi.org/10.1145/3587249 %V 66 %X Dispelling the confusion around serverless computing by capturing its essential and conceptual characteristics.

• 
  A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions. Henrichs, Elia; Lesch, Veronika; Straesser, Martin; Kounev, Samuel; Krupitzer, Christian; in Information and Software Technology (2022). 149 106940.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.

  @article{HENRICHS2022106940, abstract = {Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.}, author = {Henrichs, Elia and Lesch, Veronika and Straesser, Martin and Kounev, Samuel and Krupitzer, Christian}, journal = {Information and Software Technology}, keywords = {sys:relevantfor:se-group}, pages = 106940, title = {A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions}, volume = 149, year = 2022 }

  %0 Journal Article %1 HENRICHS2022106940 %A Henrichs, Elia %A Lesch, Veronika %A Straesser, Martin %A Kounev, Samuel %A Krupitzer, Christian %D 2022 %J Information and Software Technology %P 106940 %T A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions %U https://www.sciencedirect.com/science/article/pii/S0950584922000891 %V 149 %X Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.
• 
  CortexVR: Immersive analysis and training of cognitive executive functions of soccer players using virtual reality and machine learning. Krupitzer, Christian; Naber, Jens; Stauffert, Jan-Philipp; Mayer, Jan; Spielmann, Jan; Ehmann, Paul; Boci, Noel; Bürkle, Maurice; Ho, André; Komorek, Clemens; Heinickel, Felix; Kounev, Samuel; Becker, Christian; Latoschik, Marc Erich; in Frontiers in Psychology (2022). 5139. Frontiers.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{krupitzer2022cortexvr, author = {Krupitzer, Christian and Naber, Jens and Stauffert, Jan-Philipp and Mayer, Jan and Spielmann, Jan and Ehmann, Paul and Boci, Noel and B{\"u}rkle, Maurice and Ho, Andr{\'e} and Komorek, Clemens and Heinickel, Felix and Kounev, Samuel and Becker, Christian and Latoschik, Marc Erich}, journal = {Frontiers in Psychology}, keywords = {t_journalmagazine}, pages = 5139, publisher = {Frontiers}, title = {CortexVR: Immersive analysis and training of cognitive executive functions of soccer players using virtual reality and machine learning}, year = 2022 }

  %0 Journal Article %1 krupitzer2022cortexvr %A Krupitzer, Christian %A Naber, Jens %A Stauffert, Jan-Philipp %A Mayer, Jan %A Spielmann, Jan %A Ehmann, Paul %A Boci, Noel %A Bürkle, Maurice %A Ho, André %A Komorek, Clemens %A Heinickel, Felix %A Kounev, Samuel %A Becker, Christian %A Latoschik, Marc Erich %D 2022 %I Frontiers %J Frontiers in Psychology %P 5139 %T CortexVR: Immersive analysis and training of cognitive executive functions of soccer players using virtual reality and machine learning
• 
  A case study on the stability of performance tests for serverless applications. Eismann, Simon; Costa, Diego; Liao, Lizhi; Bezemer, Cor-Paul; Shang, Weiyi; van Hoorn, André; Kounev, Samuel; in Journal of Systems and Software (JSS) (2022). 189
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.

  @article{eismann2022study, abstract = {Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.}, author = {Eismann, Simon and Costa, Diego and Liao, Lizhi and Bezemer, Cor-Paul and Shang, Weiyi and van Hoorn, André and Kounev, Samuel}, journal = {Journal of Systems and Software (JSS)}, keywords = {t_journalmagazine}, title = {A case study on the stability of performance tests for serverless applications}, volume = 189, year = 2022 }

  %0 Journal Article %1 eismann2022study %A Eismann, Simon %A Costa, Diego %A Liao, Lizhi %A Bezemer, Cor-Paul %A Shang, Weiyi %A van Hoorn, André %A Kounev, Samuel %D 2022 %J Journal of Systems and Software (JSS) %T A case study on the stability of performance tests for serverless applications %U https://www.sciencedirect.com/science/article/abs/pii/S0164121222000498 %V 189 %X Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.
• 
  Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery. Kalisnik, Jurij Matija; Bauer, André; Vogt, Ferdinand Aurel; Stickl, Franziska Josephine; Zibert, Janez; Fittkau, Matthias; Bertsch, Thomas; Kounev, Samuel; Fischlein, Theodor; in European Journal of Cardio-Thoracic Surgery (2022).

  Joint first authorship; To be published soon
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.

  @article{kalisnik2022artificial, abstract = {Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.}, author = {Kalisnik, Jurij Matija and Bauer, Andr{é} and Vogt, Ferdinand Aurel and Stickl, Franziska Josephine and Zibert, Janez and Fittkau, Matthias and Bertsch, Thomas and Kounev, Samuel and Fischlein, Theodor}, journal = {European Journal of Cardio-Thoracic Surgery}, keywords = {t_journalmagazine}, note = {Joint first authorship; To be published soon}, title = {Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery}, year = 2022 }

  %0 Journal Article %1 kalisnik2022artificial %A Kalisnik, Jurij Matija %A Bauer, André %A Vogt, Ferdinand Aurel %A Stickl, Franziska Josephine %A Zibert, Janez %A Fittkau, Matthias %A Bertsch, Thomas %A Kounev, Samuel %A Fischlein, Theodor %D 2022 %J European Journal of Cardio-Thoracic Surgery %T Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery %X Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.
• 
  A Survey on Secure Group Communication Schemes with Focus on IoT Communication. Prantl, Thomas; Zeck, Timo; Bauer, André; Ten, Peter; Prantl, Dominik; Ben Yahya, Ala Eddine; Iffländer, Lukas; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in IEEE Access (2022). IEEE.

  Accepted but not yet published
  • [ BibTeX ]
  • [ EndNote ]
  @article{PrZeBaTePrYaIfDmKrKo_IEEEAccess_SGCSurvey, author = {Prantl, Thomas and Zeck, Timo and Bauer, André and Ten, Peter and Prantl, Dominik and Ben Yahya, Ala Eddine and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, journal = {IEEE Access}, keywords = {t_journalmagazine}, note = {Accepted but not yet published}, publisher = {IEEE}, title = {A Survey on Secure Group Communication Schemes with Focus on IoT Communication}, year = 2022 }

  %0 Journal Article %1 PrZeBaTePrYaIfDmKrKo_IEEEAccess_SGCSurvey %A Prantl, Thomas %A Zeck, Timo %A Bauer, André %A Ten, Peter %A Prantl, Dominik %A Ben Yahya, Ala Eddine %A Iffländer, Lukas %A Dmitrienko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %D 2022 %I IEEE %J IEEE Access %T A Survey on Secure Group Communication Schemes with Focus on IoT Communication

• 
  People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot. Spiess, Florian; Reinhart, Lucas; Strobel, Norbert; Kaiser, Dennis; Kounev, Samuel; Kaupp, Tobias; in Journal of Image and Graphics, (J. of Image; Graphics, eds.) (2021). 9(4) 135–139.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  @article{spiess2021people, author = {Spiess, Florian and Reinhart, Lucas and Strobel, Norbert and Kaiser, Dennis and Kounev, Samuel and Kaupp, Tobias}, editor = {of Image, Journal and Graphics}, journal = {Journal of Image and Graphics}, keywords = {t_journalmagazine}, month = 12, number = 4, pages = {135-139}, title = {People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot}, volume = 9, year = 2021 }

  %0 Journal Article %1 spiess2021people %A Spiess, Florian %A Reinhart, Lucas %A Strobel, Norbert %A Kaiser, Dennis %A Kounev, Samuel %A Kaupp, Tobias %D 2021 %E of Image, Journal %E Graphics, %J Journal of Image and Graphics %N 4 %P 135-139 %T People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot %U http://www.joig.net/index.php?m=content&c=index&a=show&catid=72&id=277 %V 9
• 
  SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation. Grohmann, Johannes; Eismann, Simon; Bauer, André; Spinner, Simon; Blum, Johannes; Herbst, Nikolas; Kounev, Samuel; in ACM Transactions on Autonomous and Adaptive Systems (2021). 15(2) Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

  @article{GrEiBaSpBlHeKo2021-TaaS-SARDE, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andr\'{e} and Spinner, Simon and Blum, Johannes and Herbst, Nikolas and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {t_journalmagazine}, month = {06}, number = 2, publisher = {Association for Computing Machinery}, title = {SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation}, volume = 15, year = 2021 }

  %0 Journal Article %1 GrEiBaSpBlHeKo2021-TaaS-SARDE %A Grohmann, Johannes %A Eismann, Simon %A Bauer, André %A Spinner, Simon %A Blum, Johannes %A Herbst, Nikolas %A Kounev, Samuel %C New York, NY, USA %D 2021 %I Association for Computing Machinery %J ACM Transactions on Autonomous and Adaptive Systems %N 2 %T SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation %U https://doi.org/10.1145/3463369 %V 15 %X Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.
• 
  A Comparison of Mechanisms for Compensating Negative Impacts of System Integration. Lesch, Veronika; Krupitzer, Christian; Stubenrauch, Kevin; Keil, Nico; Becker, Christian; Kounev, Samuel; Segata, Michele; in Future Generation Computer Systems (2021). 116 117–131.
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  • [ Download ]
  @article{lesch2020comparison, author = {Lesch, Veronika and Krupitzer, Christian and Stubenrauch, Kevin and Keil, Nico and Becker, Christian and Kounev, Samuel and Segata, Michele}, journal = {Future Generation Computer Systems}, keywords = {platooning}, month = {03}, pages = {117-131}, title = {A Comparison of Mechanisms for Compensating Negative Impacts of System Integration}, volume = 116, year = 2021 }

  %0 Journal Article %1 lesch2020comparison %A Lesch, Veronika %A Krupitzer, Christian %A Stubenrauch, Kevin %A Keil, Nico %A Becker, Christian %A Kounev, Samuel %A Segata, Michele %D 2021 %J Future Generation Computer Systems %P 117-131 %R https://doi.org/10.1016/j.future.2020.10.029 %T A Comparison of Mechanisms for Compensating Negative Impacts of System Integration %V 116
• 
  A Case Study on Optimization of Warehouses. Lesch, Veronika; Müller, Patrick; Krämer, Moritz; Kounev, Samuel; Krupitzer, Christian; in arXiv preprint arXiv:2112.12058 (2021).
  • [ BibTeX ]
  • [ EndNote ]
  @article{lesch2021case, author = {Lesch, Veronika and M{\"u}ller, Patrick and Kr{\"a}mer, Moritz and Kounev, Samuel and Krupitzer, Christian}, journal = {arXiv preprint arXiv:2112.12058}, keywords = {t_journalmagazine}, title = {A Case Study on Optimization of Warehouses}, year = 2021 }

  %0 Journal Article %1 lesch2021case %A Lesch, Veronika %A Müller, Patrick %A Krämer, Moritz %A Kounev, Samuel %A Krupitzer, Christian %D 2021 %J arXiv preprint arXiv:2112.12058 %T A Case Study on Optimization of Warehouses
• 
  Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms. Lesch, Veronika; König, Maximilian; Kounev, Samuel; Stein, Anthony; Krupitzer, Christian; in Applied Intelligence (2021). 52 9476–9500.
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  @article{lesch2021tackling, author = {Lesch, Veronika and König, Maximilian and Kounev, Samuel and Stein, Anthony and Krupitzer, Christian}, journal = {Applied Intelligence}, keywords = {t_journalmagazine}, pages = {9476–9500}, title = {Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms}, volume = 52, year = 2021 }

  %0 Journal Article %1 lesch2021tackling %A Lesch, Veronika %A König, Maximilian %A Kounev, Samuel %A Stein, Anthony %A Krupitzer, Christian %D 2021 %J Applied Intelligence %P 9476–9500 %R https://doi.org/10.1007/s10489-021-03035-5 %T Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms %V 52
• 
  An Overview on Approaches for Coordination of Platoons. Lesch, Veronika; Breitbach, Martin; Segata, Michele; Becker, Christian; Kounev, Samuel; Krupitzer, Christian; in IEEE Transactions on Intelligent Transportation Systems (2021).

  Early Access on IEEE Xplore
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{lesch2021overview, author = {Lesch, Veronika and Breitbach, Martin and Segata, Michele and Becker, Christian and Kounev, Samuel and Krupitzer, Christian}, journal = {IEEE Transactions on Intelligent Transportation Systems}, keywords = {platooning}, note = {Early Access on IEEE Xplore}, title = {An Overview on Approaches for Coordination of Platoons}, year = 2021 }

  %0 Journal Article %1 lesch2021overview %A Lesch, Veronika %A Breitbach, Martin %A Segata, Michele %A Becker, Christian %A Kounev, Samuel %A Krupitzer, Christian %D 2021 %J IEEE Transactions on Intelligent Transportation Systems %T An Overview on Approaches for Coordination of Platoons
• 
  A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools. Züfle, Marwin; Moog, Felix; Lesch, Veronika; Krupitzer, Christian; Kounev, Samuel; in ISA Transactions: The Journal of Automation (2021).

  (in press)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.

  @article{zuefle2021anomalydetection, abstract = {Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.}, author = {Züfle, Marwin and Moog, Felix and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, journal = {ISA Transactions: The Journal of Automation}, keywords = {t_journalmagazine}, note = {(in press)}, title = {A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools}, year = 2021 }

  %0 Journal Article %1 zuefle2021anomalydetection %A Züfle, Marwin %A Moog, Felix %A Lesch, Veronika %A Krupitzer, Christian %A Kounev, Samuel %D 2021 %J ISA Transactions: The Journal of Automation %T A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools %U https://www.sciencedirect.com/science/article/pii/S0019057821003773 %X Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.

• 
  The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study. Zinner, Christoph; Matzka, Manuel; Leppich, Robert; Kounev, Samuel; Holmberg, HC; Sperlich, Billy; in Frontiers in Sports and Active Living (2020). 2 127. Frontiers.
  • [ BibTeX ]
  • [ EndNote ]
  @article{zinner2020impact, author = {Zinner, Christoph and Matzka, Manuel and Leppich, Robert and Kounev, Samuel and Holmberg, HC and Sperlich, Billy}, journal = {Frontiers in Sports and Active Living}, keywords = {covid19}, pages = 127, publisher = {Frontiers}, title = {The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study}, volume = 2, year = 2020 }

  %0 Journal Article %1 zinner2020impact %A Zinner, Christoph %A Matzka, Manuel %A Leppich, Robert %A Kounev, Samuel %A Holmberg, HC %A Sperlich, Billy %D 2020 %I Frontiers %J Frontiers in Sports and Active Living %P 127 %T The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study %V 2
• 
  A Taxonomy of Techniques for SLO Failure Prediction in Software Systems. Grohmann, Johannes; Herbst, Nikolas; Chalbani, Avi; Arian, Yair; Peretz, Noam; Kounev, Samuel; in Computers (2020). 9(1) 10. Multidisciplinary Digital Publishing Institute (MDPI).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.

  @article{grohmann2020-MDPI-taxonomy, abstract = {Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.}, author = {Grohmann, Johannes and Herbst, Nikolas and Chalbani, Avi and Arian, Yair and Peretz, Noam and Kounev, Samuel}, journal = {Computers}, keywords = {automated_model_learning}, number = 1, pages = 10, publisher = {Multidisciplinary Digital Publishing Institute (MDPI)}, title = {A Taxonomy of Techniques for SLO Failure Prediction in Software Systems}, volume = 9, year = 2020 }

  %0 Journal Article %1 grohmann2020-MDPI-taxonomy %A Grohmann, Johannes %A Herbst, Nikolas %A Chalbani, Avi %A Arian, Yair %A Peretz, Noam %A Kounev, Samuel %D 2020 %I Multidisciplinary Digital Publishing Institute (MDPI) %J Computers %N 1 %P 10 %R https://doi.org/10.3390/computers9010010 %T A Taxonomy of Techniques for SLO Failure Prediction in Software Systems %U https://doi.org/10.3390/computers9010010 %V 9 %X Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.
• 
  Towards Self-Aware Multirotor Formations. Kaiser, Dennis; Lesch, Veronika; Rothe, Julian; Strohmeier, Michael; Spiess, Florian; Krupitzer, Christian; Montenegro, Sergio; Kounev, Samuel; in Computers (2020). 9(7) MDPI.

  Special Issue on Self-Aware Computing
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{Kaiser2020Towards, author = {Kaiser, Dennis and Lesch, Veronika and Rothe, Julian and Strohmeier, Michael and Spiess, Florian and Krupitzer, Christian and Montenegro, Sergio and Kounev, Samuel}, journal = {Computers}, keywords = {Intelligent_Transportation_Systems}, month = {02}, note = {Special Issue on Self-Aware Computing}, number = 7, publisher = {MDPI}, title = {{Towards Self-Aware Multirotor Formations}}, volume = 9, year = 2020 }

  %0 Journal Article %1 Kaiser2020Towards %A Kaiser, Dennis %A Lesch, Veronika %A Rothe, Julian %A Strohmeier, Michael %A Spiess, Florian %A Krupitzer, Christian %A Montenegro, Sergio %A Kounev, Samuel %D 2020 %I MDPI %J Computers %N 7 %T Towards Self-Aware Multirotor Formations %V 9
• 
  Time Series Forecasting for Self-Aware Systems. Bauer, André; Züfle, Marwin; Herbst, Nikolas; Zehe, Albin; Hotho, Andreas; Kounev, Samuel; in Proceedings of the IEEE (2020). 108(7) 1068–1093.
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  @article{BaZuHeZeHoKo-PIEEE-Forecasting, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Zehe, Albin and Hotho, Andreas and Kounev, Samuel}, journal = {Proceedings of the IEEE}, keywords = {t_journalmagazine}, month = {07}, number = 7, pages = {1068 - 1093}, title = {{Time Series Forecasting for Self-Aware Systems}}, volume = 108, year = 2020 }

  %0 Journal Article %1 BaZuHeZeHoKo-PIEEE-Forecasting %A Bauer, André %A Züfle, Marwin %A Herbst, Nikolas %A Zehe, Albin %A Hotho, Andreas %A Kounev, Samuel %D 2020 %J Proceedings of the IEEE %N 7 %P 1068 - 1093 %R 10.1109/JPROC.2020.2983857 %T Time Series Forecasting for Self-Aware Systems %V 108
• 
  Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data. Spiess, Florian; Friesslich, Jonas; Tobias, Kaupp; Kounev, Samuel; Strobel, Norbert; in International Journal of Mechanical Engineering and Robotics Research (IJMERR), (P. R. (Chunhui) Yang, ed.) (2020). 9(12) 1532–1540.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.

  @article{yang2020survey, abstract = {This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.}, author = {Spiess, Florian and Friesslich, Jonas and Tobias, Kaupp and Kounev, Samuel and Strobel, Norbert}, editor = {Yang, Prof Richard (Chunhui)}, journal = {International Journal of Mechanical Engineering and Robotics Research (IJMERR)}, keywords = {rgb}, month = 12, number = 12, pages = {1532-1540}, title = {Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data}, volume = 9, year = 2020 }

  %0 Journal Article %1 yang2020survey %A Spiess, Florian %A Friesslich, Jonas %A Tobias, Kaupp %A Kounev, Samuel %A Strobel, Norbert %D 2020 %E Yang, Prof Richard (Chunhui) %J International Journal of Mechanical Engineering and Robotics Research (IJMERR) %N 12 %P 1532-1540 %T Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data %U http://www.ijmerr.com/index.php?m=content&c=index&a=show&catid=185&id=1529 %V 9 %X This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.

• 
  Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field. Bauer, André; Herbst, Nikolas; Spinner, Simon; Ali-Eldin, Ahmed; Kounev, Samuel; in IEEE Transactions on Parallel and Distributed Systems (2019). 30(4) 800–813. IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{BaHeSpAEKo2019-TPDS-ChameleonAutoscaler, author = {Bauer, Andr{\'e} and Herbst, Nikolas and Spinner, Simon and Ali-Eldin, Ahmed and Kounev, Samuel}, journal = {IEEE Transactions on Parallel and Distributed Systems}, keywords = {Self-adaptive-systems}, month = {04}, number = 4, pages = {800–813}, publisher = {IEEE}, title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}}, volume = 30, year = 2019 }

  %0 Journal Article %1 BaHeSpAEKo2019-TPDS-ChameleonAutoscaler %A Bauer, André %A Herbst, Nikolas %A Spinner, Simon %A Ali-Eldin, Ahmed %A Kounev, Samuel %D 2019 %I IEEE %J IEEE Transactions on Parallel and Distributed Systems %N 4 %P 800--813 %T Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field %V 30
• 
  Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud. Metzger, Florian; Hoßfeld, Tobias; Bauer, André; Kounev, Samuel; Heegaard, Poul. E.; in Proceedings of the IEEE (2019). 107(4) 679–694.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{MeHoBaKoHe2019-PIEEE-IoTTraffic, author = {Metzger, Florian and Ho{\ss}feld, Tobias and Bauer, Andr{\'e} and Kounev, Samuel and Heegaard, Poul. E.}, journal = {Proceedings of the IEEE}, keywords = {t_journalmagazine}, month = {04}, number = 4, pages = {679-694}, title = {{Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud}}, volume = 107, year = 2019 }

  %0 Journal Article %1 MeHoBaKoHe2019-PIEEE-IoTTraffic %A Metzger, Florian %A Hoßfeld, Tobias %A Bauer, André %A Kounev, Samuel %A Heegaard, Poul. E. %D 2019 %J Proceedings of the IEEE %N 4 %P 679-694 %T Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud %V 107
• 
  Online model learning for self-aware computing infrastructures. Spinner, Simon; Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in Journal of Systems and Software (2019). 147 1–16.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.

  @article{SpGrEiKo2019-JSS-ModelLearning, abstract = {Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.}, author = {Spinner, Simon and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, journal = {Journal of Systems and Software}, keywords = {Virtualization}, pages = {1–16}, title = {Online model learning for self-aware computing infrastructures}, volume = 147, year = 2019 }

  %0 Journal Article %1 SpGrEiKo2019-JSS-ModelLearning %A Spinner, Simon %A Grohmann, Johannes %A Eismann, Simon %A Kounev, Samuel %D 2019 %J Journal of Systems and Software %P 1--16 %R 10.1016/j.jss.2018.09.089 %T Online model learning for self-aware computing infrastructures %U https://doi.org/10.1016/j.jss.2018.09.089 %V 147 %X Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.
• 
  Measuring and Rating the Energy-Efficiency of Servers. von Kistowski, Jóakim; Lange, Klaus-Dieter; Arnold, Jeremy A.; Beckett, John; Block, Hansfried; Tricker, Mike; Sharma, Sanjay; Pais, Johann; Kounev, Samuel; in Future Generation Computer Systems (2019). 100 579–589.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.

  @article{KiLaArBeBlTrShPaKo-FGCS2019-ServerEfficiency, abstract = {Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04\% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.}, author = {von Kistowski, J{\'o}akim and Lange, Klaus-Dieter and Arnold, Jeremy A. and Beckett, John and Block, Hansfried and Tricker, Mike and Sharma, Sanjay and Pais, Johann and Kounev, Samuel}, journal = {Future Generation Computer Systems}, keywords = {Power}, pages = {579 - 589}, title = {{Measuring and Rating the Energy-Efficiency of Servers}}, volume = 100, year = 2019 }

  %0 Journal Article %1 KiLaArBeBlTrShPaKo-FGCS2019-ServerEfficiency %A von Kistowski, Jóakim %A Lange, Klaus-Dieter %A Arnold, Jeremy A. %A Beckett, John %A Block, Hansfried %A Tricker, Mike %A Sharma, Sanjay %A Pais, Johann %A Kounev, Samuel %D 2019 %J Future Generation Computer Systems %P 579 - 589 %T Measuring and Rating the Energy-Efficiency of Servers %U http://www.sciencedirect.com/science/article/pii/S0167739X18312123 %V 100 %X Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.

• 
  Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges. Herbst, Nikolas; Bauer, André; Kounev, Samuel; Oikonomou, Giorgos; van Eyk, Erwin; Kousiouris, George; Evangelinou, Athanasia; Krebs, Rouven; Brecht, Tim; Abad, Cristina L.; Iosup, Alexandru; in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) (2018). 3(4) 19:1–19:36. ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @article{HeBaKoOiEyKoEvKrBrAbIo2018-TOMPECS-CloudMetrics, address = {New York, NY, USA}, author = {Herbst, Nikolas and Bauer, Andr{\'e} and Kounev, Samuel and Oikonomou, Giorgos and van Eyk, Erwin and Kousiouris, George and Evangelinou, Athanasia and Krebs, Rouven and Brecht, Tim and Abad, Cristina L. and Iosup, Alexandru}, journal = {ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)}, keywords = {SPEC}, month = {08}, number = 4, pages = {19:1–19:36}, publisher = {ACM}, title = {{Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges}}, volume = 3, year = 2018 }

  %0 Journal Article %1 HeBaKoOiEyKoEvKrBrAbIo2018-TOMPECS-CloudMetrics %A Herbst, Nikolas %A Bauer, André %A Kounev, Samuel %A Oikonomou, Giorgos %A van Eyk, Erwin %A Kousiouris, George %A Evangelinou, Athanasia %A Krebs, Rouven %A Brecht, Tim %A Abad, Cristina L. %A Iosup, Alexandru %C New York, NY, USA %D 2018 %I ACM %J ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) %N 4 %P 19:1--19:36 %T Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges %U http://doi.acm.org/10.1145/3236332 %V 3
• 
  CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts. Milenkoski, Aleksandar; Iosup, Alexandru; Kounev, Samuel; Sachs, Kai; Mularz, Diane E.; Curtiss, Jonathan A.; Ding, Jason J.; Rosenberg, Florian; Rygielski, Piotr; in IEEE Cloud Computing (2018). 5(3) 65–76. IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  @article{MiIoKoSaMuCuDiRoRy2018-IEEECC-cup, author = {Milenkoski, Aleksandar and Iosup, Alexandru and Kounev, Samuel and Sachs, Kai and Mularz, Diane E. and Curtiss, Jonathan A. and Ding, Jason J. and Rosenberg, Florian and Rygielski, Piotr}, journal = {IEEE Cloud Computing}, keywords = {Virtualization}, month = {06}, number = 3, pages = {65–76}, publisher = {IEEE}, title = {{CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts}}, volume = 5, year = 2018 }

  %0 Journal Article %1 MiIoKoSaMuCuDiRoRy2018-IEEECC-cup %A Milenkoski, Aleksandar %A Iosup, Alexandru %A Kounev, Samuel %A Sachs, Kai %A Mularz, Diane E. %A Curtiss, Jonathan A. %A Ding, Jason J. %A Rosenberg, Florian %A Rygielski, Piotr %D 2018 %I IEEE %J IEEE Cloud Computing %N 3 %P 65--76 %T CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts %V 5

• 
  Modeling and Extracting Load Intensity Profiles. von Kistowski, Jóakim; Herbst, Nikolas; Kounev, Samuel; Groenda, Henning; Stier, Christian; Lehrig, Sebastian; in ACM Transactions on Autonomous and Adaptive Systems (TAAS) (2017). 11(4) 23:1–23:28. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.

  @article{KiHeKo-TAAS17-ModelExtractLoadProfiles, abstract = {Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2\% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas and Kounev, Samuel and Groenda, Henning and Stier, Christian and Lehrig, Sebastian}, journal = {ACM Transactions on Autonomous and Adaptive Systems (TAAS)}, keywords = {Self-adaptive-systems}, month = {01}, number = 4, pages = {23:1–23:28}, publisher = {ACM}, title = {{Modeling and Extracting Load Intensity Profiles}}, volume = 11, year = 2017 }

  %0 Journal Article %1 KiHeKo-TAAS17-ModelExtractLoadProfiles %A von Kistowski, Jóakim %A Herbst, Nikolas %A Kounev, Samuel %A Groenda, Henning %A Stier, Christian %A Lehrig, Sebastian %C New York, NY, USA %D 2017 %I ACM %J ACM Transactions on Autonomous and Adaptive Systems (TAAS) %N 4 %P 23:1--23:28 %T Modeling and Extracting Load Intensity Profiles %U http://doi.acm.org/10.1145/3019596 %V 11 %X Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.
• 
  Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language. Huber, Nikolaus; Brosig, Fabian; Spinner, Simon; Kounev, Samuel; Bähr, Manuel; in IEEE Transactions on Software Engineering (TSE) (2017). 43(5) 432–452. IEEE Computer Society.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.

  @article{HuBrSpKoBa2017-TSE-DML, abstract = {Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.}, author = {Huber, Nikolaus and Brosig, Fabian and Spinner, Simon and Kounev, Samuel and B{\"a}hr, Manuel}, journal = {IEEE Transactions on Software Engineering (TSE)}, keywords = {DQL}, number = 5, pages = {432–452}, publisher = {IEEE Computer Society}, title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}}, volume = 43, year = 2017 }

  %0 Journal Article %1 HuBrSpKoBa2017-TSE-DML %A Huber, Nikolaus %A Brosig, Fabian %A Spinner, Simon %A Kounev, Samuel %A Bähr, Manuel %D 2017 %I IEEE Computer Society %J IEEE Transactions on Software Engineering (TSE) %N 5 %P 432--452 %T Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language %U http://dx.doi.org/10.1109/TSE.2016.2613863 %V 43 %X Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.

• 
  A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures. Kounev, Samuel; Huber, Nikolaus; Brosig, Fabian; Zhu, Xiaoyun; in IEEE Computer (2016). 49(7) 53–61. IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @article{KoHuBrZh2016-IEEEComputer-Descartes, author = {Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian and Zhu, Xiaoyun}, journal = {IEEE Computer}, keywords = {Self-adaptive-systems}, month = {07}, number = 7, pages = {53-61}, publisher = {IEEE}, title = {{A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures}}, volume = 49, year = 2016 }

  %0 Journal Article %1 KoHuBrZh2016-IEEEComputer-Descartes %A Kounev, Samuel %A Huber, Nikolaus %A Brosig, Fabian %A Zhu, Xiaoyun %D 2016 %I IEEE %J IEEE Computer %N 7 %P 53-61 %T A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures %U http://dx.doi.org/10.1109/MC.2016.198 %V 49
• 
  Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation. Müller, Christoph; Rygielski, Piotr; Spinner, Simon; Kounev, Samuel; in Electronic Notes in Theoretical Computer Science (2016). 327 71–91. Elsevier.

  The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Abstract Due to the growing size of modern IT} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN} models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

  @article{MuRySpKo2016-PASM-QPN-LQN-Transformation, abstract = {Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.}, author = {M{\"u}ller, Christoph and Rygielski, Piotr and Spinner, Simon and Kounev, Samuel}, journal = {Electronic Notes in Theoretical Computer Science}, keywords = {QPN}, note = {The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016}, pages = {71–91}, publisher = {Elsevier}, title = {{Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}}, volume = 327, year = 2016 }

  %0 Journal Article %1 MuRySpKo2016-PASM-QPN-LQN-Transformation %A Müller, Christoph %A Rygielski, Piotr %A Spinner, Simon %A Kounev, Samuel %D 2016 %I Elsevier %J Electronic Notes in Theoretical Computer Science %P 71--91 %T Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation %U http://www.sciencedirect.com/science/article/pii/S157106611630069X %V 327 %X Abstract Due to the growing size of modern IT} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN} models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

• 
  Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices. Milenkoski, Aleksandar; Vieira, Marco; Kounev, Samuel; Avritzer, Alberto; Payne, Bryan D.; in ACM Computing Surveys (2015). 48(1) 12:1–12:41. ACM, New York, NY, USA.

  5-year Impact Factor (2014): 5.949
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.

  @article{MiViKoAvPa2015-CSUR-IDSEval, abstract = {{The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.}}, address = {New York, NY, USA}, author = {Milenkoski, Aleksandar and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Payne, Bryan D.}, journal = {{ACM Computing Surveys}}, keywords = {Reliability}, month = {{September}}, note = {{<b>5-year Impact Factor (2014): 5.949</b>}}, number = 1, pages = {12:1–12:41}, publisher = {ACM}, title = {{Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}}, volume = 48, year = 2015 }

  %0 Journal Article %1 MiViKoAvPa2015-CSUR-IDSEval %A Milenkoski, Aleksandar %A Vieira, Marco %A Kounev, Samuel %A Avritzer, Alberto %A Payne, Bryan D. %C New York, NY, USA %D 2015 %I ACM %J ACM Computing Surveys %N 1 %P 12:1--12:41 %T Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices %U http://dl.acm.org/authorize?N06203 %V 48 %X The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.
• 
  Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures. Brosig, Fabian; Meier, Philipp; Becker, Steffen; Koziolek, Anne; Koziolek, Heiko; Kounev, Samuel; in IEEE Transactions on Software Engineering (TSE) (2015). 41(2) 157–175. IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @article{BrMeBeKoKoKo2014-TSE-ModelSolutionTradeOffs, author = {Brosig, Fabian and Meier, Philipp and Becker, Steffen and Koziolek, Anne and Koziolek, Heiko and Kounev, Samuel}, journal = {IEEE Transactions on Software Engineering (TSE)}, keywords = {QPN}, month = {{February}}, number = 2, pages = {157–175}, publisher = {IEEE}, title = {{Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures}}, volume = 41, year = 2015 }

  %0 Journal Article %1 BrMeBeKoKoKo2014-TSE-ModelSolutionTradeOffs %A Brosig, Fabian %A Meier, Philipp %A Becker, Steffen %A Koziolek, Anne %A Koziolek, Heiko %A Kounev, Samuel %D 2015 %I IEEE %J IEEE Transactions on Software Engineering (TSE) %N 2 %P 157--175 %T Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures %U http://dx.doi.org/10.1109/TSE.2014.2362755 %V 41
• 
  Evaluating Approaches to Resource Demand Estimation. Spinner, Simon; Casale, Giuliano; Brosig, Fabian; Kounev, Samuel; in Performance Evaluation (2015). 92 51–71. Elsevier B.V.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.

  @article{SpCaBrKo2015-PEVA-ResDemEstSurvey, abstract = {Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.}, author = {Spinner, Simon and Casale, Giuliano and Brosig, Fabian and Kounev, Samuel}, journal = {Performance Evaluation}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = 10, pages = {51–71}, publisher = {Elsevier B.V.}, title = {{Evaluating Approaches to Resource Demand Estimation}}, volume = 92, year = 2015 }

  %0 Journal Article %1 SpCaBrKo2015-PEVA-ResDemEstSurvey %A Spinner, Simon %A Casale, Giuliano %A Brosig, Fabian %A Kounev, Samuel %D 2015 %I Elsevier B.V. %J Performance Evaluation %P 51--71 %T Evaluating Approaches to Resource Demand Estimation %U http://www.sciencedirect.com/science/article/pii/S0166531615000711 %V 92 %X Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.
• 
  Parallel Simulation of Queueing Petri Nets. Walter, Jürgen; Spinner, Simon; Kounev, Samuel; in EAI Endorsed Transactions on Industrial Networks and Intelligent Systems (2015). 16(8) ACM.
  • [ BibTeX ]
  • [ EndNote ]
  @article{WaSpKo2016-EAI-parallelSimulation, author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel}, journal = {EAI Endorsed Transactions on Industrial Networks and Intelligent Systems}, keywords = {se2}, month = {08}, number = 8, publisher = {ACM}, title = {{Parallel Simulation of Queueing Petri Nets}}, volume = 16, year = 2015 }

  %0 Journal Article %1 WaSpKo2016-EAI-parallelSimulation %A Walter, Jürgen %A Spinner, Simon %A Kounev, Samuel %D 2015 %I ACM %J EAI Endorsed Transactions on Industrial Networks and Intelligent Systems %N 8 %T Parallel Simulation of Queueing Petri Nets %V 16

• 
  Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments. Krebs, Rouven; Momm, Christof; Kounev, Samuel; in Elsevier Science of Computer Programming Journal (SciCo) (2014). Vol. 90, Part B 116–134. Elsevier B.V.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{KrMoKo2013-SciCo-MetricsAndTechniquesForPerformanceIsolation, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, journal = {Elsevier Science of Computer Programming Journal (SciCo)}, keywords = {Virtualization}, month = {02}, pages = {116–134}, publisher = {Elsevier B.V.}, title = {{Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments}}, volume = {Vol. 90, Part B}, year = 2014 }

  %0 Journal Article %1 KrMoKo2013-SciCo-MetricsAndTechniquesForPerformanceIsolation %A Krebs, Rouven %A Momm, Christof %A Kounev, Samuel %D 2014 %I Elsevier B.V. %J Elsevier Science of Computer Programming Journal (SciCo) %P 116--134 %T Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments %V Vol. 90, Part B
• 
  Architecture-Level Software Performance Abstractions for Online Performance Prediction. Brosig, Fabian; Huber, Nikolaus; Kounev, Samuel; in Elsevier Science of Computer Programming Journal (SciCo) (2014). Vol. 90, Part B 71–92. Elsevier.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @article{BrHuKo2013-SciCo-SoftwarePerformanceAbstractions, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, journal = {Elsevier Science of Computer Programming Journal (SciCo)}, keywords = {t_journalmagazine}, month = {09}, pages = {71–92}, publisher = {Elsevier}, title = {{Architecture-Level Software Performance Abstractions for Online Performance Prediction}}, volume = {Vol. 90, Part B}, year = 2014 }

  %0 Journal Article %1 BrHuKo2013-SciCo-SoftwarePerformanceAbstractions %A Brosig, Fabian %A Huber, Nikolaus %A Kounev, Samuel %D 2014 %I Elsevier %J Elsevier Science of Computer Programming Journal (SciCo) %P 71--92 %T Architecture-Level Software Performance Abstractions for Online Performance Prediction %U http://authors.elsevier.com/sd/article/S0167642313001421 %V Vol. 90, Part B
• 
  Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning. Herbst, Nikolas Roman; Huber, Nikolaus; Kounev, Samuel; Amrehn, Erich; in Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd. (2014). 26(12) 2053–2078.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.

  @article{HuKoAm2013-CCPE-WorkloadClassificationAndForecasting, abstract = {As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.}, author = {Herbst, Nikolas Roman and Huber, Nikolaus and Kounev, Samuel and Amrehn, Erich}, journal = {Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd.}, keywords = {Self-adaptive-systems}, month = {03}, number = 12, pages = {2053–2078}, title = {{Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning}}, volume = 26, year = 2014 }

  %0 Journal Article %1 HuKoAm2013-CCPE-WorkloadClassificationAndForecasting %A Herbst, Nikolas Roman %A Huber, Nikolaus %A Kounev, Samuel %A Amrehn, Erich %D 2014 %J Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd. %N 12 %P 2053--2078 %T Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning %U http://dx.doi.org/10.1002/cpe.3224 %V 26 %X As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.
• 
  Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments. Huber, Nikolaus; van Hoorn, André; Koziolek, Anne; Brosig, Fabian; Kounev, Samuel; in Service Oriented Computing and Applications Journal (SOCA) (2014). 8(1) 73–89. Springer-Verlag.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{HuHoKoBrKo2014-SOCA-ModelingRuntimeAdaptation, author = {Huber, Nikolaus and van Hoorn, Andr{\'e} and Koziolek, Anne and Brosig, Fabian and Kounev, Samuel}, journal = {Service Oriented Computing and Applications Journal (SOCA)}, keywords = {Self-adaptive-systems}, month = {{March}}, number = 1, pages = {73–89}, publisher = {Springer-Verlag}, title = {{Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments}}, volume = 8, year = 2014 }

  %0 Journal Article %1 HuHoKoBrKo2014-SOCA-ModelingRuntimeAdaptation %A Huber, Nikolaus %A van Hoorn, André %A Koziolek, Anne %A Brosig, Fabian %A Kounev, Samuel %D 2014 %I Springer-Verlag %J Service Oriented Computing and Applications Journal (SOCA) %N 1 %P 73--89 %T Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments %V 8

• 
  Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions. Kounev, Samuel; Rathfelder, Christoph; Klatt, Benjamin; in Electronic Notes in Theoretical Computer Science (ENTCS) (2013). 295 3–9. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.

  @article{KoRaKl2012-FESCA-Keynote, abstract = {Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.}, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel and Rathfelder, Christoph and Klatt, Benjamin}, journal = {{Electronic Notes in Theoretical Computer Science (ENTCS)}}, keywords = {t_journalmagazine}, month = {05}, pages = {3–9}, publisher = {Elsevier Science Publishers B. V.}, title = {{Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions}}, volume = 295, year = 2013 }

  %0 Journal Article %1 KoRaKl2012-FESCA-Keynote %A Kounev, Samuel %A Rathfelder, Christoph %A Klatt, Benjamin %C Amsterdam, The Netherlands %D 2013 %I Elsevier Science Publishers B. V. %J Electronic Notes in Theoretical Computer Science (ENTCS) %P 3--9 %T Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions %U http://www.sciencedirect.com/science/article/pii/S1571066113000248 %V 295 %X Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.
• 
  Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions. Rathfelder, Christoph; Klatt, Benjamin; Sachs, Kai; Kounev, Samuel; in Journal of Software and Systems Modeling (SoSyM) (2013). 1–27. Springer Verlag.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @article{rathfelder2013a, author = {Rathfelder, Christoph and Klatt, Benjamin and Sachs, Kai and Kounev, Samuel}, journal = {Journal of Software and Systems Modeling (SoSyM)}, keywords = {t_journalmagazine}, month = {03}, pages = {1–27}, publisher = {Springer Verlag}, title = {Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions}, year = 2013 }

  %0 Journal Article %1 rathfelder2013a %A Rathfelder, Christoph %A Klatt, Benjamin %A Sachs, Kai %A Kounev, Samuel %D 2013 %I Springer Verlag %J Journal of Software and Systems Modeling (SoSyM) %P 1--27 %T Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions %U http://dx.doi.org/10.1007/s10270-013-0316-x
• 
  Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey. Rygielski, Piotr; Kounev, Samuel; in PIK --- Praxis der Informationsverarbeitung und Kommunikation (2013). 36(1) 55–64. de Gruyter.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.

  @article{RyKo2013, abstract = {The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.}, author = {Rygielski, Piotr and Kounev, Samuel}, journal = {PIK — Praxis der Informationsverarbeitung und Kommunikation}, keywords = {SDN}, month = {02}, number = 1, pages = {55–64}, publisher = {de Gruyter}, title = {{Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey}}, volume = 36, year = 2013 }

  %0 Journal Article %1 RyKo2013 %A Rygielski, Piotr %A Kounev, Samuel %D 2013 %I de Gruyter %J PIK --- Praxis der Informationsverarbeitung und Kommunikation %N 1 %P 55--64 %T Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey %U http://www.degruyter.com/view/j/piko-2013-36-issue-1/pik-2012-0136/pik-2012-0136.xml?format=INT %V 36 %X The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.
• 
  Performance modeling and analysis of message-oriented event-driven systems. Sachs, Kai; Kounev, Samuel; Buchmann, Alejandro; in Journal of Software and Systems Modeling (2013). 12(4) 705–729. Springer-Verlag.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.

  @article{SaKoBu2011-SoSyM-PerfModMoEdSys, abstract = {Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.}, author = {Sachs, Kai and Kounev, Samuel and Buchmann, Alejandro}, journal = {Journal of Software and Systems Modeling}, keywords = {QPN}, number = 4, pages = {705–729}, publisher = {Springer-Verlag}, title = {Performance modeling and analysis of message-oriented event-driven systems}, volume = 12, year = 2013 }

  %0 Journal Article %1 SaKoBu2011-SoSyM-PerfModMoEdSys %A Sachs, Kai %A Kounev, Samuel %A Buchmann, Alejandro %D 2013 %I Springer-Verlag %J Journal of Software and Systems Modeling %N 4 %P 705--729 %T Performance modeling and analysis of message-oriented event-driven systems %V 12 %X Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.

• 
  Self-Aware Software and Systems Engineering: A Vision and Research Roadmap. Kounev, Samuel; in GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928 (2011). Karlsruhe, Germany.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{Ko2011-SE-DescartesResearch, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, booktitle = {{GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928}}, keywords = {Self-adaptive-systems}, title = {{Self-Aware Software and Systems Engineering: A Vision and Research Roadmap}}, year = 2011 }

  %0 Conference Paper %1 Ko2011-SE-DescartesResearch %A Kounev, Samuel %B GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928 %C Karlsruhe, Germany %D 2011 %T Self-Aware Software and Systems Engineering: A Vision and Research Roadmap %U http://pi.informatik.uni-siegen.de/stt/31_4/index.html

• 
  Benchmarking and Performance Modeling of Event-Based Systems. Kounev, Samuel; Sachs, Kai; in it - Information Technology (2009). 51(5) 262–269. Oldenbourg Wissenschaftsverlag, Munich, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.

  @article{KoSa2009-DEBSPerformance, abstract = {Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.}, address = {Munich, Germany}, author = {Kounev, Samuel and Sachs, Kai}, journal = {it - Information Technology}, keywords = {QPN}, month = 10, number = 5, pages = {262–269}, publisher = {Oldenbourg Wissenschaftsverlag}, title = {{Benchmarking and Performance Modeling of Event-Based Systems}}, volume = 51, year = 2009 }

  %0 Journal Article %1 KoSa2009-DEBSPerformance %A Kounev, Samuel %A Sachs, Kai %C Munich, Germany %D 2009 %I Oldenbourg Wissenschaftsverlag %J it - Information Technology %N 5 %P 262--269 %T Benchmarking and Performance Modeling of Event-Based Systems %V 51 %X Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.
• 
  Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark. Sachs, Kai; Kounev, Samuel; Bacon, Jean; Buchmann, Alejandro; in Performance Evaluation (2009). 66(8) 410–434. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.

  @article{SaKoBaBu2008-PERFEVAL-SPECjms2007, abstract = {Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.}, address = {Amsterdam, The Netherlands}, author = {Sachs, Kai and Kounev, Samuel and Bacon, Jean and Buchmann, Alejandro}, journal = {Performance Evaluation}, keywords = {SPEC}, month = {08}, number = 8, pages = {410–434}, publisher = {Elsevier Science Publishers B. V.}, title = {{Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark}}, volume = 66, year = 2009 }

  %0 Journal Article %1 SaKoBaBu2008-PERFEVAL-SPECjms2007 %A Sachs, Kai %A Kounev, Samuel %A Bacon, Jean %A Buchmann, Alejandro %C Amsterdam, The Netherlands %D 2009 %I Elsevier Science Publishers B. V. %J Performance Evaluation %N 8 %P 410--434 %T Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark %U http://www.elsevier.com/wps/find/journaldescription.cws_home/505618/description %V 66 %X Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.
• 
  Autonomic QoS control in enterprise Grid environments using online simulation. Nou, Ramon; Kounev, Samuel; Julia, Ferran; Torres, Jordi; in Journal of Systems and Software (2009). 82(3) 486–502. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.

  @article{NoKoJuTo2008-JSS-GridAutoQoS, abstract = {As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.}, address = {Amsterdam, The Netherlands}, author = {Nou, Ramon and Kounev, Samuel and Julia, Ferran and Torres, Jordi}, journal = {Journal of Systems and Software}, keywords = {Self-adaptive-systems}, month = {03}, number = 3, pages = {486–502}, publisher = {Elsevier Science Publishers B. V.}, title = {{Autonomic QoS control in enterprise Grid environments using online simulation}}, volume = 82, year = 2009 }

  %0 Journal Article %1 NoKoJuTo2008-JSS-GridAutoQoS %A Nou, Ramon %A Kounev, Samuel %A Julia, Ferran %A Torres, Jordi %C Amsterdam, The Netherlands %D 2009 %I Elsevier Science Publishers B. V. %J Journal of Systems and Software %N 3 %P 486--502 %T Autonomic QoS control in enterprise Grid environments using online simulation %V 82 %X As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.
• 
  QPME - A Performance Modeling Tool Based on Queueing Petri Nets. Kounev, Samuel; Dutz, Christofer; in ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis (2009). 36 46–51. ACM.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.

  @article{KoDu2009-SIGMETRICS_PER-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.}, author = {Kounev, Samuel and Dutz, Christofer}, journal = {ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis}, keywords = {QPN}, month = {03}, pages = {46–51}, publisher = {ACM}, title = {{QPME - A Performance Modeling Tool Based on Queueing Petri Nets}}, volume = 36, year = 2009 }

  %0 Journal Article %1 KoDu2009-SIGMETRICS_PER-QPME %A Kounev, Samuel %A Dutz, Christofer %D 2009 %I ACM %J ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis %P 46--51 %T QPME - A Performance Modeling Tool Based on Queueing Petri Nets %V 36 %X Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.

• 
  Kaffeekunde - SPECjms misst Message-oriented Middleware. Kounev, Samuel; Sachs, Kai; (2008).
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  @misc{sachs2008a, author = {Kounev, Samuel and Sachs, Kai}, howpublished = {iX Magazin, Heft 02/2008, Heise Zeitschriften Verlag}, keywords = {SPEC}, title = {{Kaffeekunde - SPECjms misst Message-oriented Middleware}}, year = 2008 }

  %0 Generic %1 sachs2008a %A Kounev, Samuel %A Sachs, Kai %D 2008 %T Kaffeekunde - SPECjms misst Message-oriented Middleware %U http://www.heise.de/kiosk/archiv/ix/2008/2/121

• 
  Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets. Kounev, Samuel; in IEEE Transactions on Software Engineering (2006). 32(7) 486–502.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.

  @article{Ko2006-IEEE_TSE-QPN_ModelingMethod, abstract = {Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.}, author = {Kounev, Samuel}, journal = {IEEE Transactions on Software Engineering}, keywords = {QPME_Bibliography}, month = {07}, number = 7, pages = {486–502}, title = {{Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets}}, volume = 32, year = 2006 }

  %0 Journal Article %1 Ko2006-IEEE_TSE-QPN_ModelingMethod %A Kounev, Samuel %D 2006 %J IEEE Transactions on Software Engineering %N 7 %P 486--502 %T Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets %V 32 %X Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.
• 
  SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation. Kounev, Samuel; Buchmann, Alejandro; in Performance Evaluation (2006). 63(4-5) 364–394. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{KoBu2006-PERFEVAL-SimQPN, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel and Buchmann, Alejandro}, journal = {Performance Evaluation}, keywords = {QPN}, month = {05}, number = {4-5}, pages = {364–394}, publisher = {Elsevier Science Publishers B. V.}, title = {{SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation}}, volume = 63, year = 2006 }

  %0 Journal Article %1 KoBu2006-PERFEVAL-SimQPN %A Kounev, Samuel %A Buchmann, Alejandro %C Amsterdam, The Netherlands %D 2006 %I Elsevier Science Publishers B. V. %J Performance Evaluation %N 4-5 %P 364--394 %T SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation %V 63

• 
  Performance Tuning and Optimization of J2EE Applications on the JBoss Platform. Kounev, Samuel; Weis, Börn; Buchmann, Alejandro; in Journal of Computer Resource Management (2004). 113 Computer Measurement Group (CMG).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @article{KoWeBu2004-CMG_JCRM-JBoss, author = {Kounev, Samuel and Weis, B{\"o}rn and Buchmann, Alejandro}, journal = {Journal of Computer Resource Management}, keywords = {Optimization}, month = {09}, publisher = {Computer Measurement Group (CMG)}, title = {{Performance Tuning and Optimization of J2EE Applications on the JBoss Platform}}, volume = 113, year = 2004 }

  %0 Journal Article %1 KoWeBu2004-CMG_JCRM-JBoss %A Kounev, Samuel %A Weis, Börn %A Buchmann, Alejandro %D 2004 %I Computer Measurement Group (CMG) %J Journal of Computer Resource Management %T Performance Tuning and Optimization of J2EE Applications on the JBoss Platform %V 113

• 
  Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server. Nenov, Plamen; Kounev, Samuel; Mihailov, Dimiter; in Journal of Computing and Information (1999).
  • [ BibTeX ]
  • [ EndNote ]
  @article{NeKoMi1999-JCI-VideoConf, author = {Nenov, Plamen and Kounev, Samuel and Mihailov, Dimiter}, journal = {Journal of Computing and Information}, keywords = {t_journalmagazine}, title = {{Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server}}, year = 1999 }

  %0 Journal Article %1 NeKoMi1999-JCI-VideoConf %A Nenov, Plamen %A Kounev, Samuel %A Mihailov, Dimiter %D 1999 %J Journal of Computing and Information %T Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server

• [ 2025 ]
• [ 2024 ]
• [ 2023 ]
• [ 2022 ]
• [ 2021 ]
• [ 2020 ]
• [ 2019 ]
• [ 2018 ]
• [ 2017 ]
• [ 2016 ]
• [ 2015 ]
• [ 2014 ]
• [ 2013 ]
• [ 2012 ]
• [ 2011 ]
• [ 2010 ]
• [ 2009 ]
• [ 2008 ]
• [ 2007 ]
• [ 2003 ]
• [ 2002 ]
• [ 1999 ]

• 
  Generating Executable Microservice Applications for Performance Benchmarking. Lubas, Yannik; Straesser, Martin; Bauer, André; Kounev, Samuel; in Proceedings of the 16th ACM/SPEC International Conference on Performance Engineering (2025). 31–44. Association for Computing Machinery, Toronto ON, Canada.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Microservice applications are the building blocks of modern cloud applications. As such, their performance aspects have been receiving increasing attention in the software engineering community. However, many microservice performance studies use only a small set of popular microservice test applications for experiments, questioning the applicability of their approaches in practice. Researchers currently lack the opportunity to collect large and diverse datasets containing performance metrics of microservices. This is because popular test applications only represent specific technology stacks and often come with custom benchmark tooling (e.g., load generation and monitoring). In this paper, we present Creo, a framework for generating microservice applications that (1) are fully executable, (2) have configurable properties and resource usage profiles, and (3) have built-in support for standardized monitoring, load generation, and deployment. Our approach enables researchers to run experiments with diverse microservice applications with minimal effort. We demonstrate the value of our approach in the context of two use cases. First, we show that using generated applications when training machine learning models for predicting performance degradation can improve the prediction accuracy. Second, we evaluate a recent approach for performance anomaly classification on a set of generated applications highlighting strengths and weaknesses not discussed in the original work.

  @inproceedings{10.1145/3676151.3719361, abstract = {Microservice applications are the building blocks of modern cloud applications. As such, their performance aspects have been receiving increasing attention in the software engineering community. However, many microservice performance studies use only a small set of popular microservice test applications for experiments, questioning the applicability of their approaches in practice. Researchers currently lack the opportunity to collect large and diverse datasets containing performance metrics of microservices. This is because popular test applications only represent specific technology stacks and often come with custom benchmark tooling (e.g., load generation and monitoring). In this paper, we present Creo, a framework for generating microservice applications that (1) are fully executable, (2) have configurable properties and resource usage profiles, and (3) have built-in support for standardized monitoring, load generation, and deployment. Our approach enables researchers to run experiments with diverse microservice applications with minimal effort. We demonstrate the value of our approach in the context of two use cases. First, we show that using generated applications when training machine learning models for predicting performance degradation can improve the prediction accuracy. Second, we evaluate a recent approach for performance anomaly classification on a set of generated applications highlighting strengths and weaknesses not discussed in the original work.}, address = {New York, NY, USA}, author = {Lubas, Yannik and Straesser, Martin and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 16th ACM/SPEC International Conference on Performance Engineering}, keywords = {microservices}, pages = {31–44}, publisher = {Association for Computing Machinery}, series = {ICPE '25}, title = {Generating Executable Microservice Applications for Performance Benchmarking}, year = 2025 }

  %0 Conference Paper %1 10.1145/3676151.3719361 %A Lubas, Yannik %A Straesser, Martin %A Bauer, André %A Kounev, Samuel %B Proceedings of the 16th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2025 %I Association for Computing Machinery %P 31–44 %R 10.1145/3676151.3719361 %T Generating Executable Microservice Applications for Performance Benchmarking %U https://doi.org/10.1145/3676151.3719361 %X Microservice applications are the building blocks of modern cloud applications. As such, their performance aspects have been receiving increasing attention in the software engineering community. However, many microservice performance studies use only a small set of popular microservice test applications for experiments, questioning the applicability of their approaches in practice. Researchers currently lack the opportunity to collect large and diverse datasets containing performance metrics of microservices. This is because popular test applications only represent specific technology stacks and often come with custom benchmark tooling (e.g., load generation and monitoring). In this paper, we present Creo, a framework for generating microservice applications that (1) are fully executable, (2) have configurable properties and resource usage profiles, and (3) have built-in support for standardized monitoring, load generation, and deployment. Our approach enables researchers to run experiments with diverse microservice applications with minimal effort. We demonstrate the value of our approach in the context of two use cases. First, we show that using generated applications when training machine learning models for predicting performance degradation can improve the prediction accuracy. Second, we evaluate a recent approach for performance anomaly classification on a set of generated applications highlighting strengths and weaknesses not discussed in the original work. %@ 9798400710735

• 
  MACRO: A Multi-Head Attentional Convolutional Recurrent Network for the Classification of Co-Occurring Diseases in 12-Lead ECGs. Borst, Vanessa; Leppich, Robert; Kounev, Samuel; in 2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) (2024). IEEE.

  Acceptance Rate: 24.4%
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  @inproceedings{borst2024MACRO, author = {Borst, Vanessa and Leppich, Robert and Kounev, Samuel}, booktitle = {2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI)}, keywords = {t_interdisciplinary}, note = {Acceptance Rate: 24.4%}, organization = {IEEE}, title = {{MACRO}: A Multi-Head Attentional Convolutional Recurrent Network for the Classification of Co-Occurring Diseases in 12-Lead {ECGs}}, year = 2024 }

  %0 Conference Paper %1 borst2024MACRO %A Borst, Vanessa %A Leppich, Robert %A Kounev, Samuel %B 2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) %D 2024 %T MACRO: A Multi-Head Attentional Convolutional Recurrent Network for the Classification of Co-Occurring Diseases in 12-Lead ECGs %U https://openreview.net/pdf?id=toAq9UFmS9
• 
  Evaluating the Benefits of Model Retraining for Self-Aware Vehicle Traffic Forecasting. Hadry, Marius; Grillmeyer, Daniel; Becker, Martin; König, Maximilian; Lesch, Veronika; Kounev, Samuel; in IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS) (2024). 41–50.
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  @inproceedings{10771392, author = {Hadry, Marius and Grillmeyer, Daniel and Becker, Martin and König, Maximilian and Lesch, Veronika and Kounev, Samuel}, booktitle = {IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)}, keywords = {descartes}, pages = {41-50}, title = {Evaluating the Benefits of Model Retraining for Self-Aware Vehicle Traffic Forecasting}, year = 2024 }

  %0 Conference Paper %1 10771392 %A Hadry, Marius %A Grillmeyer, Daniel %A Becker, Martin %A König, Maximilian %A Lesch, Veronika %A Kounev, Samuel %B IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS) %D 2024 %P 41-50 %R 10.1109/ACSOS61780.2024.00022 %T Evaluating the Benefits of Model Retraining for Self-Aware Vehicle Traffic Forecasting
• 
  Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme. Prantl, Thomas; Lauer, Marco; Horn, Lukas; Engel, Simon; Dingel, David; Kounev, Samuel; Bauer, André; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 19th International Conference on Availability, Reliability and Security (2024). 1–11. ACM.
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  @inproceedings{ARES_2024_ABE, author = {Prantl, Thomas and Lauer, Marco and Horn, Lukas and Engel, Simon and Dingel, David and Kounev, Samuel and Bauer, André and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 19th International Conference on Availability, Reliability and Security}, keywords = {descartes}, month = {07}, pages = {1-11}, publisher = {ACM}, series = {ARES 2024}, title = {Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme}, year = 2024 }

  %0 Conference Paper %1 ARES_2024_ABE %A Prantl, Thomas %A Lauer, Marco %A Horn, Lukas %A Engel, Simon %A Dingel, David %A Kounev, Samuel %A Bauer, André %A Krupitzer, Christian %A Kounev, Samuel %B Proceedings of the 19th International Conference on Availability, Reliability and Security %D 2024 %I ACM %P 1-11 %R https://doi.org/10.1145/3664476.3664487 %T Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme

• 
  BLOCSIE - Benchmark for LOCalization in a Simulated Industrial Environment. Spiess, Florian; Bluemm, Daniel; Strobel, Norbert; Kaupp, Tobias; Kounev, Samuel; in 2023 21st International Conference on Advanced Robotics (ICAR) (2023). 577–584.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{10406777, author = {Spiess, Florian and Bluemm, Daniel and Strobel, Norbert and Kaupp, Tobias and Kounev, Samuel}, booktitle = {2023 21st International Conference on Advanced Robotics (ICAR)}, keywords = {descartes}, pages = {577-584}, title = {BLOCSIE - Benchmark for LOCalization in a Simulated Industrial Environment}, year = 2023 }

  %0 Conference Paper %1 10406777 %A Spiess, Florian %A Bluemm, Daniel %A Strobel, Norbert %A Kaupp, Tobias %A Kounev, Samuel %B 2023 21st International Conference on Advanced Robotics (ICAR) %D 2023 %P 577-584 %T BLOCSIE - Benchmark for LOCalization in a Simulated Industrial Environment
• 
  An Empirical Study of Container Image Configurations and Their Impact on Start Times. Straesser, Martin; Bauer, André; Leppich, Robert; Herbst, Nikolas; Chard, Kyle; Foster, Ian; Kounev, Samuel; in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid) (2023). 94–105. IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  @inproceedings{straesser2023starttime, author = {Straesser, Martin and Bauer, André and Leppich, Robert and Herbst, Nikolas and Chard, Kyle and Foster, Ian and Kounev, Samuel}, booktitle = {2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)}, keywords = {empirical_study}, pages = {94-105}, publisher = {IEEE}, title = {An Empirical Study of Container Image Configurations and Their Impact on Start Times}, year = 2023 }

  %0 Conference Paper %1 straesser2023starttime %A Straesser, Martin %A Bauer, André %A Leppich, Robert %A Herbst, Nikolas %A Chard, Kyle %A Foster, Ian %A Kounev, Samuel %B 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid) %D 2023 %I IEEE %P 94-105 %R 10.1109/CCGrid57682.2023.00019 %T An Empirical Study of Container Image Configurations and Their Impact on Start Times %U https://doi.org/10.1109/CCGrid57682.2023.00019
• 
  Autoscaler Evaluation and Configuration: A Practitioner’s Guideline. Straesser, Martin; Eismann, Simon; von Kistowski, Jóakim; Bauer, André; Kounev, Samuel; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 31–41. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.

  @inproceedings{straesser2023guideline, abstract = {Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.}, address = {New York, NY, USA}, author = {Straesser, Martin and Eismann, Simon and von Kistowski, J\'{o}akim and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {methodology}, pages = {31-41}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {Autoscaler Evaluation and Configuration: A Practitioner's Guideline}, year = 2023 }

  %0 Conference Paper %1 straesser2023guideline %A Straesser, Martin %A Eismann, Simon %A von Kistowski, Jóakim %A Bauer, André %A Kounev, Samuel %B Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2023 %I Association for Computing Machinery %P 31-41 %R 10.1145/3578244.3583721 %T Autoscaler Evaluation and Configuration: A Practitioner's Guideline %U https://doi.org/10.1145/3578244.3583721 %X Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment. %@ 9798400700682
• 
  Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration. Straesser, Martin; Haas, Patrick; Frank, Sebastian; Hakamian, Alireza; Van Hoorn, André; Kounev, Samuel; in Performance Evaluation Methodologies and Tools (2023).

  In print.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.

  @inproceedings{straesser2023kubernetesintheloop, abstract = {Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.}, author = {Straesser, Martin and Haas, Patrick and Frank, Sebastian and Hakamian, Alireza and Van Hoorn, André and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, keywords = {discrete_event_simulation}, note = {In print.}, title = {Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration}, year = 2023 }

  %0 Conference Paper %1 straesser2023kubernetesintheloop %A Straesser, Martin %A Haas, Patrick %A Frank, Sebastian %A Hakamian, Alireza %A Van Hoorn, André %A Kounev, Samuel %B Performance Evaluation Methodologies and Tools %D 2023 %T Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration %X Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.
• 
  A Systematic Approach for Benchmarking of Container Orchestration Frameworks. Straesser, Martin; Mathiasch, Jonas; Bauer, André; Kounev, Samuel; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 187–198. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.

  @inproceedings{straesser2023orchestration, abstract = {Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.}, address = {New York, NY, USA}, author = {Straesser, Martin and Mathiasch, Jonas and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {container_orchestration}, pages = {187-198}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {A Systematic Approach for Benchmarking of Container Orchestration Frameworks}, year = 2023 }

  %0 Conference Paper %1 straesser2023orchestration %A Straesser, Martin %A Mathiasch, Jonas %A Bauer, André %A Kounev, Samuel %B Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2023 %I Association for Computing Machinery %P 187-198 %R 10.1145/3578244.3583726 %T A Systematic Approach for Benchmarking of Container Orchestration Frameworks %U https://doi.org/10.1145/3578244.3583726 %X Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more. %@ 9798400700682

• 
  Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling. Straesser, Martin; Grohmann, Johannes; von Kistowski, Jóakim; Eismann, Simon; Bauer, André; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 105–115. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.

  @inproceedings{10.1145/3489525.3511680, abstract = {Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.}, address = {New York, NY, USA}, author = {Straesser, Martin and Grohmann, Johannes and von Kistowski, J\'{o}akim and Eismann, Simon and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {microservices}, pages = {105–115}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling}, year = 2022 }

  %0 Conference Paper %1 10.1145/3489525.3511680 %A Straesser, Martin %A Grohmann, Johannes %A von Kistowski, Jóakim %A Eismann, Simon %A Bauer, André %A Kounev, Samuel %B Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering %C New York, NY, USA %D 2022 %I Association for Computing Machinery %P 105–115 %T Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling %U https://doi.org/10.1145/3489525.3511680 %X Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.
• 
  SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier Petzi, Lukas; Ben Yahya, Ala Eddine; Dmitrienko, Alexandra; Tsudik, Gene; Prantl, Thomas; Kounev, Samuel; (2022). (Vol. USENIX Security ’22)

  Coming Soon
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.

  @proceedings{petzi2022scraps, abstract = {Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.}, author = {Petzi, Lukas and Ben Yahya, Ala Eddine and Dmitrienko, Alexandra and Tsudik, Gene and Prantl, Thomas and Kounev, Samuel}, keywords = {sss-group}, note = {Coming Soon}, title = {SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier}, volume = {USENIX Security '22}, year = 2022 }

  %0 Conference Proceedings %1 petzi2022scraps %A Petzi, Lukas %A Ben Yahya, Ala Eddine %A Dmitrienko, Alexandra %A Tsudik, Gene %A Prantl, Thomas %A Kounev, Samuel %D 2022 %T SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier %V USENIX Security '22 %X Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.
• 
  Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes. Prantl, Thomas; Zeck, Timo; Iffländer, Lukas; Beierlieb, Lukas; Dmitrenko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2022 5th Conference on Cloud and Internet of Things (CIoT) (2022). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark, author = {Prantl, Thomas and Zeck, Timo and Iffländer, Lukas and Beierlieb, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2022 5th Conference on Cloud and Internet of Things (CIoT)}, keywords = {descartes}, organization = {IEEE}, title = {Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes}, year = 2022 }

  %0 Conference Paper %1 PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark %A Prantl, Thomas %A Zeck, Timo %A Iffländer, Lukas %A Beierlieb, Lukas %A Dmitrenko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2022 5th Conference on Cloud and Internet of Things (CIoT) %D 2022 %T Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes
• 
  Investigating the Predictability of QoS Metrics in Cellular Networks. Herrnleben, Stefan; Grohmann, Johannes; Lesch, Veronika; Prantl, Thomas; Metzger, Florian; Hoßfeld, Tobias; Kounev, Samuel; in 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS) (2022). 1–10.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{HeGrLePrMeHoKo-2022-IWQoS-QoS-Prediction-Cellular-Networks, author = {Herrnleben, Stefan and Grohmann, Johannes and Lesch, Veronika and Prantl, Thomas and Metzger, Florian and Hoßfeld, Tobias and Kounev, Samuel}, booktitle = {2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS)}, keywords = {descartes}, pages = {1-10}, title = {{Investigating the Predictability of QoS Metrics in Cellular Networks}}, year = 2022 }

  %0 Conference Paper %1 HeGrLePrMeHoKo-2022-IWQoS-QoS-Prediction-Cellular-Networks %A Herrnleben, Stefan %A Grohmann, Johannes %A Lesch, Veronika %A Prantl, Thomas %A Metzger, Florian %A Hoßfeld, Tobias %A Kounev, Samuel %B 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS) %D 2022 %P 1-10 %T Investigating the Predictability of QoS Metrics in Cellular Networks

• 
  ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations. Herrnleben, Stefan; Leidinger, Maximilian; Lesch, Veronika; Prantl, Thomas; Grohmann, Johannes; Krupitzer, Christian; Kounev, Samuel; in Performance Evaluation Methodologies and Tools, Q. Zhao, L. Xia (eds.) (2021). 72–92. Springer International Publishing, Cham.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.

  @inproceedings{10.1007/978-3-030-92511-6_5, abstract = {Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Leidinger, Maximilian and Lesch, Veronika and Prantl, Thomas and Grohmann, Johannes and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, editor = {Zhao, Qianchuan and Xia, Li}, keywords = {networking}, pages = {72–92}, publisher = {Springer International Publishing}, title = {ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations}, year = 2021 }

  %0 Conference Paper %1 10.1007/978-3-030-92511-6_5 %A Herrnleben, Stefan %A Leidinger, Maximilian %A Lesch, Veronika %A Prantl, Thomas %A Grohmann, Johannes %A Krupitzer, Christian %A Kounev, Samuel %B Performance Evaluation Methodologies and Tools %C Cham %D 2021 %E Zhao, Qianchuan %E Xia, Li %I Springer International Publishing %P 72--92 %R https://doi.org/10.1007/978-3-030-92511-6_5 %T ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations %U https://link.springer.com/chapter/10.1007%2F978-3-030-92511-6_5 %X Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.
• 
  Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies. Lesch, Veronika; Noack, Tanja; Hefter, Johannes; Kounev, Samuel; Krupitzer, Christian; in Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021 (2021). IEEE.

  Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{lesch2021towards, author = {Lesch, Veronika and Noack, Tanja and Hefter, Johannes and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021}, keywords = {internet_of_things}, note = {Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%}, publisher = {IEEE}, title = {Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies}, year = 2021 }

  %0 Conference Paper %1 lesch2021towards %A Lesch, Veronika %A Noack, Tanja %A Hefter, Johannes %A Kounev, Samuel %A Krupitzer, Christian %B Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021 %D 2021 %I IEEE %T Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies
• 
  Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction. Züfle, Marwin; Erhard, Florian; Kounev, Samuel; in 20th IEEE International Conference on Machine Learning and Applications (ICMLA) (2021).

  (to appear)
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{zufle2021machine, author = {Z{\"u}fle, Marwin and Erhard, Florian and Kounev, Samuel}, booktitle = {20th IEEE International Conference on Machine Learning and Applications (ICMLA)}, keywords = {online_monitoring_and_forecasting}, note = {(to appear)}, title = {Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction}, year = 2021 }

  %0 Conference Paper %1 zufle2021machine %A Züfle, Marwin %A Erhard, Florian %A Kounev, Samuel %B 20th IEEE International Conference on Machine Learning and Applications (ICMLA) %D 2021 %T Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction
• 
  Sizeless: Predicting the Optimal Size of Serverless Functions. Eismann, Simon; Bui, Long; Grohmann, Johannes; Abad, Cristina; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 22nd International MIDDLEWARE Conference (2021). 248–259.

  Best Student Paper Award, ACM Artifacts Evaluated — Functional
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.

  @inproceedings{eismann2021sizeless, abstract = {Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.}, author = {Eismann, Simon and Bui, Long and Grohmann, Johannes and Abad, Cristina and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 22nd International MIDDLEWARE Conference}, keywords = {descartes}, note = {Best Student Paper Award, ACM Artifacts Evaluated — Functional}, pages = {248–259}, title = {Sizeless: Predicting the Optimal Size of Serverless Functions}, year = 2021 }

  %0 Conference Paper %1 eismann2021sizeless %A Eismann, Simon %A Bui, Long %A Grohmann, Johannes %A Abad, Cristina %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 22nd International MIDDLEWARE Conference %D 2021 %P 248–259 %R 10.1145/3464298.3493398 %T Sizeless: Predicting the Optimal Size of Serverless Functions %U https://dl.acm.org/doi/abs/10.1145/3464298.3493398 %X Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.
• 
  Performance Evaluation of a Post-Quantum Public-Key Cryptosystem. Prantl, Thomas; Prantl, Dominik; Beierlieb, Lukas; Iffländer, Lukas; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem, author = {Prantl, Thomas and Prantl, Dominik and Beierlieb, Lukas and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {t_workshop}, organization = {IEEE}, title = {Performance Evaluation of a Post-Quantum Public-Key Cryptosystem}, year = 2021 }

  %0 Conference Paper %1 PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem %A Prantl, Thomas %A Prantl, Dominik %A Beierlieb, Lukas %A Iffländer, Lukas %A Dmitrienko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) %D 2021 %T Performance Evaluation of a Post-Quantum Public-Key Cryptosystem
• 
  Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping. Finke, Nils; Mohr, Marisa; Lontke, Alexander; Züfle, Marwin; Kounev, Samuel; Möller, Ralf; in Research Challenges in Information Science (RCIS) (2021). 189–204. Springer.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{finke2021recommendations, author = {Finke, Nils and Mohr, Marisa and Lontke, Alexander and Z{\"u}fle, Marwin and Kounev, Samuel and M{\"o}ller, Ralf}, booktitle = {Research Challenges in Information Science (RCIS)}, keywords = {t_interdisciplinary}, organization = {Springer}, pages = {189–204}, title = {Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping}, year = 2021 }

  %0 Conference Paper %1 finke2021recommendations %A Finke, Nils %A Mohr, Marisa %A Lontke, Alexander %A Züfle, Marwin %A Kounev, Samuel %A Möller, Ralf %B Research Challenges in Information Science (RCIS) %D 2021 %P 189--204 %T Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping
• 
  Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT. Prantl, Thomas; Prantl, Dominik; Bauer, André; Iffländer, Lukas; Dmitrenko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption, author = {Prantl, Thomas and Prantl, Dominik and Bauer, André and Iffländer, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {sss-group}, title = {Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT}, year = 2021 }

  %0 Conference Paper %1 PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption %A Prantl, Thomas %A Prantl, Dominik %A Bauer, André %A Iffländer, Lukas %A Dmitrenko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) %D 2021 %T Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT
• 
  Libra: A Benchmark for Time Series Forecasting Methods. Bauer, André; Züfle, Marwin; Eismann, Simon; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021). ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.

  @inproceedings{bauer2021benchmark, abstract = {In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.}, address = {New York, NY, USA}, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {benchmark}, month = {04}, organization = {ACM}, title = {Libra: A Benchmark for Time Series Forecasting Methods}, year = 2021 }

  %0 Conference Paper %1 bauer2021benchmark %A Bauer, André %A Züfle, Marwin %A Eismann, Simon %A Grohmann, Johannes %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) %C New York, NY, USA %D 2021 %R 10.1145/3427921.3450241 %T Libra: A Benchmark for Time Series Forecasting Methods %U https://dl.acm.org/doi/10.1145/3427921.3450241 %X In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.
• 
  A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0. Züfle, Marwin; Agne, Joachim; Grohmann, Johannes; Dörtoluk, Ibrahim; Kounev, Samuel; in Proceedings of the 21st IEEE IES International Conference on Industrial Informatics (2021). IEEE.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.

  @inproceedings{ZuAgGrDoKo-INDIN2021-PdM, abstract = {Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.}, author = {Züfle, Marwin and Agne, Joachim and Grohmann, Johannes and Dörtoluk, Ibrahim and Kounev, Samuel}, booktitle = {Proceedings of the 21st IEEE IES International Conference on Industrial Informatics}, keywords = {t_interdisciplinary}, month = {07}, publisher = {IEEE}, series = {INDIN'21}, title = {A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0}, year = 2021 }

  %0 Conference Paper %1 ZuAgGrDoKo-INDIN2021-PdM %A Züfle, Marwin %A Agne, Joachim %A Grohmann, Johannes %A Dörtoluk, Ibrahim %A Kounev, Samuel %B Proceedings of the 21st IEEE IES International Conference on Industrial Informatics %D 2021 %I IEEE %R 10.1109/INDIN45523.2021.9557387 %T A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0 %U https://ieeexplore.ieee.org/abstract/document/9557387 %X Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.
• 
  A Simulation-based Optimization Framework for Online Adaptation of Networks. Herrnleben, Stefan; Grohmann, Johannes; Rygielski, Pitor; Lesch, Veronika; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools), H. Song, D. Jiang (eds.) (2021). 513–532. Springer International Publishing, Cham.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.

  @inproceedings{HeGrRyLeKrKo2020-SIMUtools-Network-Online-Adaptation, abstract = {Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.}, address = {Cham}, author = {Herrnleben, Stefan and Grohmann, Johannes and Rygielski, Pitor and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, editor = {Song, Houbing and Jiang, Dingde}, keywords = {networking}, month = {08}, pages = {513–532}, publisher = {Springer International Publishing}, series = {SIMUtools 2020}, title = {A Simulation-based Optimization Framework for Online Adaptation of Networks}, year = 2021 }

  %0 Conference Paper %1 HeGrRyLeKrKo2020-SIMUtools-Network-Online-Adaptation %A Herrnleben, Stefan %A Grohmann, Johannes %A Rygielski, Pitor %A Lesch, Veronika %A Krupitzer, Christian %A Kounev, Samuel %B Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) %C Cham %D 2021 %E Song, Houbing %E Jiang, Dingde %I Springer International Publishing %P 513--532 %R 10.1007/978-3-030-72792-5_41 %T A Simulation-based Optimization Framework for Online Adaptation of Networks %U https://doi.org/10.1007/978-3-030-72792-5_41 %X Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations. %@ 978-3-030-72792-5
• 
  Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment. Spiess, Florian; Friesslich, Jonas; Bluemm, Daniel; Mast, Fabio; Vinokour, Dmitrij; Kounev, Samuel; Kaupp, Tobias; Strobel, Norbert; in 2021 20th International Conference on Advanced Robotics (ICAR) (2021). 857–864.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.

  @inproceedings{9659355, abstract = {To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.}, author = {Spiess, Florian and Friesslich, Jonas and Bluemm, Daniel and Mast, Fabio and Vinokour, Dmitrij and Kounev, Samuel and Kaupp, Tobias and Strobel, Norbert}, booktitle = {2021 20th International Conference on Advanced Robotics (ICAR)}, keywords = {t_interdisciplinary}, month = 12, pages = {857-864}, title = {Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment}, year = 2021 }

  %0 Conference Paper %1 9659355 %A Spiess, Florian %A Friesslich, Jonas %A Bluemm, Daniel %A Mast, Fabio %A Vinokour, Dmitrij %A Kounev, Samuel %A Kaupp, Tobias %A Strobel, Norbert %B 2021 20th International Conference on Advanced Robotics (ICAR) %D 2021 %P 857-864 %T Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment %U https://ieeexplore.ieee.org/document/9659355/ %X To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.
• 
  SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications. Grohmann, Johannes; Straesser, Martin; Chalbani, Avi; Eismann, Simon; Arian, Yair; Herbst, Nikolas; Peretz, Noam; Kounev, Samuel; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021). ACM, New York, NY, USA.

  Acceptance Rate: 29%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.

  @inproceedings{GrStChEiArHePeKo2021-ICPE, abstract = {Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing—a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90\%.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Straesser, Martin and Chalbani, Avi and Eismann, Simon and Arian, Yair and Herbst, Nikolas and Peretz, Noam and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {performance}, month = {04}, note = {Acceptance Rate: 29%}, organization = {ACM}, title = {SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications}, year = 2021 }

  %0 Conference Paper %1 GrStChEiArHePeKo2021-ICPE %A Grohmann, Johannes %A Straesser, Martin %A Chalbani, Avi %A Eismann, Simon %A Arian, Yair %A Herbst, Nikolas %A Peretz, Noam %A Kounev, Samuel %B Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) %C New York, NY, USA %D 2021 %R https://doi.org/10.1145/3427921.3450248 %T SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications %U https://doi.org/10.1145/3427921.3450248 %X Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.

• 
  Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario. Prantl, Thomas; Ten, Peter; Iffländer, Lukas; Dmitrenko, Alexandra; Kounev, Samuel; Krupitzer, Christian; in 2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020).

  Acceptance Rate: 27%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.

  @inproceedings{PrTeIfDmKo2020-MASCOTS-GroupEncryption, abstract = {New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.}, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Performance_Evaluation}, month = 11, note = {Acceptance Rate: 27\%}, series = {MASCOTS '20}, title = {Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario}, year = 2020 }

  %0 Conference Paper %1 PrTeIfDmKo2020-MASCOTS-GroupEncryption %A Prantl, Thomas %A Ten, Peter %A Iffländer, Lukas %A Dmitrenko, Alexandra %A Kounev, Samuel %A Krupitzer, Christian %B 2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) %D 2020 %T Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario %X New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.
• 
  Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS. Grohmann, Johannes; Seybold, Daniel; Eismann, Simon; Leznik, Mark; Kounev, Samuel; Domaschka, Jörg; in 2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020). 1–8. IEEE.

  Acceptance Rate: 27%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.

  @inproceedings{GrSeEiLeKoDo2020-MASCOTS-DBMSPerformance, abstract = {Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.}, author = {Grohmann, Johannes and Seybold, Daniel and Eismann, Simon and Leznik, Mark and Kounev, Samuel and Domaschka, Jörg}, booktitle = {2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {automated_model_learning}, month = 11, note = {Acceptance Rate: 27%}, pages = {1–8}, publisher = {IEEE}, series = {MASCOTS '20}, title = {Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS}, year = 2020 }

  %0 Conference Paper %1 GrSeEiLeKoDo2020-MASCOTS-DBMSPerformance %A Grohmann, Johannes %A Seybold, Daniel %A Eismann, Simon %A Leznik, Mark %A Kounev, Samuel %A Domaschka, Jörg %B 2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) %D 2020 %I IEEE %P 1--8 %R 10.1109/MASCOTS50786.2020.9285960 %T Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS %U https://doi.org/10.1109/MASCOTS50786.2020.9285960 %X Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.
• 
  An IoT Network Emulator for Analyzing the Influence of Varying Network Quality. Herrnleben, Stefan; Ailabouni, Rudy; Grohmann, Johannes; Prantl, Thomas; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.

  @inproceedings{HeAiGrPrKrKo2020-SIMUtools-IoT-Network-Emulator, abstract = {IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.}, author = {Herrnleben, Stefan and Ailabouni, Rudy and Grohmann, Johannes and Prantl, Thomas and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, keywords = {networking}, month = {08}, series = {SIMUtools 2020}, title = {An IoT Network Emulator for Analyzing the Influence of Varying Network Quality}, year = 2020 }

  %0 Conference Paper %1 HeAiGrPrKrKo2020-SIMUtools-IoT-Network-Emulator %A Herrnleben, Stefan %A Ailabouni, Rudy %A Grohmann, Johannes %A Prantl, Thomas %A Krupitzer, Christian %A Kounev, Samuel %B Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) %D 2020 %R https://doi.org/10.1007/978-3-030-72795-6_47 %T An IoT Network Emulator for Analyzing the Influence of Varying Network Quality %U https://doi.org/10.1007/978-3-030-72795-6_47 %X IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.
• 
  Predicting the Costs of Serverless Workflows. Eismann, Simon; Grohmann, Johannes; van Eyk, Erwin; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) (2020). 265–276. Association for Computing Machinery (ACM), New York, NY, USA.

  Acceptance Rate: 23.4% (15/64)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.

  @inproceedings{EiGrEyHeKo2020-ICPE-ServerlessWorkflows, abstract = {Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.}, address = {New York, NY, USA}, author = {Eismann, Simon and Grohmann, Johannes and van Eyk, Erwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {se2}, month = {04}, note = {{Acceptance Rate: 23.4% (15/64)}}, pages = {265–276}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {Predicting the Costs of Serverless Workflows}, year = 2020 }

  %0 Conference Paper %1 EiGrEyHeKo2020-ICPE-ServerlessWorkflows %A Eismann, Simon %A Grohmann, Johannes %A van Eyk, Erwin %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) %C New York, NY, USA %D 2020 %I Association for Computing Machinery (ACM) %P 265--276 %R 10.1145/3358960.3379133 %T Predicting the Costs of Serverless Workflows %U https://doi.org/10.1145/3358960.3379133 %X Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%. %@ 9781450369916
• 
  Model-based Performance Predictions for SDN-based Networks: A Case Study. Herrnleben, Stefan; Rygielski, Piotr; Grohmann, Johannes; Eismann, Simon; Hossfeld, Tobias; Kounev, Samuel; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020). Springer, Cham.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.

  @inproceedings{HeRyGrEiHoKo-MMB2020-Model-based-SDN-Performance, abstract = {Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Rygielski, Piotr and Grohmann, Johannes and Eismann, Simon and Hossfeld, Tobias and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {SDN}, month = {03}, publisher = {Springer}, series = {MMB 2020}, title = {{Model-based Performance Predictions for SDN-based Networks: A Case Study}}, year = 2020 }

  %0 Conference Paper %1 HeRyGrEiHoKo-MMB2020-Model-based-SDN-Performance %A Herrnleben, Stefan %A Rygielski, Piotr %A Grohmann, Johannes %A Eismann, Simon %A Hossfeld, Tobias %A Kounev, Samuel %B Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems %C Cham %D 2020 %I Springer %R 10.1007/978-3-030-43024-5_6 %T Model-based Performance Predictions for SDN-based Networks: A Case Study %U https://doi.org/10.1007/978-3-030-43024-5_6 %X Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios. %@ 978-3-030-43024-5
• 
  To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows. Züfle, Marwin; Krupitzer, Christian; Erhard, Florian; Grohmann, Johannes; Kounev, Samuel; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020). 19–36. Springer, Cham.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.

  @inproceedings{ZuKrErGrKo-MMB2020-ToFailOrNotToFail, abstract = {Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.}, address = {Cham}, author = {Z{\"u}fle, Marwin and Krupitzer, Christian and Erhard, Florian and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {Reliability}, month = {03}, pages = {19–36}, publisher = {Springer}, series = {MMB 2020}, title = {{To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows}}, year = 2020 }

  %0 Conference Paper %1 ZuKrErGrKo-MMB2020-ToFailOrNotToFail %A Züfle, Marwin %A Krupitzer, Christian %A Erhard, Florian %A Grohmann, Johannes %A Kounev, Samuel %B Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems %C Cham %D 2020 %I Springer %P 19--36 %R 10.1007/978-3-030-43024-5_2 %T To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows %U https://doi.org/10.1007/978-3-030-43024-5_2 %X Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure. %@ 978-3-030-43024-5
• 
  Quantifying measurement quality and load distribution in Tor. Greubel, Andre; Pohl, Steffen; Kounev, Samuel; in Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.

  @article{greubel2020quantifying, abstract = {Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80\%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21\% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.}, author = {Greubel, Andre and Pohl, Steffen and Kounev, Samuel}, journal = {Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020)}, keywords = {tor}, title = {Quantifying measurement quality and load distribution in Tor}, year = 2020 }

  %0 Journal Article %1 greubel2020quantifying %A Greubel, Andre %A Pohl, Steffen %A Kounev, Samuel %D 2020 %J Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020) %T Quantifying measurement quality and load distribution in Tor %X Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.

• 
  Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning. Grohmann, Johannes; Nicholson, Patrick K.; Iglesias, Jesus Omana; Kounev, Samuel; Lugones, Diego; in Proceedings of the 20th International Middleware Conference (2019). 149–162. Association for Computing Machinery (ACM), New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics -- while still fulfilling application KPIs -- to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.

  @inproceedings{GrNiIgKoLu-MIDDLEWARE2019-Monitorless, abstract = {Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics – while still fulfilling application KPIs – to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Nicholson, Patrick K. and Iglesias, Jesus Omana and Kounev, Samuel and Lugones, Diego}, booktitle = {Proceedings of the 20th International Middleware Conference}, keywords = {Virtualization}, pages = {149–162}, publisher = {Association for Computing Machinery (ACM)}, series = {Middleware '19}, title = {Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning}, year = 2019 }

  %0 Conference Paper %1 GrNiIgKoLu-MIDDLEWARE2019-Monitorless %A Grohmann, Johannes %A Nicholson, Patrick K. %A Iglesias, Jesus Omana %A Kounev, Samuel %A Lugones, Diego %B Proceedings of the 20th International Middleware Conference %C New York, NY, USA %D 2019 %I Association for Computing Machinery (ACM) %P 149--162 %R 10.1145/3361525.3361543 %T Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning %U https://doi.org/10.1145/3361525.3361543 %X Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics -- while still fulfilling application KPIs -- to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload. %@ 9781450370097
• 
  Measuring the Energy Efficiency of Transactional Loads on GPGPU. von Kistowski, Jóakim; Pais, Johann; Wahl, Tobias; Lange, Klaus-Dieter; Block, Hansfried; Beckett, John; Kounev, Samuel; in Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering (2019). ACM, New York, NY, USA.

  Best Industry Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4% regarding power consumption.

  @inproceedings{KiPaWaLaBlBeKo2019-ICPE-GPGPUPower, abstract = {{General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4\% regarding power consumption.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Pais, Johann and Wahl, Tobias and Lange, Klaus-Dieter and Block, Hansfried and Beckett, John and Kounev, Samuel}, booktitle = {Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering}, keywords = {Power}, note = {{<b>Best Industry Paper Award</b>}}, publisher = {ACM}, series = {ICPE '19}, title = {{Measuring the Energy Efficiency of Transactional Loads on GPGPU}}, year = 2019 }

  %0 Conference Paper %1 KiPaWaLaBlBeKo2019-ICPE-GPGPUPower %A von Kistowski, Jóakim %A Pais, Johann %A Wahl, Tobias %A Lange, Klaus-Dieter %A Block, Hansfried %A Beckett, John %A Kounev, Samuel %B Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2019 %I ACM %T Measuring the Energy Efficiency of Transactional Loads on GPGPU %X General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4% regarding power consumption.
• 
  Predicting Server Power Consumption from Standard Rating Results. von Kistowski, Jóakim; Grohmann, Johannes; Schmitt, Norbert; Kounev, Samuel; in Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering (2019). 301–312. Association for Computing Machinery (ACM), New York, NY, USA.

  Full Paper Acceptance Rate: 18.6% (13/70)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.

  @inproceedings{KiGrScKo2019-ICPE-PowerPrediction, abstract = {Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Grohmann, Johannes and Schmitt, Norbert and Kounev, Samuel}, booktitle = {Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering}, keywords = {Power}, note = {{Full Paper Acceptance Rate: 18.6\% (13/70)}}, pages = {301–312}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '19}, title = {{Predicting Server Power Consumption from Standard Rating Results}}, year = 2019 }

  %0 Conference Paper %1 KiGrScKo2019-ICPE-PowerPrediction %A von Kistowski, Jóakim %A Grohmann, Johannes %A Schmitt, Norbert %A Kounev, Samuel %B Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2019 %I Association for Computing Machinery (ACM) %P 301--312 %R 10.1145/3297663.3310298 %T Predicting Server Power Consumption from Standard Rating Results %U https://doi.org/10.1145/3297663.3310298 %X Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers. %@ 9781450362399
• 
  Online Power Consumption Estimation for Functions in Cloud Applications. Schmitt, Norbert; Iffländer, Lukas; Bauer, André; Kounev, Samuel; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{ScIfBaKo2019-ICAC-OnlinePowerEstimation, author = {Schmitt, Norbert and Iffländer, Lukas and Bauer, André and Kounev, Samuel}, booktitle = {Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC)}, keywords = {Power}, month = {06}, publisher = {IEEE}, title = {Online Power Consumption Estimation for Functions in Cloud Applications}, year = 2019 }

  %0 Conference Paper %1 ScIfBaKo2019-ICAC-OnlinePowerEstimation %A Schmitt, Norbert %A Iffländer, Lukas %A Bauer, André %A Kounev, Samuel %B Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) %D 2019 %I IEEE %T Online Power Consumption Estimation for Functions in Cloud Applications
• 
  Integrating Statistical Response Time Models in Architectural Performance Models. Eismann, Simon; Grohmann, Johannes; Walter, Jürgen; von Kistowski, Jóakim; Kounev, Samuel; in Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) (2019). 71–80. IEEE.

  Acceptance Rate: 21,9% (21/96)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.

  @inproceedings{EiGrWaKiKo2019-ICSA-Integrating, abstract = {Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.}, author = {Eismann, Simon and Grohmann, Johannes and Walter, J{\"u}rgen and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA)}, keywords = {PRISMA}, month = {03}, note = {Acceptance Rate: 21,9\% (21/96)}, pages = {71–80}, publisher = {IEEE}, title = {{Integrating Statistical Response Time Models in Architectural Performance Models}}, year = 2019 }

  %0 Conference Paper %1 EiGrWaKiKo2019-ICSA-Integrating %A Eismann, Simon %A Grohmann, Johannes %A Walter, Jürgen %A von Kistowski, Jóakim %A Kounev, Samuel %B Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) %D 2019 %I IEEE %P 71--80 %R 10.1109/ICSA.2019.00016 %T Integrating Statistical Response Time Models in Architectural Performance Models %U https://doi.org/10.1109/ICSA.2019.00016 %X Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.
• 
  Autonomic Forecasting Method Selection: Examination and Ways Ahead. Züfle, Marwin; Bauer, André; Lesch, Veronika; Krupitzer, Christian; Herbst, Nikolas; Kounev, Samuel; Curtef, Valentin; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{ZuBaLeKrHeKoCu2019-ICAC-Autonomic, author = {Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Lesch, Veronika and Krupitzer, Christian and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 16th {IEEE} International Conference on Autonomic Computing (ICAC)}, keywords = {PRISMA}, month = {06}, publisher = {IEEE}, title = {{Autonomic Forecasting Method Selection: Examination and Ways Ahead}}, year = 2019 }

  %0 Conference Paper %1 ZuBaLeKrHeKoCu2019-ICAC-Autonomic %A Züfle, Marwin %A Bauer, André %A Lesch, Veronika %A Krupitzer, Christian %A Herbst, Nikolas %A Kounev, Samuel %A Curtef, Valentin %B Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) %D 2019 %I IEEE %T Autonomic Forecasting Method Selection: Examination and Ways Ahead
• 
  Chamulteon: Coordinated Auto-Scaling of Micro-Services. Bauer, André; Lesch, Veronika; Versluis, Laurens; Ilyushkin, Alexey; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS) (2019).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{BaLeVeIlHeKo2019-ICDCS-Chamulteon, author = {Bauer, Andr{\'e} and Lesch, Veronika and Versluis, Laurens and Ilyushkin, Alexey and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS)}, keywords = {SPEC}, month = {07}, title = {Chamulteon: Coordinated Auto-Scaling of Micro-Services}, year = 2019 }

  %0 Conference Paper %1 BaLeVeIlHeKo2019-ICDCS-Chamulteon %A Bauer, André %A Lesch, Veronika %A Versluis, Laurens %A Ilyushkin, Alexey %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS) %D 2019 %T Chamulteon: Coordinated Auto-Scaling of Micro-Services
• 
  Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques. Grohmann, Johannes; Eismann, Simon; Elflein, Sven; Mazkatli, Manar; von Kistowski, Jóakim; Kounev, Samuel; in 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2019). 309–322. IEEE Computer Society.

  Acceptance Rate: 23.8% (29/122)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.

  @inproceedings{GrEiElMaKiKo2019-MASCOTS-DependencyIdentification, abstract = {Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.}, author = {Grohmann, Johannes and Eismann, Simon and Elflein, Sven and Mazkatli, Manar and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Automated_model_learning}, month = 10, note = {{Acceptance Rate: 23.8% (29/122)}}, pages = {309–322}, publisher = {IEEE Computer Society}, series = {MASCOTS '19}, title = {{Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques}}, year = 2019 }

  %0 Conference Paper %1 GrEiElMaKiKo2019-MASCOTS-DependencyIdentification %A Grohmann, Johannes %A Eismann, Simon %A Elflein, Sven %A Mazkatli, Manar %A von Kistowski, Jóakim %A Kounev, Samuel %B 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) %D 2019 %I IEEE Computer Society %P 309--322 %R 10.1109/MASCOTS.2019.00042 %T Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques %U https://doi.org/10.1109/MASCOTS.2019.00042 %X Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard. %@ 978-1-7281-4950-9

• 
  SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network. Greubel, Andre; Dmitrienko, Alexandra; Kounev, Samuel; in Proceedings of the 34th Annual Computer Security Applications Conference (2018). ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{GrDmKo2018-ACSAC-SmarTor, address = {New York, NY, USA}, author = {Greubel, Andre and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {Proceedings of the 34th Annual Computer Security Applications Conference}, keywords = {sss-group}, month = 12, organization = {ACM}, series = {ACSAC '18}, title = {{SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network}}, year = 2018 }

  %0 Conference Paper %1 GrDmKo2018-ACSAC-SmarTor %A Greubel, Andre %A Dmitrienko, Alexandra %A Kounev, Samuel %B Proceedings of the 34th Annual Computer Security Applications Conference %C New York, NY, USA %D 2018 %T SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network
• 
  TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research. von Kistowski, Jóakim; Eismann, Simon; Schmitt, Norbert; Bauer, André; Grohmann, Johannes; Kounev, Samuel; in Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (2018). 223–236. IEEE Computer Society.

  Acceptance Rate: 29.5% (23/78)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.

  @inproceedings{KiEiScBaGrKo2018-MASCOTS-TeaStore, abstract = {Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.}, author = {von Kistowski, J{\'o}akim and Eismann, Simon and Schmitt, Norbert and Bauer, Andr{\'e} and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems}, keywords = {Power}, month = {09}, note = {{Acceptance Rate: 29.5\% (23/78)}}, pages = {223–236}, publisher = {IEEE Computer Society}, series = {MASCOTS '18}, title = {{TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research}}, year = 2018 }

  %0 Conference Paper %1 KiEiScBaGrKo2018-MASCOTS-TeaStore %A von Kistowski, Jóakim %A Eismann, Simon %A Schmitt, Norbert %A Bauer, André %A Grohmann, Johannes %A Kounev, Samuel %B Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems %D 2018 %I IEEE Computer Society %P 223--236 %R 10.1109/MASCOTS.2018.00030 %T TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research %U https://doi.org/10.1109/MASCOTS.2018.00030 %X Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.
• 
  Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time. Eismann, Simon; Walter, Jürgen; von Kistowski, Jóakim; Kounev, Samuel; in 2018 IEEE International Conference on Software Architecture (ICSA) (2018). 135–144.

  Acceptance Rate: 25,6% (22/86)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.

  @inproceedings{EiWaKiKo2018-ParametricDependencies, abstract = {Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.}, author = {Eismann, Simon and Walter, J{\"u}rgen and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {2018 IEEE International Conference on Software Architecture (ICSA)}, keywords = {Meta-models}, month = {04}, note = {Acceptance Rate: 25,6\% (22/86)}, pages = {135-144}, title = {{Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time}}, year = 2018 }

  %0 Conference Paper %1 EiWaKiKo2018-ParametricDependencies %A Eismann, Simon %A Walter, Jürgen %A von Kistowski, Jóakim %A Kounev, Samuel %B 2018 IEEE International Conference on Software Architecture (ICSA) %D 2018 %P 135-144 %T Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time %U https://ieeexplore.ieee.org/document/8417146 %X Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.
• 
  FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds. Lesch, Veronika; Bauer, André; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) (2018). ACM, New York, NY, USA.

  Full Paper Acceptance Ratio: 23,7%
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{LeBaHeKo-ICPE18-FOX-AS-COST, address = {New York, NY, USA}, author = {Lesch, Veronika and Bauer, Andr{\'e} and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018)}, keywords = {BUNGEE}, month = {04}, note = {Full Paper Acceptance Ratio: 23,7%}, publisher = {ACM}, title = {{FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds}}, year = 2018 }

  %0 Conference Paper %1 LeBaHeKo-ICPE18-FOX-AS-COST %A Lesch, Veronika %A Bauer, André %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) %C New York, NY, USA %D 2018 %I ACM %T FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds %U https://doi.org/10.1145/3184407.3184415
• 
  On the Value of Service Demand Estimation for Auto-Scaling. Bauer, André; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018) (2018). (Vol. 10740) 142–156. Springer, Cham.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.

  @inproceedings{BaGrHeKo2018-MMB-ServiceDemand, abstract = {In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.}, address = {Cham}, author = {Bauer, Andr{\'e} and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018)}, keywords = {Virtualization}, month = {02}, pages = {142–156}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{On the Value of Service Demand Estimation for Auto-Scaling}}, volume = 10740, year = 2018 }

  %0 Conference Paper %1 BaGrHeKo2018-MMB-ServiceDemand %A Bauer, André %A Grohmann, Johannes %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018) %C Cham %D 2018 %I Springer %P 142--156 %R 10.1007/978-3-319-74947-1_10 %T On the Value of Service Demand Estimation for Auto-Scaling %U https://doi.org/10.1007/978-3-319-74947-1_10 %V 10740 %X In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers. %@ 978-3-319-74947-1

• 
  Performance Assessment of Cloud Migrations from Network and Application Point of View. Iffländer, Lukas; Wamser, Florian; Metter, Christopher; Tran-Gia, Phuoc; Kounev, Samuel; in Proceedings of 9th EAI International Conference on Mobile Networks and Management (MONAMI 2018) (2017).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{IfWaMeTrKo-Monami-Migration-Performance, author = {Iffländer, Lukas and Wamser, Florian and Metter, Christopher and Tran-Gia, Phuoc and Kounev, Samuel}, booktitle = {{Proceedings of 9th EAI International Conference on Mobile Networks and Management (MONAMI 2018)}}, keywords = {Virtualization}, month = 12, title = {Performance Assessment of Cloud Migrations from Network and Application Point of View}, year = 2017 }

  %0 Conference Paper %1 IfWaMeTrKo-Monami-Migration-Performance %A Iffländer, Lukas %A Wamser, Florian %A Metter, Christopher %A Tran-Gia, Phuoc %A Kounev, Samuel %B Proceedings of 9th EAI International Conference on Mobile Networks and Management (MONAMI 2018) %D 2017 %T Performance Assessment of Cloud Migrations from Network and Application Point of View
• 
  Automated and Adaptable Decision Support for Software Performance Engineering. Walter, Jürgen; van Hoorn, Andre; Kounev, Samuel; in Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.

  @inproceedings{WalterHoKo2017-DecisionSupport, abstract = {Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.}, author = {Walter, J{\"u}rgen and van Hoorn, Andre and Kounev, Samuel}, booktitle = {Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools}, keywords = {DQL}, month = 12, title = {{Automated and Adaptable Decision Support for Software Performance Engineering}}, year = 2017 }

  %0 Conference Paper %1 WalterHoKo2017-DecisionSupport %A Walter, Jürgen %A van Hoorn, Andre %A Kounev, Samuel %B Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools %D 2017 %T Automated and Adaptable Decision Support for Software Performance Engineering %X Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.

• 
  Variations in CPU Power Consumption. von Kistowski, Jóakim; Block, Hansfried; Beckett, John; Spradling, Cloyce; Lange, Klaus-Dieter; Kounev, Samuel; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016). ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Experimental analysis of computer systems' power consumption has become an integral part of system performance evaluation, efficiency management, and model-based analysis. As with all measurements, repeatability and reproducibility of power measurements is a major challenge. Nominally identical systems can have different power consumption running the same workload under otherwise identical conditions. This behavior can also be observed for individual system components. Specifically, CPU power consumption can vary amongst different samples of nominally identical CPUs. This in turn has a significant impact on the overall system power, considering that a system's processor is the largest and most dynamic power consumer of the overall system. The concrete impact of CPU sample power variations is unknown, as comprehensive studies about differences in power consumption for nominally identical systems are currently missing. We address this lack of studies by conducting measurements on four different processor types from two different architectures. For each of these types, we compare up to 30 physical processor samples with a total sum of 90 samples over all processor types. We analyze the variations in power consumption for the different samples using six different workloads over five load levels. Additionally, we analyze how these variations change for different processor core counts and architectures. The results of this paper show that selection of a processor sample can have a statistically significant impact on power consumption. With no correlation to performance, power consumption for nominally identical processors can differ as much as 29.6% in idle and 19.5% at full load. We also show that these variations change over different architectures and processor types.

  @inproceedings{KiBlBeSpLaKo2016-ICPE-PowerVariation, abstract = {{Experimental analysis of computer systems' power consumption has become an integral part of system performance evaluation, efficiency management, and model-based analysis. As with all measurements, repeatability and reproducibility of power measurements is a major challenge. Nominally identical systems can have different power consumption running the same workload under otherwise identical conditions. This behavior can also be observed for individual system components. Specifically, CPU power consumption can vary amongst different samples of nominally identical CPUs. This in turn has a significant impact on the overall system power, considering that a system's processor is the largest and most dynamic power consumer of the overall system. The concrete impact of CPU sample power variations is unknown, as comprehensive studies about differences in power consumption for nominally identical systems are currently missing. We address this lack of studies by conducting measurements on four different processor types from two different architectures. For each of these types, we compare up to 30 physical processor samples with a total sum of 90 samples over all processor types. We analyze the variations in power consumption for the different samples using six different workloads over five load levels. Additionally, we analyze how these variations change for different processor core counts and architectures. The results of this paper show that selection of a processor sample can have a statistically significant impact on power consumption. With no correlation to performance, power consumption for nominally identical processors can differ as much as 29.6\% in idle and 19.5\% at full load. We also show that these variations change over different architectures and processor types.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Block, Hansfried and Beckett, John and Spradling, Cloyce and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}, keywords = {Power}, month = {03}, publisher = {ACM}, title = {{Variations in CPU Power Consumption}}, year = 2016 }

  %0 Conference Paper %1 KiBlBeSpLaKo2016-ICPE-PowerVariation %A von Kistowski, Jóakim %A Block, Hansfried %A Beckett, John %A Spradling, Cloyce %A Lange, Klaus-Dieter %A Kounev, Samuel %B Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) %C New York, NY, USA %D 2016 %I ACM %T Variations in CPU Power Consumption %X Experimental analysis of computer systems' power consumption has become an integral part of system performance evaluation, efficiency management, and model-based analysis. As with all measurements, repeatability and reproducibility of power measurements is a major challenge. Nominally identical systems can have different power consumption running the same workload under otherwise identical conditions. This behavior can also be observed for individual system components. Specifically, CPU power consumption can vary amongst different samples of nominally identical CPUs. This in turn has a significant impact on the overall system power, considering that a system's processor is the largest and most dynamic power consumer of the overall system. The concrete impact of CPU sample power variations is unknown, as comprehensive studies about differences in power consumption for nominally identical systems are currently missing. We address this lack of studies by conducting measurements on four different processor types from two different architectures. For each of these types, we compare up to 30 physical processor samples with a total sum of 90 samples over all processor types. We analyze the variations in power consumption for the different samples using six different workloads over five load levels. Additionally, we analyze how these variations change for different processor core counts and architectures. The results of this paper show that selection of a processor sample can have a statistically significant impact on power consumption. With no correlation to performance, power consumption for nominally identical processors can differ as much as 29.6% in idle and 19.5% at full load. We also show that these variations change over different architectures and processor types.
• 
  Automated Extraction of Network Traffic Models Suitable for Performance Simulation. Rygielski, Piotr; Simko, Viliam; Sittner, Felix; Aschenbrenner, Doris; Kounev, Samuel; Schilling, Klaus; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016). 27–35. ACM.

  Acceptance rate (Full Paper): 19/57 = 33%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Data centers are increasingly becoming larger and dynamic due to virtualization. In order to leverage the performance modeling and prediction techniques, such as Palladio Component Model or Descartes Modeling Language, in such a dynamic environments, it is necessary to automate the model extraction. Building and maintaining such models manually is not feasible anymore due to their size and the level of details. This paper is focused on traffic models that are an essential part of network infrastructure. Our goal is to decompose real traffic dumps into models suitable for performance prediction using Descartes Network Infrastructure modeling approach. The main challenge was to efficiently encode an arbitrary signal in the form of simple traffic generators while maintaining the shape of the original signal. We show that a typical 15 minute long tcpdump trace can be compressed to 0.4--15% of its original size whereas the relative median of extraction error is 0% for the most of the 69 examined traces.

  @inproceedings{RySiSiAschKoSch2016-dni-traffic-extraction-icpe2016, abstract = {{Data centers are increasingly becoming larger and dynamic due to virtualization. In order to leverage the performance modeling and prediction techniques, such as Palladio Component Model or Descartes Modeling Language, in such a dynamic environments, it is necessary to automate the model extraction. Building and maintaining such models manually is not feasible anymore due to their size and the level of details. This paper is focused on traffic models that are an essential part of network infrastructure. Our goal is to decompose real traffic dumps into models suitable for performance prediction using Descartes Network Infrastructure modeling approach. The main challenge was to efficiently encode an arbitrary signal in the form of simple traffic generators while maintaining the shape of the original signal. We show that a typical 15 minute long tcpdump trace can be compressed to 0.4–15% of its original size whereas the relative median of extraction error is 0% for the most of the 69 examined traces.}}, author = {Rygielski, Piotr and Simko, Viliam and Sittner, Felix and Aschenbrenner, Doris and Kounev, Samuel and Schilling, Klaus}, booktitle = {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}, keywords = {DNI}, month = {03}, note = {{Acceptance rate (Full Paper): 19/57 = 33\%}}, pages = {27–35}, publisher = {ACM}, title = {{Automated Extraction of Network Traffic Models Suitable for Performance Simulation}}, year = 2016 }

  %0 Conference Paper %1 RySiSiAschKoSch2016-dni-traffic-extraction-icpe2016 %A Rygielski, Piotr %A Simko, Viliam %A Sittner, Felix %A Aschenbrenner, Doris %A Kounev, Samuel %A Schilling, Klaus %B Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) %D 2016 %I ACM %P 27--35 %T Automated Extraction of Network Traffic Models Suitable for Performance Simulation %U http://dl.acm.org/citation.cfm?doid=2851553.2851570 %X Data centers are increasingly becoming larger and dynamic due to virtualization. In order to leverage the performance modeling and prediction techniques, such as Palladio Component Model or Descartes Modeling Language, in such a dynamic environments, it is necessary to automate the model extraction. Building and maintaining such models manually is not feasible anymore due to their size and the level of details. This paper is focused on traffic models that are an essential part of network infrastructure. Our goal is to decompose real traffic dumps into models suitable for performance prediction using Descartes Network Infrastructure modeling approach. The main challenge was to efficiently encode an arbitrary signal in the form of simple traffic generators while maintaining the shape of the original signal. We show that a typical 15 minute long tcpdump trace can be compressed to 0.4--15% of its original size whereas the relative median of extraction error is 0% for the most of the 69 examined traces.
• 
  Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments. Milenkoski, Aleksandar; Jayaram, K. R.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; in Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016) (2016). IEEE Computer Society, Washington DC, USA.

  Acceptance rate (Full Paper): 45/130 = 34%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.

  @inproceedings{MiJaAnViKo2016-ISSRE-Quantifying, abstract = {With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.}, address = {{Washington DC, USA}}, author = {Milenkoski, Aleksandar and Jayaram, K. R. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel}, booktitle = {{Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016)}}, keywords = {Virtualization}, month = 10, note = {{Acceptance rate (Full Paper): 45/130 = 34\%}}, organization = {IEEE}, publisher = {IEEE Computer Society}, title = {{Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments}}, year = 2016 }

  %0 Conference Paper %1 MiJaAnViKo2016-ISSRE-Quantifying %A Milenkoski, Aleksandar %A Jayaram, K. R. %A Antunes, Nuno %A Vieira, Marco %A Kounev, Samuel %B Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016) %C Washington DC, USA %D 2016 %I IEEE Computer Society %T Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments %U http://ieeexplore.ieee.org/document/7774527/?reload=true %X With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.
• 
  Modeling and Prediction of Software-Defined Networks Performance using Queueing Petri Nets. Rygielski, Piotr; Seliuchenko, Marian; Kounev, Samuel; in Proceedings of the Ninth International Conference on Simulation Tools and Techniques (SIMUTools 2016) (2016). 66–75.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Using various modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. Descartes Network Infrastructure (DNI) is a data center network performance modeling approach that addresses this challenge by offering multiple performance models but requiring to use only a single modeling language. In this paper, we thoroughly extend DNI to support new networking paradigms like, among others, Software-Defined Networking (SDN) and Network-Function Virtualization (NFV). Additionally, we demonstrate how SDN-based networks can be modeled using DNI and how are they transformed later into Queueing Petri Nets (QPN) using a model-to-model transformation. In the analysis of the performance prediction accuracy, we show that automatically generated QPN models represent the performance of heterogeneous SDN hardware with maximal prediction accuracy error of 12 percent.

  @inproceedings{RySeKo2016-SIMUTools-SDN-switches-QPN, abstract = {{Using various modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. Descartes Network Infrastructure (DNI) is a data center network performance modeling approach that addresses this challenge by offering multiple performance models but requiring to use only a single modeling language. In this paper, we thoroughly extend DNI to support new networking paradigms like, among others, Software-Defined Networking (SDN) and Network-Function Virtualization (NFV). Additionally, we demonstrate how SDN-based networks can be modeled using DNI and how are they transformed later into Queueing Petri Nets (QPN) using a model-to-model transformation. In the analysis of the performance prediction accuracy, we show that automatically generated QPN models represent the performance of heterogeneous SDN hardware with maximal prediction accuracy error of 12 percent.}}, author = {Rygielski, Piotr and Seliuchenko, Marian and Kounev, Samuel}, booktitle = {Proceedings of the Ninth International Conference on Simulation Tools and Techniques (SIMUTools 2016)}, keywords = {QPN}, month = {08}, pages = {66–75}, title = {{Modeling and Prediction of Software-Defined Networks Performance using Queueing Petri Nets}}, year = 2016 }

  %0 Conference Paper %1 RySeKo2016-SIMUTools-SDN-switches-QPN %A Rygielski, Piotr %A Seliuchenko, Marian %A Kounev, Samuel %B Proceedings of the Ninth International Conference on Simulation Tools and Techniques (SIMUTools 2016) %D 2016 %P 66--75 %T Modeling and Prediction of Software-Defined Networks Performance using Queueing Petri Nets %U http://dl.acm.org/citation.cfm?id=3021426.3021437 %X Using various modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. Descartes Network Infrastructure (DNI) is a data center network performance modeling approach that addresses this challenge by offering multiple performance models but requiring to use only a single modeling language. In this paper, we thoroughly extend DNI to support new networking paradigms like, among others, Software-Defined Networking (SDN) and Network-Function Virtualization (NFV). Additionally, we demonstrate how SDN-based networks can be modeled using DNI and how are they transformed later into Queueing Petri Nets (QPN) using a model-to-model transformation. In the analysis of the performance prediction accuracy, we show that automatically generated QPN models represent the performance of heterogeneous SDN hardware with maximal prediction accuracy error of 12 percent.
• 
  Performance Analysis of SDN Switches with Hardware and Software Flow Tables. Rygielski, Piotr; Seliuchenko, Marian; Kounev, Samuel; Klymash, Mykhailo; in Proceedings of the 10th EAI International Conference on Performance Evaluation Methodologies and Tools (ValueTools 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Nowadays data centers are increasingly becoming larger and dynamic due to virtualization. Software-Defined Networking is the leading approach to network virtualization and flexible management. The wide variety of hardware implementations have brought strong heterogeneity to the market of networking devices which are different in terms of OpenFlow features and performance. In this paper we address the issue of heterogeneity of four hardware OpenFlow switches by characterizing selected performance relevant parameters for the hardware and software flow tables. We characterize maximum size of hardware flow tables for each switch including the behavior of a rule promotion engine that moves the rules between tables. We show that in the worst case forwarding packets using software table decreases the throughput by two orders of magnitude (from 940 to 14 Mbit/s). Our results can help the developers of SDN applications to account performance limitations of hardware and software processing as well as limited hardware support for a specific rule types.

  @inproceedings{RySeKoKl2016-valuetools2016-sdn-switches-hw-sw-flow-tables, abstract = {{Nowadays data centers are increasingly becoming larger and dynamic due to virtualization. Software-Defined Networking is the leading approach to network virtualization and flexible management. The wide variety of hardware implementations have brought strong heterogeneity to the market of networking devices which are different in terms of OpenFlow features and performance. In this paper we address the issue of heterogeneity of four hardware OpenFlow switches by characterizing selected performance relevant parameters for the hardware and software flow tables. We characterize maximum size of hardware flow tables for each switch including the behavior of a rule promotion engine that moves the rules between tables. We show that in the worst case forwarding packets using software table decreases the throughput by two orders of magnitude (from 940 to 14 Mbit/s). Our results can help the developers of SDN applications to account performance limitations of hardware and software processing as well as limited hardware support for a specific rule types. }}, author = {Rygielski, Piotr and Seliuchenko, Marian and Kounev, Samuel and Klymash, Mykhailo}, booktitle = {Proceedings of the 10th EAI International Conference on Performance Evaluation Methodologies and Tools (ValueTools 2016)}, keywords = {DNI}, month = 10, title = {{Performance Analysis of SDN Switches with Hardware and Software Flow Tables}}, year = 2016 }

  %0 Conference Paper %1 RySeKoKl2016-valuetools2016-sdn-switches-hw-sw-flow-tables %A Rygielski, Piotr %A Seliuchenko, Marian %A Kounev, Samuel %A Klymash, Mykhailo %B Proceedings of the 10th EAI International Conference on Performance Evaluation Methodologies and Tools (ValueTools 2016) %D 2016 %T Performance Analysis of SDN Switches with Hardware and Software Flow Tables %X Nowadays data centers are increasingly becoming larger and dynamic due to virtualization. Software-Defined Networking is the leading approach to network virtualization and flexible management. The wide variety of hardware implementations have brought strong heterogeneity to the market of networking devices which are different in terms of OpenFlow features and performance. In this paper we address the issue of heterogeneity of four hardware OpenFlow switches by characterizing selected performance relevant parameters for the hardware and software flow tables. We characterize maximum size of hardware flow tables for each switch including the behavior of a rule promotion engine that moves the rules between tables. We show that in the worst case forwarding packets using software table decreases the throughput by two orders of magnitude (from 940 to 14 Mbit/s). Our results can help the developers of SDN applications to account performance limitations of hardware and software processing as well as limited hardware support for a specific rule types.

• 
  Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection. Milenkoski, Aleksandar; Payne, Bryan D.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; Avritzer, Alberto; Luft, Matthias; in The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015) (2015). Springer.

  Acceptance Rate: 23%.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.

  @inproceedings{MiPaAnViKoAvLu2015-RAID-Challenges, abstract = {{The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.}}, author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Luft, Matthias}, booktitle = {The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015)}, keywords = {Virtualization}, month = {{November}}, note = {{Acceptance Rate: 23%.}}, publisher = {{Springer}}, title = {{Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection}}, year = 2015 }

  %0 Conference Paper %1 MiPaAnViKoAvLu2015-RAID-Challenges %A Milenkoski, Aleksandar %A Payne, Bryan D. %A Antunes, Nuno %A Vieira, Marco %A Kounev, Samuel %A Avritzer, Alberto %A Luft, Matthias %B The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015) %D 2015 %I Springer %T Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection %U http://link.springer.com/chapter/10.1007/978-3-319-26362-5_22 %X The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.
• 
  Proactive Memory Scaling of Virtualized Applications. Spinner, Simon; Herbst, Nikolas; Kounev, Samuel; Zhu, Xiaoyun; Lu, Lei; Uysal, Mustafa; Griffith, Rean; in Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015) (2015). 277–284. IEEE.

  Acceptance Rate: 15%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.

  @inproceedings{SpHeKoZhLuUyGr2015-CLOUD-ProactiveMemoryScaling, abstract = {{Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.}}, author = {Spinner, Simon and Herbst, Nikolas and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Griffith, Rean}, booktitle = {Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015)}, keywords = {Self-adaptive-systems}, month = {06}, note = {Acceptance Rate: 15%}, pages = {277–284}, publisher = {IEEE}, title = {{Proactive Memory Scaling of Virtualized Applications}}, year = 2015 }

  %0 Conference Paper %1 SpHeKoZhLuUyGr2015-CLOUD-ProactiveMemoryScaling %A Spinner, Simon %A Herbst, Nikolas %A Kounev, Samuel %A Zhu, Xiaoyun %A Lu, Lei %A Uysal, Mustafa %A Griffith, Rean %B Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015) %D 2015 %I IEEE %P 277--284 %T Proactive Memory Scaling of Virtualized Applications %X Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.
• 
  Energy Efficiency of Hierarchical Server Load Distribution Strategies. von Kistowski, Jóakim; Beckett, John; Lange, Klaus-Dieter; Block, Hansfried; Arnold, Jeremy A.; Kounev, Samuel; in Proceedings of the IEEE 23nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2015) (2015). IEEE.

  Full paper acceptance rate: 19%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Energy efficiency of servers has become a significant issue over the last years. Load distribution plays a crucial role in the improvement of energy efficiency as (un-)balancing strategies can be leveraged to distribute load over one or multiple systems in a way in which resources are utilized at high performance, yet low overall power consumption. This can be achieved on multiple levels, from load distribution on single CPU cores to machine level load balancing on distributed systems. With modern day server architectures providing load balancing opportunities at several layers, answering the question of optimal load distribution has become non-trivial. Work has to be distributed hierarchically in a fashion that enables maximum energy efficiency at each level. Current approaches balance load based on generalized assumptions about the energy efficiency of servers. These assumptions are based either on very machine-specific or highly generalized observations that may or may not hold true over a variety of systems and configurations. In this paper, we use a modified version of the SPEC SERT suite to measure the energy efficiency of a variety of hierarchical load distribution strategies on single and multi-node systems. We introduce a new strategy and evaluate energy efficiency for homogeneous and heterogeneous workloads over different hardware configurations. Our results show that the selection of a load distribution strategy depends heavily on workload, system utilization, as well as hardware. Used in conjunction with existing strategies, our new load distribution strategy can reduce a single system's power consumption by up to 10.7%.

  @inproceedings{KiBeLaBlArKo2015-MASCOTS, abstract = {{Energy efficiency of servers has become a significant issue over the last years. Load distribution plays a crucial role in the improvement of energy efficiency as (un-)balancing strategies can be leveraged to distribute load over one or multiple systems in a way in which resources are utilized at high performance, yet low overall power consumption. This can be achieved on multiple levels, from load distribution on single CPU cores to machine level load balancing on distributed systems. With modern day server architectures providing load balancing opportunities at several layers, answering the question of optimal load distribution has become non-trivial. Work has to be distributed hierarchically in a fashion that enables maximum energy efficiency at each level. Current approaches balance load based on generalized assumptions about the energy efficiency of servers. These assumptions are based either on very machine-specific or highly generalized observations that may or may not hold true over a variety of systems and configurations. In this paper, we use a modified version of the SPEC SERT suite to measure the energy efficiency of a variety of hierarchical load distribution strategies on single and multi-node systems. We introduce a new strategy and evaluate energy efficiency for homogeneous and heterogeneous workloads over different hardware configurations. Our results show that the selection of a load distribution strategy depends heavily on workload, system utilization, as well as hardware. Used in conjunction with existing strategies, our new load distribution strategy can reduce a single system's power consumption by up to 10.7%.}}, author = {von Kistowski, J{\'o}akim and Beckett, John and Lange, Klaus-Dieter and Block, Hansfried and Arnold, Jeremy A. and Kounev, Samuel}, booktitle = {Proceedings of the IEEE 23nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2015)}, keywords = {Power}, month = 10, note = {Full paper acceptance rate: 19\%}, publisher = {IEEE}, title = {{Energy Efficiency of Hierarchical Server Load Distribution Strategies}}, year = 2015 }

  %0 Conference Paper %1 KiBeLaBlArKo2015-MASCOTS %A von Kistowski, Jóakim %A Beckett, John %A Lange, Klaus-Dieter %A Block, Hansfried %A Arnold, Jeremy A. %A Kounev, Samuel %B Proceedings of the IEEE 23nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2015) %D 2015 %I IEEE %T Energy Efficiency of Hierarchical Server Load Distribution Strategies %X Energy efficiency of servers has become a significant issue over the last years. Load distribution plays a crucial role in the improvement of energy efficiency as (un-)balancing strategies can be leveraged to distribute load over one or multiple systems in a way in which resources are utilized at high performance, yet low overall power consumption. This can be achieved on multiple levels, from load distribution on single CPU cores to machine level load balancing on distributed systems. With modern day server architectures providing load balancing opportunities at several layers, answering the question of optimal load distribution has become non-trivial. Work has to be distributed hierarchically in a fashion that enables maximum energy efficiency at each level. Current approaches balance load based on generalized assumptions about the energy efficiency of servers. These assumptions are based either on very machine-specific or highly generalized observations that may or may not hold true over a variety of systems and configurations. In this paper, we use a modified version of the SPEC SERT suite to measure the energy efficiency of a variety of hierarchical load distribution strategies on single and multi-node systems. We introduce a new strategy and evaluate energy efficiency for homogeneous and heterogeneous workloads over different hardware configurations. Our results show that the selection of a load distribution strategy depends heavily on workload, system utilization, as well as hardware. Used in conjunction with existing strategies, our new load distribution strategy can reduce a single system's power consumption by up to 10.7%.
• 
  Modeling and Extracting Load Intensity Profiles. von Kistowski, Jóakim; Herbst, Nikolas Roman; Zoller, Daniel; Kounev, Samuel; Hotho, Andreas; in Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) (2015).

  Acceptance rate: 29%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Benchmarking of systems under these constraints is difficult, as state-of-the-art benchmarking frameworks provide only limited support for emulating such dynamic and highly variable load profiles for the creation of realistic workload scenarios. Industrial benchmarks typically confine themselves to workloads with constant or stepwise increasing loads. Alternatively, they support replaying of recorded load traces. Statistical load intensity descriptions also do not sufficiently capture concrete pattern load profile variations over time. To address these issues, we present the Descartes Load Intensity Model (DLIM). DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance can be used as a compact representation of a recorded load intensity trace, providing a powerful tool for benchmarking and performance analysis. As manually obtaining DLIM instances can be time consuming, we present three different automated extraction methods, which also help to enable autonomous system analysis for self-adaptive systems. Model expressiveness is validated using the presented extraction methods. Extracted DLIM instances exhibit a median modeling error of 12.4% on average over nine different real-world traces covering between two weeks and seven months. Additionally, extraction methods perform orders of magnitude faster than existing time series decomposition approaches.

  @inproceedings{KiHeZoKoHo2015_SEAMS_DLIMExtraction, abstract = {Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Benchmarking of systems under these constraints is difficult, as state-of-the-art benchmarking frameworks provide only limited support for emulating such dynamic and highly variable load profiles for the creation of realistic workload scenarios. Industrial benchmarks typically confine themselves to workloads with constant or stepwise increasing loads. Alternatively, they support replaying of recorded load traces. Statistical load intensity descriptions also do not sufficiently capture concrete pattern load profile variations over time. To address these issues, we present the Descartes Load Intensity Model (DLIM). DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance can be used as a compact representation of a recorded load intensity trace, providing a powerful tool for benchmarking and performance analysis. As manually obtaining DLIM instances can be time consuming, we present three different automated extraction methods, which also help to enable autonomous system analysis for self-adaptive systems. Model expressiveness is validated using the presented extraction methods. Extracted DLIM instances exhibit a median modeling error of 12.4% on average over nine different real-world traces covering between two weeks and seven months. Additionally, extraction methods perform orders of magnitude faster than existing time series decomposition approaches.}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas Roman and Zoller, Daniel and Kounev, Samuel and Hotho, Andreas}, booktitle = {Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015)}, keywords = {LIMBO}, month = {05}, note = {Acceptance rate: 29\%}, title = {{Modeling and Extracting Load Intensity Profiles}}, year = 2015 }

  %0 Conference Paper %1 KiHeZoKoHo2015_SEAMS_DLIMExtraction %A von Kistowski, Jóakim %A Herbst, Nikolas Roman %A Zoller, Daniel %A Kounev, Samuel %A Hotho, Andreas %B Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) %D 2015 %T Modeling and Extracting Load Intensity Profiles %X Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Benchmarking of systems under these constraints is difficult, as state-of-the-art benchmarking frameworks provide only limited support for emulating such dynamic and highly variable load profiles for the creation of realistic workload scenarios. Industrial benchmarks typically confine themselves to workloads with constant or stepwise increasing loads. Alternatively, they support replaying of recorded load traces. Statistical load intensity descriptions also do not sufficiently capture concrete pattern load profile variations over time. To address these issues, we present the Descartes Load Intensity Model (DLIM). DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance can be used as a compact representation of a recorded load intensity trace, providing a powerful tool for benchmarking and performance analysis. As manually obtaining DLIM instances can be time consuming, we present three different automated extraction methods, which also help to enable autonomous system analysis for self-adaptive systems. Model expressiveness is validated using the presented extraction methods. Extracted DLIM instances exhibit a median modeling error of 12.4% on average over nine different real-world traces covering between two weeks and seven months. Additionally, extraction methods perform orders of magnitude faster than existing time series decomposition approaches.
• 
  BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments. Herbst, Nikolas Roman; Kounev, Samuel; Weber, Andreas; Groenda, Henning; in Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) (2015).

  Acceptance rate: 29%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Today's infrastructure clouds provide resource elasticity (i.e. auto-scaling) mechanisms enabling self-adaptive resource provisioning to reflect variations in the load intensity over time. These mechanisms impact on the application performance, however, their effect in specific situations is hard to quantify and compare. To evaluate the quality of elasticity mechanisms provided by different platforms and configurations, respective metrics and benchmarks are required. Existing metrics for elasticity only consider the time required to provision and deprovision resources or the costs impact of adaptations. Existing benchmarks lack the capability to handle open workloads with realistic load intensity profiles and do not explicitly distinguish between the performance exhibited by the provisioned underlying resources, on the one hand, and the quality of the elasticity mechanisms themselves, on the other hand. In this paper, we propose reliable metrics for quantifying the timing aspects and accuracy of elasticity. Based on these metrics, we propose a novel approach for benchmarking the elasticity of Infrastructure-as-a-Service (IaaS) cloud platforms independent of the performance exhibited by the provisioned underlying resources. We show that the proposed metrics provide consistent ranking of elastic platforms on an ordinal scale. Finally, we present an extensive case study of real-world complexity demonstrating that the proposed approach is applicable in realistic scenarios and can cope with different levels of resource efficiency.

  @inproceedings{HeKoWeGr_2015_SEAMS_BUNGEE, abstract = {Today's infrastructure clouds provide resource elasticity (i.e. auto-scaling) mechanisms enabling self-adaptive resource provisioning to reflect variations in the load intensity over time. These mechanisms impact on the application performance, however, their effect in specific situations is hard to quantify and compare. To evaluate the quality of elasticity mechanisms provided by different platforms and configurations, respective metrics and benchmarks are required. Existing metrics for elasticity only consider the time required to provision and deprovision resources or the costs impact of adaptations. Existing benchmarks lack the capability to handle open workloads with realistic load intensity profiles and do not explicitly distinguish between the performance exhibited by the provisioned underlying resources, on the one hand, and the quality of the elasticity mechanisms themselves, on the other hand. In this paper, we propose reliable metrics for quantifying the timing aspects and accuracy of elasticity. Based on these metrics, we propose a novel approach for benchmarking the elasticity of Infrastructure-as-a-Service (IaaS) cloud platforms independent of the performance exhibited by the provisioned underlying resources. We show that the proposed metrics provide consistent ranking of elastic platforms on an ordinal scale. Finally, we present an extensive case study of real-world complexity demonstrating that the proposed approach is applicable in realistic scenarios and can cope with different levels of resource efficiency.}, author = {Herbst, Nikolas Roman and Kounev, Samuel and Weber, Andreas and Groenda, Henning}, booktitle = {Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015)}, keywords = {LIMBO}, month = {05}, note = {Acceptance rate: 29\%}, title = {{BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments}}, year = 2015 }

  %0 Conference Paper %1 HeKoWeGr_2015_SEAMS_BUNGEE %A Herbst, Nikolas Roman %A Kounev, Samuel %A Weber, Andreas %A Groenda, Henning %B Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) %D 2015 %T BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments %X Today's infrastructure clouds provide resource elasticity (i.e. auto-scaling) mechanisms enabling self-adaptive resource provisioning to reflect variations in the load intensity over time. These mechanisms impact on the application performance, however, their effect in specific situations is hard to quantify and compare. To evaluate the quality of elasticity mechanisms provided by different platforms and configurations, respective metrics and benchmarks are required. Existing metrics for elasticity only consider the time required to provision and deprovision resources or the costs impact of adaptations. Existing benchmarks lack the capability to handle open workloads with realistic load intensity profiles and do not explicitly distinguish between the performance exhibited by the provisioned underlying resources, on the one hand, and the quality of the elasticity mechanisms themselves, on the other hand. In this paper, we propose reliable metrics for quantifying the timing aspects and accuracy of elasticity. Based on these metrics, we propose a novel approach for benchmarking the elasticity of Infrastructure-as-a-Service (IaaS) cloud platforms independent of the performance exhibited by the provisioned underlying resources. We show that the proposed metrics provide consistent ranking of elastic platforms on an ordinal scale. Finally, we present an extensive case study of real-world complexity demonstrating that the proposed approach is applicable in realistic scenarios and can cope with different levels of resource efficiency.
• 
  Analysis of the Influences on Server Power Consumption and Energy Efficiency for CPU-Intensive Workloads. von Kistowski, Jóakim; Block, Hansfried; Beckett, John; Lange, Klaus-Dieter; Arnold, Jeremy A.; Kounev, Samuel; in Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015) (2015). ACM, New York, NY, USA.

  acceptance rate: 27%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Energy efficiency of servers has become a significant research topic over the last years, as server energy consumption varies depending on multiple factors, such as server utilization and workload type. Server energy analysis and estimation must take all relevant factors into account to ensure reliable estimates and conclusions. Thorough system analysis requires benchmarks capable of testing different system resources at different load levels using multiple workload types. Server energy estimation approaches, on the other hand, require knowledge about the interactions of these factors for the creation of accurate power models. Common approaches to energy-aware workload classification classify workloads depending on the resource types used by the different workloads. However, they rarely take into account differences in workloads targeting the same resources. Industrial energy-efficiency benchmarks typically do not evaluate the system's energy consumption at different resource load levels, and they only provide data for system analysis at maximum system load. In this paper, we benchmark multiple server configurations using the CPU worklets included in SPEC's Server Efficiency Rating Tool (SERT). We evaluate the impact of load levels and different CPU workloads on power consumption and energy efficiency. We analyze how functions approximating the measured power consumption differ over multiple server configurations and architectures. We show that workloads targeting the same resource can differ significantly in their power draw and energy efficiency. The power consumption of a given workload type varies depending on utilization, hardware and software configuration. The power consumption of CPU-intensive workloads does not scale uniformly with increased load, nor do hardware or software configuration changes affect it in a uniform manner.

  @inproceedings{KiBlBeLaArKo2015-ICPE-SERT, abstract = {{ Energy efficiency of servers has become a significant research topic over the last years, as server energy consumption varies depending on multiple factors, such as server utilization and workload type. Server energy analysis and estimation must take all relevant factors into account to ensure reliable estimates and conclusions. Thorough system analysis requires benchmarks capable of testing different system resources at different load levels using multiple workload types. Server energy estimation approaches, on the other hand, require knowledge about the interactions of these factors for the creation of accurate power models. Common approaches to energy-aware workload classification classify workloads depending on the resource types used by the different workloads. However, they rarely take into account differences in workloads targeting the same resources. Industrial energy-efficiency benchmarks typically do not evaluate the system's energy consumption at different resource load levels, and they only provide data for system analysis at maximum system load. In this paper, we benchmark multiple server configurations using the CPU worklets included in SPEC's Server Efficiency Rating Tool (SERT). We evaluate the impact of load levels and different CPU workloads on power consumption and energy efficiency. We analyze how functions approximating the measured power consumption differ over multiple server configurations and architectures. We show that workloads targeting the same resource can differ significantly in their power draw and energy efficiency. The power consumption of a given workload type varies depending on utilization, hardware and software configuration. The power consumption of CPU-intensive workloads does not scale uniformly with increased load, nor do hardware or software configuration changes affect it in a uniform manner.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Block, Hansfried and Beckett, John and Lange, Klaus-Dieter and Arnold, Jeremy A. and Kounev, Samuel}, booktitle = {Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015)}, keywords = {Power}, month = {02}, note = {acceptance rate: 27\%}, publisher = {ACM}, series = {ICPE '15}, title = {{Analysis of the Influences on Server Power Consumption and Energy Efficiency for CPU-Intensive Workloads}}, year = 2015 }

  %0 Conference Paper %1 KiBlBeLaArKo2015-ICPE-SERT %A von Kistowski, Jóakim %A Block, Hansfried %A Beckett, John %A Lange, Klaus-Dieter %A Arnold, Jeremy A. %A Kounev, Samuel %B Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015) %C New York, NY, USA %D 2015 %I ACM %T Analysis of the Influences on Server Power Consumption and Energy Efficiency for CPU-Intensive Workloads %X Energy efficiency of servers has become a significant research topic over the last years, as server energy consumption varies depending on multiple factors, such as server utilization and workload type. Server energy analysis and estimation must take all relevant factors into account to ensure reliable estimates and conclusions. Thorough system analysis requires benchmarks capable of testing different system resources at different load levels using multiple workload types. Server energy estimation approaches, on the other hand, require knowledge about the interactions of these factors for the creation of accurate power models. Common approaches to energy-aware workload classification classify workloads depending on the resource types used by the different workloads. However, they rarely take into account differences in workloads targeting the same resources. Industrial energy-efficiency benchmarks typically do not evaluate the system's energy consumption at different resource load levels, and they only provide data for system analysis at maximum system load. In this paper, we benchmark multiple server configurations using the CPU worklets included in SPEC's Server Efficiency Rating Tool (SERT). We evaluate the impact of load levels and different CPU workloads on power consumption and energy efficiency. We analyze how functions approximating the measured power consumption differ over multiple server configurations and architectures. We show that workloads targeting the same resource can differ significantly in their power draw and energy efficiency. The power consumption of a given workload type varies depending on utilization, hardware and software configuration. The power consumption of CPU-intensive workloads does not scale uniformly with increased load, nor do hardware or software configuration changes affect it in a uniform manner.
• 
  Parallel Simulation of Queueing Petri Nets. Walter, Jürgen; Spinner, Simon; Kounev, Samuel; in Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

  @inproceedings{WaSpKo-SIMUTools-QPNParallelSimulation, abstract = {Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.}, author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel}, booktitle = {Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015)}, keywords = {QPN}, month = {08}, title = {{Parallel Simulation of Queueing Petri Nets}}, year = 2015 }

  %0 Conference Paper %1 WaSpKo-SIMUTools-QPNParallelSimulation %A Walter, Jürgen %A Spinner, Simon %A Kounev, Samuel %B Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015) %D 2015 %T Parallel Simulation of Queueing Petri Nets %X Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.
• 
  Flexible Performance Prediction of Data Center Networks using Automatically Generated Simulation Models. Rygielski, Piotr; Kounev, Samuel; Tran-Gia, Phuoc; in Proceedings of the Eighth International Conference on Simulation Tools and Techniques (SIMUTools 2015) (2015). 119–128.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Using different modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. In this paper, we propose a generic approach to modeling the performance of data center networks. The approach offers multiple performance models but requires to use only a single modeling language. We propose a two-step modeling methodology, in which a high-level descriptive model of the network is built in the first step, and in the second step model-to-model transformations are used to automatically transform the descriptive model to different network simulation models. We automatically generate three performance models defined at different levels of abstraction to analyze network throughput. By offering multiple simulation models in parallel, we provide flexibility in trading-off between the modeling accuracy and the simulation overhead. We analyze the simulation models by comparing the prediction accuracy with respect to the simulation duration. We observe, that in the investigated scenarios the solution duration of coarser simulation models is up to 300 times shorter, whereas the average prediction accuracy decreases only by 4 percent.

  @inproceedings{RyKoTr2015-SIMUTools-FlexPerformancePrediction, abstract = {{Using different modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. In this paper, we propose a generic approach to modeling the performance of data center networks. The approach offers multiple performance models but requires to use only a single modeling language. We propose a two-step modeling methodology, in which a high-level descriptive model of the network is built in the first step, and in the second step model-to-model transformations are used to automatically transform the descriptive model to different network simulation models. We automatically generate three performance models defined at different levels of abstraction to analyze network throughput. By offering multiple simulation models in parallel, we provide flexibility in trading-off between the modeling accuracy and the simulation overhead. We analyze the simulation models by comparing the prediction accuracy with respect to the simulation duration. We observe, that in the investigated scenarios the solution duration of coarser simulation models is up to 300 times shorter, whereas the average prediction accuracy decreases only by 4 percent.}}, author = {Rygielski, Piotr and Kounev, Samuel and Tran-Gia, Phuoc}, booktitle = {Proceedings of the Eighth International Conference on Simulation Tools and Techniques (SIMUTools 2015)}, keywords = {QPN}, month = {08}, pages = {119–128}, title = {{Flexible Performance Prediction of Data Center Networks using Automatically Generated Simulation Models}}, year = 2015 }

  %0 Conference Paper %1 RyKoTr2015-SIMUTools-FlexPerformancePrediction %A Rygielski, Piotr %A Kounev, Samuel %A Tran-Gia, Phuoc %B Proceedings of the Eighth International Conference on Simulation Tools and Techniques (SIMUTools 2015) %D 2015 %P 119--128 %T Flexible Performance Prediction of Data Center Networks using Automatically Generated Simulation Models %X Using different modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. In this paper, we propose a generic approach to modeling the performance of data center networks. The approach offers multiple performance models but requires to use only a single modeling language. We propose a two-step modeling methodology, in which a high-level descriptive model of the network is built in the first step, and in the second step model-to-model transformations are used to automatically transform the descriptive model to different network simulation models. We automatically generate three performance models defined at different levels of abstraction to analyze network throughput. By offering multiple simulation models in parallel, we provide flexibility in trading-off between the modeling accuracy and the simulation overhead. We analyze the simulation models by comparing the prediction accuracy with respect to the simulation duration. We observe, that in the investigated scenarios the solution duration of coarser simulation models is up to 300 times shorter, whereas the average prediction accuracy decreases only by 4 percent.
• 
  Specialized Heuristics for the Controller Placement Problem in Large Scale SDN Networks. Lange, Stanislav; Gebert, Steffen; Spoerhase, Joachim; Rygielski, Piotr; Zinner, Thomas; Kounev, Samuel; Tran-Gia, Phuoc; in 27th International Teletraffic Congress (ITC 2015}) (2015). 210–218. Ghent, Belgium.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  The Software Defined Networking~(SDN) concept introduces a paradigm shift in the networking world towards an externalized control plane which is logically centralized. When designing an SDN-based WAN architecture, it is of vital importance to find a feasible solution to the controller placement problem, i.e., to decide where to position a limited amount of resources within the network. In addition to time-independent constraints regarding aspects like scalability, resilience, and control plane communication delays, dynamically changing network conditions like traffic patterns or bandwidth demands need to be considered as well. Consequently, such dynamic environments call for a regular and fast recalculation of placements in order to adapt to the current situation in a timely manner. While an exhaustive evaluation of all possible solutions can be performed within a practically feasible time frame for small and medium-sized networks, such an approach is out of scope for large problem instances which have significantly higher time and memory requirements. Therefore, this work investigates a specialized heuristic, which takes into account a particular set of optimization objectives and returns solutions representing the possible trade-offs between them. Due to its low computation time and acceptable margin of error, this heuristic can be employed by automatic decision systems operating in dynamic environments.

  @inproceedings{LaGeSpRyZiKoTG-SDN-Controller-Placement-ITC2015, abstract = {{The Software Defined Networking (SDN) concept introduces a paradigm shift in the networking world towards an externalized control plane which is logically centralized. When designing an SDN-based WAN architecture, it is of vital importance to find a feasible solution to the controller placement problem, i.e., to decide where to position a limited amount of resources within the network. In addition to time-independent constraints regarding aspects like scalability, resilience, and control plane communication delays, dynamically changing network conditions like traffic patterns or bandwidth demands need to be considered as well. Consequently, such dynamic environments call for a regular and fast recalculation of placements in order to adapt to the current situation in a timely manner. While an exhaustive evaluation of all possible solutions can be performed within a practically feasible time frame for small and medium-sized networks, such an approach is out of scope for large problem instances which have significantly higher time and memory requirements. Therefore, this work investigates a specialized heuristic, which takes into account a particular set of optimization objectives and returns solutions representing the possible trade-offs between them. Due to its low computation time and acceptable margin of error, this heuristic can be employed by automatic decision systems operating in dynamic environments.}}, address = {Ghent, Belgium}, author = {Lange, Stanislav and Gebert, Steffen and Spoerhase, Joachim and Rygielski, Piotr and Zinner, Thomas and Kounev, Samuel and Tran-Gia, Phuoc}, booktitle = {27th International Teletraffic Congress ({ITC 2015})}, keywords = {Optimization}, month = {09}, pages = {210–218}, title = {{Specialized Heuristics for the Controller Placement Problem in Large Scale SDN Networks}}, year = 2015 }

  %0 Conference Paper %1 LaGeSpRyZiKoTG-SDN-Controller-Placement-ITC2015 %A Lange, Stanislav %A Gebert, Steffen %A Spoerhase, Joachim %A Rygielski, Piotr %A Zinner, Thomas %A Kounev, Samuel %A Tran-Gia, Phuoc %B 27th International Teletraffic Congress (ITC 2015}) %C Ghent, Belgium %D 2015 %P 210--218 %T Specialized Heuristics for the Controller Placement Problem in Large Scale SDN Networks %U http://dx.doi.org/10.1109/ITC.2015.32 %X The Software Defined Networking~(SDN) concept introduces a paradigm shift in the networking world towards an externalized control plane which is logically centralized. When designing an SDN-based WAN architecture, it is of vital importance to find a feasible solution to the controller placement problem, i.e., to decide where to position a limited amount of resources within the network. In addition to time-independent constraints regarding aspects like scalability, resilience, and control plane communication delays, dynamically changing network conditions like traffic patterns or bandwidth demands need to be considered as well. Consequently, such dynamic environments call for a regular and fast recalculation of placements in order to adapt to the current situation in a timely manner. While an exhaustive evaluation of all possible solutions can be performed within a practically feasible time frame for small and medium-sized networks, such an approach is out of scope for large problem instances which have significantly higher time and memory requirements. Therefore, this work investigates a specialized heuristic, which takes into account a particular set of optimization objectives and returns solutions representing the possible trade-offs between them. Due to its low computation time and acceptable margin of error, this heuristic can be employed by automatic decision systems operating in dynamic environments.
• 
  Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study. Huber, Nikolaus; Walter, Jürgen; Bähr, Manuel; Kounev, Samuel; in Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC) (2015). IEEE.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.

  @inproceedings{HuWaBaKo2015-ICCAC-BlueYonder, abstract = {Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.}, author = {Huber, Nikolaus and Walter, J{\"u}rgen and B{\"a}hr, Manuel and Kounev, Samuel}, booktitle = {Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC)}, keywords = {Self-adaptive-systems}, month = {09}, publisher = {IEEE}, title = {{Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study}}, year = 2015 }

  %0 Conference Paper %1 HuWaBaKo2015-ICCAC-BlueYonder %A Huber, Nikolaus %A Walter, Jürgen %A Bähr, Manuel %A Kounev, Samuel %B Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC) %D 2015 %I IEEE %T Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study %X Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.
• 
  Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments. Busch, Axel; Noorshams, Qais; Kounev, Samuel; Koziolek, Anne; Reussner, Ralf; Amrehn, Erich; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2015). ACM, New York, NY, USA.

  Acceptance Rate (Full Paper): 15/56 = 27%.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{BuNoKoKoReAm2015-ICPE-WorkloadCharacterization, address = {New York, NY, USA}, author = {Busch, Axel and Noorshams, Qais and Kounev, Samuel and Koziolek, Anne and Reussner, Ralf and Amrehn, Erich}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {se2}, month = {02}, note = {Acceptance Rate (Full Paper): 15/56 = 27\%.}, publisher = {ACM}, series = {ICPE '15}, title = {{Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments}}, year = 2015 }

  %0 Conference Paper %1 BuNoKoKoReAm2015-ICPE-WorkloadCharacterization %A Busch, Axel %A Noorshams, Qais %A Kounev, Samuel %A Koziolek, Anne %A Reussner, Ralf %A Amrehn, Erich %B Proceedings of the ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2015 %I ACM %T Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments %U http://dx.doi.org/10.1145/2668930.2688050

• 
  Performance Queries for Architecture-Level Performance Models. Gorsler, Fabian; Brosig, Fabian; Kounev, Samuel; in Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). ACM, New York, NY, USA.

  Acceptance Rate (Full Paper): 14/56 = 25%.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{GoBrKo2014-PerformanceQueries, address = {New York, NY, USA}, author = {Gorsler, Fabian and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {DQL}, month = {{March}}, note = {Acceptance Rate (Full Paper): 14/56 = 25\%.}, publisher = {ACM}, title = {{Performance Queries for Architecture-Level Performance Models}}, year = 2014 }

  %0 Conference Paper %1 GoBrKo2014-PerformanceQueries %A Gorsler, Fabian %A Brosig, Fabian %A Kounev, Samuel %B Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) %C New York, NY, USA %D 2014 %I ACM %T Performance Queries for Architecture-Level Performance Models
• 
  Experience Report: An Analysis of Hypercall Handler Vulnerabilities. Milenkoski, Aleksandar; Payne, Bryan D.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; in Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) --- Research Track (2014). IEEE Computer Society, Washington DC, USA.

  Acceptance Rate: 25%, Best Paper Award Nomination
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.

  @inproceedings{MiPaAnViKo2014-ISSRE-AnAnalHypHanVulns, abstract = {{Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.}}, address = {{Washington DC, USA}}, author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel}, booktitle = {{Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) — Research Track}}, keywords = {Reliability}, month = 11, note = {Acceptance Rate: 25\%, <b>Best Paper Award Nomination</b>}, organization = {IEEE}, publisher = {IEEE Computer Society}, title = {{Experience Report: An Analysis of Hypercall Handler Vulnerabilities}}, year = 2014 }

  %0 Conference Paper %1 MiPaAnViKo2014-ISSRE-AnAnalHypHanVulns %A Milenkoski, Aleksandar %A Payne, Bryan D. %A Antunes, Nuno %A Vieira, Marco %A Kounev, Samuel %B Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) --- Research Track %C Washington DC, USA %D 2014 %I IEEE Computer Society %T Experience Report: An Analysis of Hypercall Handler Vulnerabilities %U http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6982618 %X Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.
• 
  Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation. Spinner, Simon; Kounev, Samuel; Zhu, Xiaoyun; Lu, Lei; Uysal, Mustafa; Holler, Anne; Griffith, Rean; in Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO) (2014). 157–166. IEEE.

  Acceptance Rate (Full Papers): 26%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.

  @inproceedings{SpKoZhLuUyHoGr2014-SASO, abstract = {Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.}, author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Holler, Anne and Griffith, Rean}, booktitle = {Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO)}, keywords = {Self-adaptive-systems}, month = {09}, note = {Acceptance Rate (Full Papers): 26%}, pages = {157–166}, publisher = {IEEE}, title = {{Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation}}, year = 2014 }

  %0 Conference Paper %1 SpKoZhLuUyHoGr2014-SASO %A Spinner, Simon %A Kounev, Samuel %A Zhu, Xiaoyun %A Lu, Lei %A Uysal, Mustafa %A Holler, Anne %A Griffith, Rean %B Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO) %D 2014 %I IEEE %P 157--166 %T Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation %X Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.
• 
  Platform-as-a-Service Architecture for Performance Isolated Multi-Tenant Applications. Krebs, Rouven; Loesch, Manuel; Kounev, Samuel; in Proceedings of the 7th IEEE International Conference on Cloud Computing (IEEE Cloud 2014) (2014). IEEE.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. This is a significant problem as performance is one of the major obstacles for cloud customers. The application does intentionally not manage hardware resources, and the operating system is not aware of application level entities like tenants which makes the performance control a challenge. In case the SaaS is hosted on a Platform-as-a-Service (PaaS), the SaaS developer usually wants to control performance-related issues according to individual needs, and available information is even more limited. Thus, it is difficult to control the performance of different tenants to keep them isolated. Existing work focuses on concrete methods to provide performance isolation in systems where the whole stack is under control. In this paper we present a concrete PaaS enhancement which enables application developers to realize isolation methods for their hosted SaaS application. In a case study we evaluated the applicability and effectiveness of the enhancement in different environments.

  @inproceedings{KrLoKo2014_CLOUD_Framework, abstract = {{Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. This is a significant problem as performance is one of the major obstacles for cloud customers. The application does intentionally not manage hardware resources, and the operating system is not aware of application level entities like tenants which makes the performance control a challenge. In case the SaaS is hosted on a Platform-as-a-Service (PaaS), the SaaS developer usually wants to control performance-related issues according to individual needs, and available information is even more limited. Thus, it is difficult to control the performance of different tenants to keep them isolated. Existing work focuses on concrete methods to provide performance isolation in systems where the whole stack is under control. In this paper we present a concrete PaaS enhancement which enables application developers to realize isolation methods for their hosted SaaS application. In a case study we evaluated the applicability and effectiveness of the enhancement in different environments.}}, author = {Krebs, Rouven and Loesch, Manuel and Kounev, Samuel}, booktitle = {Proceedings of the 7th IEEE International Conference on Cloud Computing (IEEE Cloud 2014)}, keywords = {Application_quality_of_service_management}, month = {07}, publisher = {IEEE}, title = {{Platform-as-a-Service Architecture for Performance Isolated Multi-Tenant Applications}}, year = 2014 }

  %0 Conference Paper %1 KrLoKo2014_CLOUD_Framework %A Krebs, Rouven %A Loesch, Manuel %A Kounev, Samuel %B Proceedings of the 7th IEEE International Conference on Cloud Computing (IEEE Cloud 2014) %D 2014 %I IEEE %T Platform-as-a-Service Architecture for Performance Isolated Multi-Tenant Applications %X Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. This is a significant problem as performance is one of the major obstacles for cloud customers. The application does intentionally not manage hardware resources, and the operating system is not aware of application level entities like tenants which makes the performance control a challenge. In case the SaaS is hosted on a Platform-as-a-Service (PaaS), the SaaS developer usually wants to control performance-related issues according to individual needs, and available information is even more limited. Thus, it is difficult to control the performance of different tenants to keep them isolated. Existing work focuses on concrete methods to provide performance isolation in systems where the whole stack is under control. In this paper we present a concrete PaaS enhancement which enables application developers to realize isolation methods for their hosted SaaS application. In a case study we evaluated the applicability and effectiveness of the enhancement in different environments.
• 
  Enriching Software Architecture Models with Statistical Models for Performance Prediction in Modern Storage Environments. Noorshams, Qais; Reeb, Roland; Rentschler, Andreas; Kounev, Samuel; Reussner, Ralf; in Proceedings of the 17th International ACM Sigsoft Symposium on Component-based Software Engineering (2014). 45–54. ACM, New York, NY, USA.

  Acceptance Rate (Full Paper): 14/62 = 23%.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoReReKoRe2014-CBSE-ArchitectureModelsWithStatisticalModels, address = {New York, NY, USA}, author = {Noorshams, Qais and Reeb, Roland and Rentschler, Andreas and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the 17th International ACM Sigsoft Symposium on Component-based Software Engineering}, keywords = {Simulation}, month = {06}, note = {Acceptance Rate (Full Paper): 14/62 = 23\%.}, pages = {45–54}, publisher = {ACM}, series = {CBSE '14}, title = {Enriching Software Architecture Models with Statistical Models for Performance Prediction in Modern Storage Environments}, year = 2014 }

  %0 Conference Paper %1 NoReReKoRe2014-CBSE-ArchitectureModelsWithStatisticalModels %A Noorshams, Qais %A Reeb, Roland %A Rentschler, Andreas %A Kounev, Samuel %A Reussner, Ralf %B Proceedings of the 17th International ACM Sigsoft Symposium on Component-based Software Engineering %C New York, NY, USA %D 2014 %I ACM %P 45--54 %T Enriching Software Architecture Models with Statistical Models for Performance Prediction in Modern Storage Environments %U http://doi.acm.org/10.1145/2602458.2602475
• 
  Resource Usage Control In Multi-Tenant Applications. Krebs, Rouven; Spinner, Simon; Ahmed, Nadia; Kounev, Samuel; in Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014) (2014). 122–131. IEEE/ACM.

  Acceptance Rate: 19% (54/283)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.

  @inproceedings{KrSpAhKo2014_CCGrid_ResourceIsolation, abstract = {{Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.}}, author = {Krebs, Rouven and Spinner, Simon and Ahmed, Nadia and Kounev, Samuel}, booktitle = {Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014)}, keywords = {Self-adaptive-systems}, month = {05}, note = {Acceptance Rate: 19\% (54/283)}, pages = {122-131}, publisher = {IEEE/ACM}, title = {{Resource Usage Control In Multi-Tenant Applications}}, year = 2014 }

  %0 Conference Paper %1 KrSpAhKo2014_CCGrid_ResourceIsolation %A Krebs, Rouven %A Spinner, Simon %A Ahmed, Nadia %A Kounev, Samuel %B Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014) %D 2014 %I IEEE/ACM %P 122-131 %T Resource Usage Control In Multi-Tenant Applications %X Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.

• 
  Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning. Herbst, Nikolas Roman; Huber, Nikolaus; Kounev, Samuel; Amrehn, Erich; in Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013) (2013). 187–198. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.

  @inproceedings{HeHuKoAm2013-ICPE-WorkloadClassificationAndForecasting, abstract = {{As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.}}, address = {New York, NY, USA}, author = {Herbst, Nikolas Roman and Huber, Nikolaus and Kounev, Samuel and Amrehn, Erich}, booktitle = {Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013)}, keywords = {Self-adaptive-systems}, month = {04}, pages = {187–198}, publisher = {ACM}, title = {{Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning}}, year = 2013 }

  %0 Conference Paper %1 HeHuKoAm2013-ICPE-WorkloadClassificationAndForecasting %A Herbst, Nikolas Roman %A Huber, Nikolaus %A Kounev, Samuel %A Amrehn, Erich %B Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013) %C New York, NY, USA %D 2013 %I ACM %P 187--198 %T Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning %U http://doi.acm.org/10.1145/2479871.2479899 %X As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.
• 
  I/O Performance Modeling of Virtualized Storage Systems. Noorshams, Qais; Rostami, Kiana; Kounev, Samuel; Tuma, Petr; Reussner, Ralf; in Proceedings of the 21st IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013) (2013). 121–130.

  Acceptance Rate (Full Paper): 44/163 = 27%
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoRoKoTuRe2013-MASCOTS-IOQNs, author = {Noorshams, Qais and Rostami, Kiana and Kounev, Samuel and T{u}ma, Petr and Reussner, Ralf}, booktitle = {Proceedings of the 21st IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013)}, keywords = {Instrumentation_profiling_and_workload_characterization}, note = {Acceptance Rate (Full Paper): 44/163 = 27\%}, pages = {121-130}, series = {MASCOTS'13}, title = {{I/O Performance Modeling of Virtualized Storage Systems}}, year = 2013 }

  %0 Conference Paper %1 NoRoKoTuRe2013-MASCOTS-IOQNs %A Noorshams, Qais %A Rostami, Kiana %A Kounev, Samuel %A Tuma, Petr %A Reussner, Ralf %B Proceedings of the 21st IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013) %D 2013 %P 121-130 %T I/O Performance Modeling of Virtualized Storage Systems %U http://dx.doi.org/10.1109/MASCOTS.2013.20
• 
  Predictive Performance Modeling of Virtualized Storage Systems using Optimized Statistical Regression Techniques. Noorshams, Qais; Bruhn, Dominik; Kounev, Samuel; Reussner, Ralf; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE 2013) (2013). 283–294. ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoBrKoRe2013-ICPE-IORegressionModels, address = {New York, NY, USA}, author = {Noorshams, Qais and Bruhn, Dominik and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE 2013)}, keywords = {Virtualization}, pages = {283–294}, publisher = {ACM}, series = {ICPE'13}, title = {{Predictive Performance Modeling of Virtualized Storage Systems using Optimized Statistical Regression Techniques}}, year = 2013 }

  %0 Conference Paper %1 NoBrKoRe2013-ICPE-IORegressionModels %A Noorshams, Qais %A Bruhn, Dominik %A Kounev, Samuel %A Reussner, Ralf %B Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE 2013) %C New York, NY, USA %D 2013 %I ACM %P 283--294 %T Predictive Performance Modeling of Virtualized Storage Systems using Optimized Statistical Regression Techniques %U http://doi.acm.org/10.1145/2479871.2479910

• 
  Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments. Krebs, Rouven; Momm, Christof; Kounev, Samuel; in Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012), B. Buhnova, A. Vallecillo (eds.) (2012). 91–100. ACM Press, New York, USA.

  Acceptance Rate (Full Paper): 25.6%
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{KrMoKo2012-QoSA-QuantifyingPerfIsoMetrics, address = {New York, USA}, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012)}, editor = {Buhnova, Barbora and Vallecillo, Antonio}, keywords = {Application_quality_of_service_management}, month = {06}, note = {Acceptance Rate (Full Paper): 25.6\%}, pages = {91–100}, publisher = {ACM Press}, title = {{M}etrics and {T}echniques for {Q}uantifying {P}erformance {I}solation in {C}loud {E}nvironments}, year = 2012 }

  %0 Conference Paper %1 KrMoKo2012-QoSA-QuantifyingPerfIsoMetrics %A Krebs, Rouven %A Momm, Christof %A Kounev, Samuel %B Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012) %C New York, USA %D 2012 %E Buhnova, Barbora %E Vallecillo, Antonio %I ACM Press %P 91--100 %T Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments %U http://qosa.ipd.kit.edu/qosa_2012/
• 
  S/T/A: Meta-Modeling Run-Time Adaptation in Component-Based System Architectures. Huber, Nikolaus; van Hoorn, André; Koziolek, Anne; Brosig, Fabian; Kounev, Samuel; in Proceedings of the 9th IEEE International Conference on e-Business Engineering (ICEBE 2012) (2012). 70–77. IEEE Computer Society, Los Alamitos, CA, USA.

  Acceptance Rate (Full Paper): 19.7% (26/132)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Modern virtualized system environments usually host diverse applications of different parties and aim at utilizing resources efficiently while ensuring that quality-of-service requirements are continuously satisfied. In such scenarios, complex adaptations to changes in the system environment are still largely performed manually by humans. Over the past decade, autonomic self-adaptation techniques aiming to minimize human intervention have become increasingly popular. However, given that adaptation processes are usually highly system specific, it is a challenge to abstract from system details enabling the reuse of adaptation strategies. In this paper, we propose a novel modeling language (meta-model) providing means to describe system adaptation processes at the system architecture level in a generic, human-understandable and reusable way. We apply our approach to three different realistic contexts (dynamic resource allocation, software architecture optimization, and run-time adaptation planning) showing how the gap between complex manual adaptations and their autonomous execution can be closed by using a holistic model-based approach.

  @inproceedings{HuHoKoBrKo2012-ICEBE-STA, abstract = {Modern virtualized system environments usually host diverse applications of different parties and aim at utilizing resources efficiently while ensuring that quality-of-service requirements are continuously satisfied. In such scenarios, complex adaptations to changes in the system environment are still largely performed manually by humans. Over the past decade, autonomic self-adaptation techniques aiming to minimize human intervention have become increasingly popular. However, given that adaptation processes are usually highly system specific, it is a challenge to abstract from system details enabling the reuse of adaptation strategies. In this paper, we propose a novel modeling language (meta-model) providing means to describe system adaptation processes at the system architecture level in a generic, human-understandable and reusable way. We apply our approach to three different realistic contexts (dynamic resource allocation, software architecture optimization, and run-time adaptation planning) showing how the gap between complex manual adaptations and their autonomous execution can be closed by using a holistic model-based approach.}, address = {Los Alamitos, CA, USA}, author = {Huber, Nikolaus and van Hoorn, Andr{\'e} and Koziolek, Anne and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the 9th IEEE International Conference on e-Business Engineering (ICEBE 2012)}, keywords = {Self-adaptive-systems}, month = {09}, note = {Acceptance Rate (Full Paper): 19.7\% (26/132)}, pages = {70–77}, publisher = {IEEE Computer Society}, title = {{S/T/A: Meta-Modeling Run-Time Adaptation in Component-Based System Architectures}}, year = 2012 }

  %0 Conference Paper %1 HuHoKoBrKo2012-ICEBE-STA %A Huber, Nikolaus %A van Hoorn, André %A Koziolek, Anne %A Brosig, Fabian %A Kounev, Samuel %B Proceedings of the 9th IEEE International Conference on e-Business Engineering (ICEBE 2012) %C Los Alamitos, CA, USA %D 2012 %I IEEE Computer Society %P 70--77 %T S/T/A: Meta-Modeling Run-Time Adaptation in Component-Based System Architectures %U http://conferences.computer.org/icebe/2012/index.htm %X Modern virtualized system environments usually host diverse applications of different parties and aim at utilizing resources efficiently while ensuring that quality-of-service requirements are continuously satisfied. In such scenarios, complex adaptations to changes in the system environment are still largely performed manually by humans. Over the past decade, autonomic self-adaptation techniques aiming to minimize human intervention have become increasingly popular. However, given that adaptation processes are usually highly system specific, it is a challenge to abstract from system details enabling the reuse of adaptation strategies. In this paper, we propose a novel modeling language (meta-model) providing means to describe system adaptation processes at the system architecture level in a generic, human-understandable and reusable way. We apply our approach to three different realistic contexts (dynamic resource allocation, software architecture optimization, and run-time adaptation planning) showing how the gap between complex manual adaptations and their autonomous execution can be closed by using a holistic model-based approach.
• 
  Modeling Dynamic Virtualized Resource Landscapes. Huber, Nikolaus; Brosig, Fabian; Kounev, Samuel; in Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012) (2012). 81–90. ACM, New York, NY, USA.

  Acceptance Rate (Full Paper): 25.6%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Modern data centers are subject to an increasing demand for flexibility. Increased flexibility and dynamics, however, also result in a higher system complexity. This complexity carries on to run-time resource management for Quality-of-Service (QoS) enforcement, rendering design-time approaches for QoS assurance inadequate. In this paper, we present a set of novel meta-models that can be used to describe the resource landscape, the architecture and resource layers of dynamic virtualized data center infrastructures, as well as their run-time adaptation and resource management aspects. With these meta-models we introduce new modeling concepts to improve model-based run-time QoS assurance. We evaluate our meta-models by modeling a representative virtualized service infrastructure and using these model instances for run-time resource allocation. The results demonstrate the benefits of the new meta-models and show how they can be used to improve model-based system adaptation and run-time resource management in dynamic virtualized data centers.

  @inproceedings{HuBrKo2012-QoSA-ModelingVirtResLandscapes, abstract = {Modern data centers are subject to an increasing demand for flexibility. Increased flexibility and dynamics, however, also result in a higher system complexity. This complexity carries on to run-time resource management for Quality-of-Service (QoS) enforcement, rendering design-time approaches for QoS assurance inadequate. In this paper, we present a set of novel meta-models that can be used to describe the resource landscape, the architecture and resource layers of dynamic virtualized data center infrastructures, as well as their run-time adaptation and resource management aspects. With these meta-models we introduce new modeling concepts to improve model-based run-time QoS assurance. We evaluate our meta-models by modeling a representative virtualized service infrastructure and using these model instances for run-time resource allocation. The results demonstrate the benefits of the new meta-models and show how they can be used to improve model-based system adaptation and run-time resource management in dynamic virtualized data centers.}, address = {New York, NY, USA}, author = {Huber, Nikolaus and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012)}, keywords = {Self-adaptive-systems}, month = {06}, note = {Acceptance Rate (Full Paper): 25.6\%}, pages = {81–90}, publisher = {ACM}, title = {{Modeling Dynamic Virtualized Resource Landscapes}}, year = 2012 }

  %0 Conference Paper %1 HuBrKo2012-QoSA-ModelingVirtResLandscapes %A Huber, Nikolaus %A Brosig, Fabian %A Kounev, Samuel %B Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012) %C New York, NY, USA %D 2012 %I ACM %P 81--90 %T Modeling Dynamic Virtualized Resource Landscapes %U http://qosa.ipd.kit.edu/qosa_2012/ %X Modern data centers are subject to an increasing demand for flexibility. Increased flexibility and dynamics, however, also result in a higher system complexity. This complexity carries on to run-time resource management for Quality-of-Service (QoS) enforcement, rendering design-time approaches for QoS assurance inadequate. In this paper, we present a set of novel meta-models that can be used to describe the resource landscape, the architecture and resource layers of dynamic virtualized data center infrastructures, as well as their run-time adaptation and resource management aspects. With these meta-models we introduce new modeling concepts to improve model-based run-time QoS assurance. We evaluate our meta-models by modeling a representative virtualized service infrastructure and using these model instances for run-time resource allocation. The results demonstrate the benefits of the new meta-models and show how they can be used to improve model-based system adaptation and run-time resource management in dynamic virtualized data centers.
• 
  Modeling Parameter and Context Dependencies in Online Architecture-Level Performance Models. Brosig, Fabian; Huber, Nikolaus; Kounev, Samuel; in Proceedings of the 15th ACM SIGSOFT International Symposium on Component Based Software Engineering (CBSE 2012), June 26--28, 2012, Bertinoro, Italy (2012).

  Acceptance Rate (Full Paper): 28.5%.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Modern enterprise applications have to satisfy increasingly stringent Quality-of-Service requirements. To ensure that a system meets its performance requirements, the ability to predict its performance under different configurations and workloads is essential. Architecture-level performance models describe performance-relevant aspects of software architectures and execution environments allowing to evaluate different usage profiles as well as system deployment and configuration options. However, building performance models manually requires a lot of time and effort. In this paper, we present a novel automated method for the extraction of architecture-level performance models of distributed component-based systems, based on monitoring data collected at run-time. The method is validated in a case study with the industry-standard SPECjEnterprise2010 Enterprise Java benchmark, a representative software system executed in a realistic environment. The obtained performance predictions match the measurements on the real system within an error margin of mostly 10-20 percent.

  @inproceedings{BrHuKo2012-CBSE-ParamAndContextDep, abstract = {Modern enterprise applications have to satisfy increasingly stringent Quality-of-Service requirements. To ensure that a system meets its performance requirements, the ability to predict its performance under different configurations and workloads is essential. Architecture-level performance models describe performance-relevant aspects of software architectures and execution environments allowing to evaluate different usage profiles as well as system deployment and configuration options. However, building performance models manually requires a lot of time and effort. In this paper, we present a novel automated method for the extraction of architecture-level performance models of distributed component-based systems, based on monitoring data collected at run-time. The method is validated in a case study with the industry-standard SPECjEnterprise2010 Enterprise Java benchmark, a representative software system executed in a realistic environment. The obtained performance predictions match the measurements on the real system within an error margin of mostly 10-20 percent.}, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, booktitle = {Proceedings of the 15th ACM SIGSOFT International Symposium on Component Based Software Engineering (CBSE 2012), June 26–28, 2012, Bertinoro, Italy}, keywords = {Application_quality_of_service_management}, month = {06}, note = {Acceptance Rate (Full Paper): 28.5\%.}, title = {{Modeling Parameter and Context Dependencies in Online Architecture-Level Performance Models}}, year = 2012 }

  %0 Conference Paper %1 BrHuKo2012-CBSE-ParamAndContextDep %A Brosig, Fabian %A Huber, Nikolaus %A Kounev, Samuel %B Proceedings of the 15th ACM SIGSOFT International Symposium on Component Based Software Engineering (CBSE 2012), June 26--28, 2012, Bertinoro, Italy %D 2012 %T Modeling Parameter and Context Dependencies in Online Architecture-Level Performance Models %U http://cbse-conferences.org/2012/ %X Modern enterprise applications have to satisfy increasingly stringent Quality-of-Service requirements. To ensure that a system meets its performance requirements, the ability to predict its performance under different configurations and workloads is essential. Architecture-level performance models describe performance-relevant aspects of software architectures and execution environments allowing to evaluate different usage profiles as well as system deployment and configuration options. However, building performance models manually requires a lot of time and effort. In this paper, we present a novel automated method for the extraction of architecture-level performance models of distributed component-based systems, based on monitoring data collected at run-time. The method is validated in a case study with the industry-standard SPECjEnterprise2010 Enterprise Java benchmark, a representative software system executed in a realistic environment. The obtained performance predictions match the measurements on the real system within an error margin of mostly 10-20 percent.

• 
  Capacity Planning for Event-based Systems using Automated Performance Predictions. Rathfelder, Christoph; Kounev, Samuel; Evans, David; in 26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011) (2011). 352–361. IEEE.

  Acceptance Rate (Full Paper): 14.7% (37/252)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. The loose coupling of components in such systems makes it easy to vary the deployment. At the same time, the complexity to estimate the behavior and performance of the whole system is increased, which complicates capacity planning. In this paper, we present an automated performance prediction method supporting capacity planning for event-based systems. The performance prediction is based on an extended version of the Palladio Component Model -- a performance meta-model for component-based systems. We apply this method on a real-world case study of a traffic monitoring system. In addition to the application of our performance prediction techniques for capacity planning, we evaluate the prediction results against measurements in the context of the case study. The results demonstrate the practicality and effectiveness of the proposed approach.

  @inproceedings{RaKoEv2011-ASE-CapacityPlanning, abstract = {Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. The loose coupling of components in such systems makes it easy to vary the deployment. At the same time, the complexity to estimate the behavior and performance of the whole system is increased, which complicates capacity planning. In this paper, we present an automated performance prediction method supporting capacity planning for event-based systems. The performance prediction is based on an extended version of the Palladio Component Model – a performance meta-model for component-based systems. We apply this method on a real-world case study of a traffic monitoring system. In addition to the application of our performance prediction techniques for capacity planning, we evaluate the prediction results against measurements in the context of the case study. The results demonstrate the practicality and effectiveness of the proposed approach.}, author = {Rathfelder, Christoph and Kounev, Samuel and Evans, David}, booktitle = {26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011)}, keywords = {Application_quality_of_service_management}, month = 11, note = {Acceptance Rate (Full Paper): 14.7\% (37/252)}, pages = {352–361}, publisher = {IEEE}, title = {{C}apacity {P}lanning for {E}vent-based {S}ystems using {A}utomated {P}erformance {P}redictions}, year = 2011 }

  %0 Conference Paper %1 RaKoEv2011-ASE-CapacityPlanning %A Rathfelder, Christoph %A Kounev, Samuel %A Evans, David %B 26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011) %D 2011 %I IEEE %P 352--361 %T Capacity Planning for Event-based Systems using Automated Performance Predictions %X Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. The loose coupling of components in such systems makes it easy to vary the deployment. At the same time, the complexity to estimate the behavior and performance of the whole system is increased, which complicates capacity planning. In this paper, we present an automated performance prediction method supporting capacity planning for event-based systems. The performance prediction is based on an extended version of the Palladio Component Model -- a performance meta-model for component-based systems. We apply this method on a real-world case study of a traffic monitoring system. In addition to the application of our performance prediction techniques for capacity planning, we evaluate the prediction results against measurements in the context of the case study. The results demonstrate the practicality and effectiveness of the proposed approach.
• 
  Automated Extraction of Architecture-Level Performance Models of Distributed Component-Based Systems. Brosig, Fabian; Huber, Nikolaus; Kounev, Samuel; in 26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011) (2011). Oread, Lawrence, Kansas.

  Acceptance Rate (Full Paper): 14.7% (37/252)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Modern service-oriented enterprise systems have increasingly complex and dynamic loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment and adapt the system configuration accordingly. Architecture-level performance models provide a powerful tool for performance prediction, however, current approaches to modeling the execution context of software components are not suitable for use at run-time. In this paper, we analyze the typical online performance prediction scenarios and propose a novel performance meta-model for expressing and resolving parameter and context dependencies, specifically designed for use in online scenarios. We motivate and validate our approach in the context of a realistic and representative online performance prediction scenario based on the SPECjEnterprise2010 standard benchmark.

  @inproceedings{BrHuKo2011-ASE-AutomExtraction, abstract = {Modern service-oriented enterprise systems have increasingly complex and dynamic loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment and adapt the system configuration accordingly. Architecture-level performance models provide a powerful tool for performance prediction, however, current approaches to modeling the execution context of software components are not suitable for use at run-time. In this paper, we analyze the typical online performance prediction scenarios and propose a novel performance meta-model for expressing and resolving parameter and context dependencies, specifically designed for use in online scenarios. We motivate and validate our approach in the context of a realistic and representative online performance prediction scenario based on the SPECjEnterprise2010 standard benchmark.}, address = {Oread, Lawrence, Kansas}, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, booktitle = {26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011)}, keywords = {Application_quality_of_service_management}, month = 11, note = {Acceptance Rate (Full Paper): 14.7\% (37/252)}, title = {{A}utomated {E}xtraction of {A}rchitecture-{L}evel {P}erformance {M}odels of {D}istributed {C}omponent-{B}ased {S}ystems}, year = 2011 }

  %0 Conference Paper %1 BrHuKo2011-ASE-AutomExtraction %A Brosig, Fabian %A Huber, Nikolaus %A Kounev, Samuel %B 26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011) %C Oread, Lawrence, Kansas %D 2011 %T Automated Extraction of Architecture-Level Performance Models of Distributed Component-Based Systems %X Modern service-oriented enterprise systems have increasingly complex and dynamic loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment and adapt the system configuration accordingly. Architecture-level performance models provide a powerful tool for performance prediction, however, current approaches to modeling the execution context of software components are not suitable for use at run-time. In this paper, we analyze the typical online performance prediction scenarios and propose a novel performance meta-model for expressing and resolving parameter and context dependencies, specifically designed for use in online scenarios. We motivate and validate our approach in the context of a realistic and representative online performance prediction scenario based on the SPECjEnterprise2010 standard benchmark.
• 
  Performance Engineering of Business Information Systems - Filling the Gap between High-level Business Services and Low-level Performance Models. Kounev, Samuel; in International Symposium on Business Modeling and Software Design (BMSD 2011), Sofia, Bulgaria, July 27--28, 2011 (2011).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{Ko2011-BMSD-PerfEngOfBis, author = {Kounev, Samuel}, booktitle = {International Symposium on Business Modeling and Software Design (BMSD 2011), Sofia, Bulgaria, July 27–28, 2011}, keywords = {Application_quality_of_service_management}, month = {07}, title = {{Performance Engineering of Business Information Systems - Filling the Gap between High-level Business Services and Low-level Performance Models}}, year = 2011 }

  %0 Conference Paper %1 Ko2011-BMSD-PerfEngOfBis %A Kounev, Samuel %B International Symposium on Business Modeling and Software Design (BMSD 2011), Sofia, Bulgaria, July 27--28, 2011 %D 2011 %T Performance Engineering of Business Information Systems - Filling the Gap between High-level Business Services and Low-level Performance Models
• 
  Integration of event-based communication in the palladio software quality prediction framework. Klatt, Benjamin; Rathfelder, Christoph; Kounev, Samuel; in Proceedings of the joint ACM SIGSOFT conference -- QoSA and ACM SIGSOFT symposium -- ISARCS on Quality of software architectures -- QoSA and architecting critical systems -- ISARCS (QoSA-ISARCS 2011) (2011). 43–52. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Today, software engineering is challenged to handle more and more large-scale distributed systems with guaranteed quality-of-service. Component-based architectures have been established to build such systems in a more structured and manageable way. Modern architectures often utilize event-based communication which enables loosely-coupled interactions between components and leads to improved system scalability. However, the loose coupling of components makes it challenging to model such architectures in order to predict their quality properties, e.g., performance and reliability, at system design time. In this paper, we present an extension of the Palladio Component Model (PCM) and the Palladio software quality prediction framework, enabling the modeling of event-based communication in component-based architectures. The contributions include: i) a meta-model extension supporting events as first class entities, ii) a model-to-model transformation from the extended to the original PCM, iii) an integration of the transformation into the Palladio tool chain allowing to use existing model solution techniques, and iv) a detailed evaluation of the reduction of the modeling effort enabled by the transformation in the context of a real-world case study.

  @inproceedings{KlRaKo2011-QoSA-PCMEvents, abstract = {Today, software engineering is challenged to handle more and more large-scale distributed systems with guaranteed quality-of-service. Component-based architectures have been established to build such systems in a more structured and manageable way. Modern architectures often utilize event-based communication which enables loosely-coupled interactions between components and leads to improved system scalability. However, the loose coupling of components makes it challenging to model such architectures in order to predict their quality properties, e.g., performance and reliability, at system design time. In this paper, we present an extension of the Palladio Component Model (PCM) and the Palladio software quality prediction framework, enabling the modeling of event-based communication in component-based architectures. The contributions include: i) a meta-model extension supporting events as first class entities, ii) a model-to-model transformation from the extended to the original PCM, iii) an integration of the transformation into the Palladio tool chain allowing to use existing model solution techniques, and iv) a detailed evaluation of the reduction of the modeling effort enabled by the transformation in the context of a real-world case study.}, address = {New York, NY, USA}, author = {Klatt, Benjamin and Rathfelder, Christoph and Kounev, Samuel}, booktitle = {Proceedings of the joint ACM SIGSOFT conference – QoSA and ACM SIGSOFT symposium – ISARCS on Quality of software architectures – QoSA and architecting critical systems – ISARCS (QoSA-ISARCS 2011)}, keywords = {Application_quality_of_service_management}, month = {06}, organization = {SIGSOFT}, pages = {43–52}, publisher = {ACM}, title = {Integration of event-based communication in the palladio software quality prediction framework}, year = 2011 }

  %0 Conference Paper %1 KlRaKo2011-QoSA-PCMEvents %A Klatt, Benjamin %A Rathfelder, Christoph %A Kounev, Samuel %B Proceedings of the joint ACM SIGSOFT conference -- QoSA and ACM SIGSOFT symposium -- ISARCS on Quality of software architectures -- QoSA and architecting critical systems -- ISARCS (QoSA-ISARCS 2011) %C New York, NY, USA %D 2011 %I ACM %P 43--52 %T Integration of event-based communication in the palladio software quality prediction framework %U http://doi.acm.org/10.1145/2000259.2000268 %X Today, software engineering is challenged to handle more and more large-scale distributed systems with guaranteed quality-of-service. Component-based architectures have been established to build such systems in a more structured and manageable way. Modern architectures often utilize event-based communication which enables loosely-coupled interactions between components and leads to improved system scalability. However, the loose coupling of components makes it challenging to model such architectures in order to predict their quality properties, e.g., performance and reliability, at system design time. In this paper, we present an extension of the Palladio Component Model (PCM) and the Palladio software quality prediction framework, enabling the modeling of event-based communication in component-based architectures. The contributions include: i) a meta-model extension supporting events as first class entities, ii) a model-to-model transformation from the extended to the original PCM, iii) an integration of the transformation into the Palladio tool chain allowing to use existing model solution techniques, and iv) a detailed evaluation of the reduction of the modeling effort enabled by the transformation in the context of a real-world case study.
• 
  Model-based Self-Adaptive Resource Allocation in Virtualized Environments. Huber, Nikolaus; Brosig, Fabian; Kounev, Samuel; in 6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011) (2011). 90–99. ACM, New York, NY, USA.

  Acceptance Rate (Full Paper): 27% (21/76)
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  The adoption of virtualization and Cloud Computing technologies promises a number of benefits such as increased flexibility, better energy efficiency and lower operating costs for IT systems. However, highly variable workloads make it challenging to provide quality-of-service guarantees while at the same time ensuring efficient resource utilization. To avoid violations of service-level agreements (SLAs) or inefficient resource usage, resource allocations have to be adapted continuously during operation to reflect changes in application workloads. In this paper, we present a novel approach to self-adaptive resource allocation in virtualized environments based on online architecture-level performance models. We present a detailed case study of a representative enterprise application, the new SPECjEnterprise2010 benchmark, deployed in a virtualized cluster environment. The case study serves as a proof-of-concept demonstrating the effectiveness and practical applicability of our approach.

  @inproceedings{HuBrKo2011-SEAMS-ResAlloc, abstract = {The adoption of virtualization and Cloud Computing technologies promises a number of benefits such as increased flexibility, better energy efficiency and lower operating costs for IT systems. However, highly variable workloads make it challenging to provide quality-of-service guarantees while at the same time ensuring efficient resource utilization. To avoid violations of service-level agreements (SLAs) or inefficient resource usage, resource allocations have to be adapted continuously during operation to reflect changes in application workloads. In this paper, we present a novel approach to self-adaptive resource allocation in virtualized environments based on online architecture-level performance models. We present a detailed case study of a representative enterprise application, the new SPECjEnterprise2010 benchmark, deployed in a virtualized cluster environment. The case study serves as a proof-of-concept demonstrating the effectiveness and practical applicability of our approach.}, address = {New York, NY, USA}, author = {Huber, Nikolaus and Brosig, Fabian and Kounev, Samuel}, booktitle = {6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011)}, keywords = {Self-adaptive-systems}, month = {05}, note = {Acceptance Rate (Full Paper): 27\% (21/76)}, pages = {90–99}, publisher = {ACM}, title = {{Model-based Self-Adaptive Resource Allocation in Virtualized Environments}}, year = 2011 }

  %0 Conference Paper %1 HuBrKo2011-SEAMS-ResAlloc %A Huber, Nikolaus %A Brosig, Fabian %A Kounev, Samuel %B 6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011) %C New York, NY, USA %D 2011 %I ACM %P 90--99 %T Model-based Self-Adaptive Resource Allocation in Virtualized Environments %U http://dl.acm.org/authorize?425581 %X The adoption of virtualization and Cloud Computing technologies promises a number of benefits such as increased flexibility, better energy efficiency and lower operating costs for IT systems. However, highly variable workloads make it challenging to provide quality-of-service guarantees while at the same time ensuring efficient resource utilization. To avoid violations of service-level agreements (SLAs) or inefficient resource usage, resource allocations have to be adapted continuously during operation to reflect changes in application workloads. In this paper, we present a novel approach to self-adaptive resource allocation in virtualized environments based on online architecture-level performance models. We present a detailed case study of a representative enterprise application, the new SPECjEnterprise2010 benchmark, deployed in a virtualized cluster environment. The case study serves as a proof-of-concept demonstrating the effectiveness and practical applicability of our approach.
• 
  Evaluating and Modeling Virtualization Performance Overhead for Cloud Environments. Huber, Nikolaus; von Quast, Marcel; Hauck, Michael; Kounev, Samuel; in Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER 2011) (2011). 563–573. SciTePress.

  Acceptance Rate: 18/164 = 10.9%, Best Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Due to trends like Cloud Computing and Green IT, virtualization technologies are gaining increasing importance. They promise energy and cost savings by sharing physical resources, thus making resource usage more efficient. However, resource sharing and other factors have direct effects on system performance, which are not yet well-understood. Hence, performance prediction and performance management of services deployed in virtualized environments like public and private Clouds is a challenging task. Because of the large variety of virtualization solutions, a generic approach to predict the performance overhead of services running on virtualization platforms is highly desirable. In this paper, we present experimental results on two popular state-of-the-art virtualization platforms, Citrix XenServer 5.5 and VMware ESX 4.0, as representatives of the two major hypervisor architectures. Based on these results, we propose a basic, generic performance prediction model for the two different types of hypervisor architectures. The target is to predict the performance overhead for executing services on virtualized platforms.

  @inproceedings{HuQuHaKo2011-CLOSER-ModelVirtOverhead, abstract = {Due to trends like Cloud Computing and Green IT, virtualization technologies are gaining increasing importance. They promise energy and cost savings by sharing physical resources, thus making resource usage more efficient. However, resource sharing and other factors have direct effects on system performance, which are not yet well-understood. Hence, performance prediction and performance management of services deployed in virtualized environments like public and private Clouds is a challenging task. Because of the large variety of virtualization solutions, a generic approach to predict the performance overhead of services running on virtualization platforms is highly desirable. In this paper, we present experimental results on two popular state-of-the-art virtualization platforms, Citrix XenServer 5.5 and VMware ESX 4.0, as representatives of the two major hypervisor architectures. Based on these results, we propose a basic, generic performance prediction model for the two different types of hypervisor architectures. The target is to predict the performance overhead for executing services on virtualized platforms.}, author = {Huber, Nikolaus and von Quast, Marcel and Hauck, Michael and Kounev, Samuel}, booktitle = {Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER 2011)}, keywords = {Virtualization}, month = {05}, note = {Acceptance Rate: 18/164 = 10.9\%, <b>Best Paper Award</b>}, pages = {563 – 573}, publisher = {SciTePress}, title = {{E}valuating and {M}odeling {V}irtualization {P}erformance {O}verhead for {C}loud {E}nvironments}, year = 2011 }

  %0 Conference Paper %1 HuQuHaKo2011-CLOSER-ModelVirtOverhead %A Huber, Nikolaus %A von Quast, Marcel %A Hauck, Michael %A Kounev, Samuel %B Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER 2011) %D 2011 %I SciTePress %P 563 -- 573 %T Evaluating and Modeling Virtualization Performance Overhead for Cloud Environments %X Due to trends like Cloud Computing and Green IT, virtualization technologies are gaining increasing importance. They promise energy and cost savings by sharing physical resources, thus making resource usage more efficient. However, resource sharing and other factors have direct effects on system performance, which are not yet well-understood. Hence, performance prediction and performance management of services deployed in virtualized environments like public and private Clouds is a challenging task. Because of the large variety of virtualization solutions, a generic approach to predict the performance overhead of services running on virtualization platforms is highly desirable. In this paper, we present experimental results on two popular state-of-the-art virtualization platforms, Citrix XenServer 5.5 and VMware ESX 4.0, as representatives of the two major hypervisor architectures. Based on these results, we propose a basic, generic performance prediction model for the two different types of hypervisor architectures. The target is to predict the performance overhead for executing services on virtualized platforms.
• 
  Automated Simulation-Based Capacity Planning for Enterprise Data Fabrics. Kounev, Samuel; Bender, Konstantin; Brosig, Fabian; Huber, Nikolaus; Okamoto, Russell; in 4th International ICST Conference on Simulation Tools and Techniques (SIMUtools 2011), March 21--25, 2011, Barcelona, Spain (2011).

  Acceptance Rate (Full Paper): 29.8% (23/77), ICST Best Paper Award
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{KoBeBrHuOk2011-SIMUTools-DataFabrics, author = {Kounev, Samuel and Bender, Konstantin and Brosig, Fabian and Huber, Nikolaus and Okamoto, Russell}, booktitle = {4th International ICST Conference on Simulation Tools and Techniques (SIMUtools 2011), March 21–25, 2011, Barcelona, Spain}, keywords = {QPME_Bibliography}, month = {03}, note = {Acceptance Rate (Full Paper): 29.8\% (23/77), <b>ICST Best Paper Award</b>}, title = {Automated Simulation-Based Capacity Planning for Enterprise Data Fabrics}, year = 2011 }

  %0 Conference Paper %1 KoBeBrHuOk2011-SIMUTools-DataFabrics %A Kounev, Samuel %A Bender, Konstantin %A Brosig, Fabian %A Huber, Nikolaus %A Okamoto, Russell %B 4th International ICST Conference on Simulation Tools and Techniques (SIMUtools 2011), March 21--25, 2011, Barcelona, Spain %D 2011 %T Automated Simulation-Based Capacity Planning for Enterprise Data Fabrics
• 
  Automated Transformation of Component-based Software Architecture Models to Queueing Petri Nets. Meier, Philipp; Kounev, Samuel; Koziolek, Heiko; in 19th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2011), Singapore, July 25--27 (2011).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{MeKoKo2011-MASCOTS-PCMtoQPN, author = {Meier, Philipp and Kounev, Samuel and Koziolek, Heiko}, booktitle = {19th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2011), Singapore, July 25–27}, keywords = {QPN}, title = {Automated Transformation of Component-based Software Architecture Models to Queueing Petri Nets}, year = 2011 }

  %0 Conference Paper %1 MeKoKo2011-MASCOTS-PCMtoQPN %A Meier, Philipp %A Kounev, Samuel %A Koziolek, Heiko %B 19th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2011), Singapore, July 25--27 %D 2011 %T Automated Transformation of Component-based Software Architecture Models to Queueing Petri Nets

• 
  Analysis of the Performance-Influencing Factors of Virtualization Platforms. Huber, Nikolaus; von Quast, Marcel; Brosig, Fabian; Kounev, Samuel; in The 12th International Symposium on Distributed Objects, Middleware, and Applications (DOA 2010) (2010). Springer Verlag, Crete, Greece.

  Acceptance Rate (Full Paper): 33%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Nowadays, virtualization solutions are gaining increasing importance. By enabling the sharing of physical resources, thus making resource usage more efficient, they promise energy and cost savings. Additionally, virtualization is the key enabling technology for Cloud Computing and server consolidation. However, the effects of sharing resources on system performance are not yet well-understood. This makes performance prediction and performance management of services deployed in such dynamic systems very challenging. Because of the large variety of virtualization solutions, a generic approach to predict the performance influences of virtualization platforms is highly desirable. In this paper, we present a hierarchical model capturing the major performance-relevant factors of virtualization platforms. We then propose a general methodology to quantify the influence of the identified factors based on an empirical approach using benchmarks. Finally, we present a case study of Citrix XenServer 5.5, a state-of-the-art virtualization platform.

  @inproceedings{HuQuBrKo2010-DOA-AnalysisVirt, abstract = {Nowadays, virtualization solutions are gaining increasing importance. By enabling the sharing of physical resources, thus making resource usage more efficient, they promise energy and cost savings. Additionally, virtualization is the key enabling technology for Cloud Computing and server consolidation. However, the effects of sharing resources on system performance are not yet well-understood. This makes performance prediction and performance management of services deployed in such dynamic systems very challenging. Because of the large variety of virtualization solutions, a generic approach to predict the performance influences of virtualization platforms is highly desirable. In this paper, we present a hierarchical model capturing the major performance-relevant factors of virtualization platforms. We then propose a general methodology to quantify the influence of the identified factors based on an empirical approach using benchmarks. Finally, we present a case study of Citrix XenServer 5.5, a state-of-the-art virtualization platform.}, address = {Crete, Greece}, author = {Huber, Nikolaus and von Quast, Marcel and Brosig, Fabian and Kounev, Samuel}, booktitle = {The 12th International Symposium on Distributed Objects, Middleware, and Applications (DOA 2010)}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = 10, note = {Acceptance Rate (Full Paper): 33\%}, publisher = {Springer Verlag}, title = {{Analysis of the Performance-Influencing Factors of Virtualization Platforms}}, year = 2010 }

  %0 Conference Paper %1 HuQuBrKo2010-DOA-AnalysisVirt %A Huber, Nikolaus %A von Quast, Marcel %A Brosig, Fabian %A Kounev, Samuel %B The 12th International Symposium on Distributed Objects, Middleware, and Applications (DOA 2010) %C Crete, Greece %D 2010 %I Springer Verlag %T Analysis of the Performance-Influencing Factors of Virtualization Platforms %X Nowadays, virtualization solutions are gaining increasing importance. By enabling the sharing of physical resources, thus making resource usage more efficient, they promise energy and cost savings. Additionally, virtualization is the key enabling technology for Cloud Computing and server consolidation. However, the effects of sharing resources on system performance are not yet well-understood. This makes performance prediction and performance management of services deployed in such dynamic systems very challenging. Because of the large variety of virtualization solutions, a generic approach to predict the performance influences of virtualization platforms is highly desirable. In this paper, we present a hierarchical model capturing the major performance-relevant factors of virtualization platforms. We then propose a general methodology to quantify the influence of the identified factors based on an empirical approach using benchmarks. Finally, we present a case study of Citrix XenServer 5.5, a state-of-the-art virtualization platform.
• 
  Stochastic Performance Analysis and Capacity Planning of Publish/Subscribe Systems. Schröter, Arnd; Mühl, Gero; Kounev, Samuel; Parzyjegla, Helge; Richling, Jan; in 4th ACM International Conference on Distributed Event-Based Systems (DEBS 2010), July 12-15, Cambridge, United Kingdom (2010). ACM, New York, USA.

  Acceptance Rate: 25%
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{SchroterMuhlRichling2010Stochastic, author = {Schr{{\"o}}ter, Arnd and M{{\"u}}hl, Gero and Kounev, Samuel and Parzyjegla, Helge and Richling, Jan}, booktitle = {4th ACM International Conference on Distributed Event-Based Systems (DEBS 2010), July 12-15, Cambridge, United Kingdom}, keywords = {Optimization}, note = {Acceptance Rate: 25\%}, publisher = {ACM, New York, USA}, title = {{Stochastic Performance Analysis and Capacity Planning of Publish/Subscribe Systems}}, year = 2010 }

  %0 Conference Paper %1 SchroterMuhlRichling2010Stochastic %A Schröter, Arnd %A Mühl, Gero %A Kounev, Samuel %A Parzyjegla, Helge %A Richling, Jan %B 4th ACM International Conference on Distributed Event-Based Systems (DEBS 2010), July 12-15, Cambridge, United Kingdom %D 2010 %I ACM, New York, USA %T Stochastic Performance Analysis and Capacity Planning of Publish/Subscribe Systems

• 
  Stochastic Analysis of Hierarchical Publish/Subscribe Systems. Mühl, Gero; Schröter, Arnd; Parzyjegla, Helge; Kounev, Samuel; Richling, Jan; in Proceedings of the 15th International European Conference on Parallel and Distributed Computing (Euro-Par 2009), Delft, The Netherlands, August 25-28, 2009. (2009). Springer Verlag.

  Acceptance Rate (Full Paper): 33%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  With the gradual adoption of publish/subscribe systems in mission critical areas, it is essential that systems are subjected to rigorous performance analysis before they are put into production. However, existing approaches to performance modeling and analysis of publish/subscribe systems suffer from many limitations that seriously constrain their practical applicability. In this paper, we present a generalized method for stochastic analysis of publish/subscribe systems employing identity-based hierarchical routing. The method is based on an analytical model that addresses the major limitations underlying existing work in this area. In particular, it supports arbitrary broker overlay topologies and allows to set workload parameters, e.g., publication rates and subscription lifetimes, individually for each broker. The analysis is illustrated by a running example that helps to gain better understanding of the derived mathematical relationships.

  @inproceedings{MuScPaKoRi2009-EuroPar-StoAnalPubSub, abstract = {With the gradual adoption of publish/subscribe systems in mission critical areas, it is essential that systems are subjected to rigorous performance analysis before they are put into production. However, existing approaches to performance modeling and analysis of publish/subscribe systems suffer from many limitations that seriously constrain their practical applicability. In this paper, we present a generalized method for stochastic analysis of publish/subscribe systems employing identity-based hierarchical routing. The method is based on an analytical model that addresses the major limitations underlying existing work in this area. In particular, it supports arbitrary broker overlay topologies and allows to set workload parameters, e.g., publication rates and subscription lifetimes, individually for each broker. The analysis is illustrated by a running example that helps to gain better understanding of the derived mathematical relationships.}, author = {M{\"u}hl, Gero and Schr{\"o}ter, Arnd and Parzyjegla, Helge and Kounev, Samuel and Richling, Jan}, booktitle = {Proceedings of the 15th International European Conference on Parallel and Distributed Computing (Euro-Par 2009), Delft, The Netherlands, August 25-28, 2009.}, keywords = {Optimization}, note = {Acceptance Rate (Full Paper): 33%}, publisher = {Springer Verlag}, title = {{Stochastic Analysis of Hierarchical Publish/Subscribe Systems}}, year = 2009 }

  %0 Conference Paper %1 MuScPaKoRi2009-EuroPar-StoAnalPubSub %A Mühl, Gero %A Schröter, Arnd %A Parzyjegla, Helge %A Kounev, Samuel %A Richling, Jan %B Proceedings of the 15th International European Conference on Parallel and Distributed Computing (Euro-Par 2009), Delft, The Netherlands, August 25-28, 2009. %D 2009 %I Springer Verlag %T Stochastic Analysis of Hierarchical Publish/Subscribe Systems %U http://europar2009.ewi.tudelft.nl/ %X With the gradual adoption of publish/subscribe systems in mission critical areas, it is essential that systems are subjected to rigorous performance analysis before they are put into production. However, existing approaches to performance modeling and analysis of publish/subscribe systems suffer from many limitations that seriously constrain their practical applicability. In this paper, we present a generalized method for stochastic analysis of publish/subscribe systems employing identity-based hierarchical routing. The method is based on an analytical model that addresses the major limitations underlying existing work in this area. In particular, it supports arbitrary broker overlay topologies and allows to set workload parameters, e.g., publication rates and subscription lifetimes, individually for each broker. The analysis is illustrated by a running example that helps to gain better understanding of the derived mathematical relationships.

• 
  A Methodology for Performance Modeling of Distributed Event-Based Systems. Kounev, Samuel; Sachs, Kai; Bacon, Jean; Buchmann, Alejandro; in Proceedings of the 11th IEEE International Symposium on Object Oriented Real-Time Distributed Computing (ISORC 2008), Orlando, Florida, USA, May 5-7, 2008 (2008). 13–22. IEEE Computer Society, Washington, DC, USA.

  Acceptance Rate (Full Paper): 30%, Best-Paper-Award-Nomination
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Distributed event-based systems (DEBS) are gaining increasing attention in new application areas such as transport information monitoring, event-driven supply-chain management and ubiquitous sensor-rich environments. However, as DEBS increasingly enter the enterprise and commercial domains, performance and quality of service issues are becoming a major concern. While numerous approaches to performance modeling and evaluation of conventional request/reply-based distributed systems are available in the literature, no general approach exists for DEBS. This paper is the first to provide a comprehensive methodology for workload characterization and performance modeling of DEBS. A workload model of a generic DEBS is developed and operational analysis techniques are used to characterize the system traffic and derive an approximation for the mean event delivery latency. Following this, a modeling technique is presented that can be used for accurate performance prediction. The paper is concluded with a case study of a real life system demonstrating the effectiveness and practicality of the proposed approach.

  @inproceedings{KoSaBaBu2008-ISORC-ModelingDEBS, abstract = {Distributed event-based systems (DEBS) are gaining increasing attention in new application areas such as transport information monitoring, event-driven supply-chain management and ubiquitous sensor-rich environments. However, as DEBS increasingly enter the enterprise and commercial domains, performance and quality of service issues are becoming a major concern. While numerous approaches to performance modeling and evaluation of conventional request/reply-based distributed systems are available in the literature, no general approach exists for DEBS. This paper is the first to provide a comprehensive methodology for workload characterization and performance modeling of DEBS. A workload model of a generic DEBS is developed and operational analysis techniques are used to characterize the system traffic and derive an approximation for the mean event delivery latency. Following this, a modeling technique is presented that can be used for accurate performance prediction. The paper is concluded with a case study of a real life system demonstrating the effectiveness and practicality of the proposed approach.}, address = {Washington, DC, USA}, author = {Kounev, Samuel and Sachs, Kai and Bacon, Jean and Buchmann, Alejandro}, booktitle = {Proceedings of the 11th IEEE International Symposium on Object Oriented Real-Time Distributed Computing (ISORC 2008), Orlando, Florida, USA, May 5-7, 2008}, keywords = {QPME_Bibliography}, note = {Acceptance Rate (Full Paper): 30\%, <b>Best-Paper-Award-Nomination</b>}, pages = {13–22}, publisher = {IEEE Computer Society}, title = {{A Methodology for Performance Modeling of Distributed Event-Based Systems}}, year = 2008 }

  %0 Conference Paper %1 KoSaBaBu2008-ISORC-ModelingDEBS %A Kounev, Samuel %A Sachs, Kai %A Bacon, Jean %A Buchmann, Alejandro %B Proceedings of the 11th IEEE International Symposium on Object Oriented Real-Time Distributed Computing (ISORC 2008), Orlando, Florida, USA, May 5-7, 2008 %C Washington, DC, USA %D 2008 %I IEEE Computer Society %P 13--22 %T A Methodology for Performance Modeling of Distributed Event-Based Systems %X Distributed event-based systems (DEBS) are gaining increasing attention in new application areas such as transport information monitoring, event-driven supply-chain management and ubiquitous sensor-rich environments. However, as DEBS increasingly enter the enterprise and commercial domains, performance and quality of service issues are becoming a major concern. While numerous approaches to performance modeling and evaluation of conventional request/reply-based distributed systems are available in the literature, no general approach exists for DEBS. This paper is the first to provide a comprehensive methodology for workload characterization and performance modeling of DEBS. A workload model of a generic DEBS is developed and operational analysis techniques are used to characterize the system traffic and derive an approximation for the mean event delivery latency. Following this, a modeling technique is presented that can be used for accurate performance prediction. The paper is concluded with a case study of a real life system demonstrating the effectiveness and practicality of the proposed approach.

• 
  Autonomic QoS-Aware Resource Management in Grid Computing using Online Performance Models. Kounev, Samuel; Nou, Ramon; Torres, Jordi; in Proceedings of the Second International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2007), Nantes, France, October 23-25, 2007 (2007). 1–10. ICST, Brussels, Belgium.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, an approach to autonomic QoS-aware resource management in Grid computing based on online performance models is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.

  @inproceedings{KoNoTo2007-VALUETOOLS-GridAuton_QoS_Control, abstract = {As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, an approach to autonomic QoS-aware resource management in Grid computing based on online performance models is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.}, address = {Brussels, Belgium}, author = {Kounev, Samuel and Nou, Ramon and Torres, Jordi}, booktitle = {Proceedings of the Second International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2007), Nantes, France, October 23-25, 2007}, keywords = {Self-adaptive-systems}, pages = {1–10}, publisher = {ICST}, title = {{Autonomic QoS-Aware Resource Management in Grid Computing using Online Performance Models}}, year = 2007 }

  %0 Conference Paper %1 KoNoTo2007-VALUETOOLS-GridAuton_QoS_Control %A Kounev, Samuel %A Nou, Ramon %A Torres, Jordi %B Proceedings of the Second International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2007), Nantes, France, October 23-25, 2007 %C Brussels, Belgium %D 2007 %I ICST %P 1--10 %T Autonomic QoS-Aware Resource Management in Grid Computing using Online Performance Models %X As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, an approach to autonomic QoS-aware resource management in Grid computing based on online performance models is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.

• 
  Performance Modeling and Evaluation of Large-Scale J2EE Applications. Kounev, Samuel; Buchmann, Alejandro; in Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003 (2003). 273–283. Computer Measurement Group (CMG).

  Best-Paper-Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Modern J2EE applications are typically based on highly distributed architectures comprising multiple components deployed in a clustered environment. This makes it difficult for deployers to estimate the capacity of the deployment environment needed to guarantee that Service Level Agreements are met. This paper looks at the different approaches to this problem and discusses the difficulties that arise when one tries to apply them to large, real-world systems. The authors study a realistic J2EE application (the SPECjAppServer2002 benchmark) and show how analytical models can be exploited for capacity planning.

  @inproceedings{KoBu2003-CMG-PerfModeling_J2EEApps, abstract = {Modern J2EE applications are typically based on highly distributed architectures comprising multiple components deployed in a clustered environment. This makes it difficult for deployers to estimate the capacity of the deployment environment needed to guarantee that Service Level Agreements are met. This paper looks at the different approaches to this problem and discusses the difficulties that arise when one tries to apply them to large, real-world systems. The authors study a realistic J2EE application (the SPECjAppServer2002 benchmark) and show how analytical models can be exploited for capacity planning.}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = 12, note = {<b>Best-Paper-Award</b>}, organization = {Computer Measurement Group (CMG)}, pages = {273–283}, title = {{Performance Modeling and Evaluation of Large-Scale J2EE Applications}}, year = 2003 }

  %0 Conference Paper %1 KoBu2003-CMG-PerfModeling_J2EEApps %A Kounev, Samuel %A Buchmann, Alejandro %B Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003 %D 2003 %P 273--283 %T Performance Modeling and Evaluation of Large-Scale J2EE Applications %U http://www.cmg.org/proceedings/2003/3173.pdf %X Modern J2EE applications are typically based on highly distributed architectures comprising multiple components deployed in a clustered environment. This makes it difficult for deployers to estimate the capacity of the deployment environment needed to guarantee that Service Level Agreements are met. This paper looks at the different approaches to this problem and discusses the difficulties that arise when one tries to apply them to large, real-world systems. The authors study a realistic J2EE application (the SPECjAppServer2002 benchmark) and show how analytical models can be exploited for capacity planning.
• 
  PetStore-WS: Measuring the Performance Implications of Web Services. Juse, Kai S.; Kounev, Samuel; Buchmann, Alejandro; in Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003 (2003). 113–123. Computer Measurement Group (CMG).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Web Services are increasingly used to enable loosely coupled integration among heterogeneous systems but are perceived as a source of severe performance degradation. This paper looks at the impact on system performance when introducing Web Service interfaces to an originally tightly coupled application. Using two implementation variants of Sun's Java Pet Store application, one based strictly on the J2EE platform and the other implementing some interfaces as Web Services, performance is compared in terms of the achieved overall throughput, response times and latency.

  @inproceedings{JuKoBu2003-CMG-PetStoreWS, abstract = {Web Services are increasingly used to enable loosely coupled integration among heterogeneous systems but are perceived as a source of severe performance degradation. This paper looks at the impact on system performance when introducing Web Service interfaces to an originally tightly coupled application. Using two implementation variants of Sun's Java Pet Store application, one based strictly on the J2EE platform and the other implementing some interfaces as Web Services, performance is compared in terms of the achieved overall throughput, response times and latency.}, author = {Juse, Kai S. and Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003}, keywords = {Optimization}, month = 12, organization = {Computer Measurement Group (CMG)}, pages = {113–123}, title = {{PetStore-WS: Measuring the Performance Implications of Web Services}}, year = 2003 }

  %0 Conference Paper %1 JuKoBu2003-CMG-PetStoreWS %A Juse, Kai S. %A Kounev, Samuel %A Buchmann, Alejandro %B Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003 %D 2003 %P 113--123 %T PetStore-WS: Measuring the Performance Implications of Web Services %U http://www.cmg.org/proceedings/2003/3187.pdf %X Web Services are increasingly used to enable loosely coupled integration among heterogeneous systems but are perceived as a source of severe performance degradation. This paper looks at the impact on system performance when introducing Web Service interfaces to an originally tightly coupled application. Using two implementation variants of Sun's Java Pet Store application, one based strictly on the J2EE platform and the other implementing some interfaces as Web Services, performance is compared in terms of the achieved overall throughput, response times and latency.
• 
  Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets. Kounev, Samuel; Buchmann, Alejandro; in Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS 2003), Austin, Texas, USA, March 6-8, 2003 (2003). 143–155. IEEE Computer Society, Washington, DC, USA.

  Best-Paper-Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  In this paper we show how Queuing Petri Net (QPN) models can be exploited for performance analysis of distributed e-business systems. We study a real-world application, and demonstrate the benefits, in terms of modelling power and expressiveness, that QPN models provide over conventional modelling paradigms such as Queuing Networks and Petri Nets. As shown, QPNs facilitate the integration of both hardware and software aspects of system behavior in the same model. In addition to hardware contention and scheduling strategies, using QPNs one can easily model simultaneous resource possession, synchronization, blocking and contention for software resources. By validating the models presented through measurements, we show that they are not just powerful as a specification mechanism, but are also very powerful as a performance analysis and prediction tool. However, currently available tools and techniques for QPN analysis are limited. Improved solution methods, which enable larger models to be analyzed, need to be developed. By demonstrating the power of QPNs as a modelling paradigm in realistic scenarios, we hope to motivate further research in this area.

  @inproceedings{KoBu2003-ISPASS-ModelingUsingQPNs, abstract = {In this paper we show how Queuing Petri Net (QPN) models can be exploited for performance analysis of distributed e-business systems. We study a real-world application, and demonstrate the benefits, in terms of modelling power and expressiveness, that QPN models provide over conventional modelling paradigms such as Queuing Networks and Petri Nets. As shown, QPNs facilitate the integration of both hardware and software aspects of system behavior in the same model. In addition to hardware contention and scheduling strategies, using QPNs one can easily model simultaneous resource possession, synchronization, blocking and contention for software resources. By validating the models presented through measurements, we show that they are not just powerful as a specification mechanism, but are also very powerful as a performance analysis and prediction tool. However, currently available tools and techniques for QPN analysis are limited. Improved solution methods, which enable larger models to be analyzed, need to be developed. By demonstrating the power of QPNs as a modelling paradigm in realistic scenarios, we hope to motivate further research in this area.}, address = {Washington, DC, USA}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS 2003), Austin, Texas, USA, March 6-8, 2003}, keywords = {QPN}, month = {03}, note = {<b>Best-Paper-Award</b>}, pages = {143–155}, publisher = {IEEE Computer Society}, title = {Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets}, year = 2003 }

  %0 Conference Paper %1 KoBu2003-ISPASS-ModelingUsingQPNs %A Kounev, Samuel %A Buchmann, Alejandro %B Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS 2003), Austin, Texas, USA, March 6-8, 2003 %C Washington, DC, USA %D 2003 %I IEEE Computer Society %P 143--155 %T Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets %X In this paper we show how Queuing Petri Net (QPN) models can be exploited for performance analysis of distributed e-business systems. We study a real-world application, and demonstrate the benefits, in terms of modelling power and expressiveness, that QPN models provide over conventional modelling paradigms such as Queuing Networks and Petri Nets. As shown, QPNs facilitate the integration of both hardware and software aspects of system behavior in the same model. In addition to hardware contention and scheduling strategies, using QPNs one can easily model simultaneous resource possession, synchronization, blocking and contention for software resources. By validating the models presented through measurements, we show that they are not just powerful as a specification mechanism, but are also very powerful as a performance analysis and prediction tool. However, currently available tools and techniques for QPN analysis are limited. Improved solution methods, which enable larger models to be analyzed, need to be developed. By demonstrating the power of QPNs as a modelling paradigm in realistic scenarios, we hope to motivate further research in this area.

• 
  Performance Issues in E-Business Systems. Kounev, Samuel; Buchmann, Alejandro; in Proceedings of the International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, and e-Medicine on the Internet (SSGRR 2002w), L’Aquila, Italy, January 21-27, 2002 (2002).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Performance and scalability issues in e-business systems are gaining in importance as we move from hype and prototypes to real operational systems. Typical for this development is also the emergence of standard benchmarks of which TPC-W for transactional B2C systems and ECperf for performance and scalability measurement of application servers are two of the better known examples. In this paper we present an experience report with the ECperf benchmark defined by Sun and discuss performance issues that we observed in our implementation of the benchmark. Some of these issues are related to the specification of the benchmark, for which we made suggestions how to correct them and others are related to database connectivity, locking patterns, and the need for asynchronous processing.

  @inproceedings{KoBu2002-SSGRR-PerfIssuesEB, abstract = {Performance and scalability issues in e-business systems are gaining in importance as we move from hype and prototypes to real operational systems. Typical for this development is also the emergence of standard benchmarks of which TPC-W for transactional B2C systems and ECperf for performance and scalability measurement of application servers are two of the better known examples. In this paper we present an experience report with the ECperf benchmark defined by Sun and discuss performance issues that we observed in our implementation of the benchmark. Some of these issues are related to the specification of the benchmark, for which we made suggestions how to correct them and others are related to database connectivity, locking patterns, and the need for asynchronous processing.}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, and e-Medicine on the Internet (SSGRR 2002w), L'Aquila, Italy, January 21-27, 2002}, keywords = {Optimization}, month = {01}, title = {{Performance Issues in E-Business Systems}}, year = 2002 }

  %0 Conference Paper %1 KoBu2002-SSGRR-PerfIssuesEB %A Kounev, Samuel %A Buchmann, Alejandro %B Proceedings of the International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, and e-Medicine on the Internet (SSGRR 2002w), L'Aquila, Italy, January 21-27, 2002 %D 2002 %T Performance Issues in E-Business Systems %X Performance and scalability issues in e-business systems are gaining in importance as we move from hype and prototypes to real operational systems. Typical for this development is also the emergence of standard benchmarks of which TPC-W for transactional B2C systems and ECperf for performance and scalability measurement of application servers are two of the better known examples. In this paper we present an experience report with the ECperf benchmark defined by Sun and discuss performance issues that we observed in our implementation of the benchmark. Some of these issues are related to the specification of the benchmark, for which we made suggestions how to correct them and others are related to database connectivity, locking patterns, and the need for asynchronous processing.
• 
  Improving Data Access of J2EE Applications by Exploiting Asynchronous Messaging and Caching Services. Kounev, Samuel; Buchmann, Alejandro; in Proceedings of the 28th International Conference on Very Large Data Bases (VLDB 2002), Hong Kong, China, August 20--23, 2002 (2002). 574–585. VLDB Endowment, Morgan Kaufmann.

  Acceptance Rate (Full Paper): 14% Best-Paper-Award Nomination
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  The J2EE platform provides a variety of options for making business data persistent using DBMS technology. However, the integration with existing backend database systems has proven to be of crucial importance for the scalability and performance of J2EE applications, because modern e-business systems are extremely data-intensive. As a result, the data access layer, and the link between the application server and the database server in particular, are very susceptible to turning into a system bottleneck. In this paper we use the ECperf benchmark as an example of a realistic application in order to illustrate the problems mentioned above and discuss how they could be approached and eliminated. In particular, we show how asynchronous, message-based processing could be exploited to reduce the load on the DBMS and improve system performance, scalability and reliability. Furthermore, we discuss the major issues related to the correct use of entity beans (the components provided by J2EE for modelling persistent data) and present a number of methods to optimize their performance utilizing caching mechanisms. We have evaluated the proposed techniques through measurements and have documented the performance gains that they provide.

  @inproceedings{KoBu2002-VLDB-ImprovingDataAccessJ2EE, abstract = {The J2EE platform provides a variety of options for making business data persistent using DBMS technology. However, the integration with existing backend database systems has proven to be of crucial importance for the scalability and performance of J2EE applications, because modern e-business systems are extremely data-intensive. As a result, the data access layer, and the link between the application server and the database server in particular, are very susceptible to turning into a system bottleneck. In this paper we use the ECperf benchmark as an example of a realistic application in order to illustrate the problems mentioned above and discuss how they could be approached and eliminated. In particular, we show how asynchronous, message-based processing could be exploited to reduce the load on the DBMS and improve system performance, scalability and reliability. Furthermore, we discuss the major issues related to the correct use of entity beans (the components provided by J2EE for modelling persistent data) and present a number of methods to optimize their performance utilizing caching mechanisms. We have evaluated the proposed techniques through measurements and have documented the performance gains that they provide.}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 28th International Conference on Very Large Data Bases (VLDB 2002), Hong Kong, China, August 20–23, 2002}, keywords = {Optimization}, note = {Acceptance Rate (Full Paper): 14\% <b>Best-Paper-Award Nomination</b>}, pages = {574–585}, publisher = {VLDB Endowment, Morgan Kaufmann}, title = {{Improving Data Access of J2EE Applications by Exploiting Asynchronous Messaging and Caching Services}}, year = 2002 }

  %0 Conference Paper %1 KoBu2002-VLDB-ImprovingDataAccessJ2EE %A Kounev, Samuel %A Buchmann, Alejandro %B Proceedings of the 28th International Conference on Very Large Data Bases (VLDB 2002), Hong Kong, China, August 20--23, 2002 %D 2002 %I VLDB Endowment, Morgan Kaufmann %P 574--585 %T Improving Data Access of J2EE Applications by Exploiting Asynchronous Messaging and Caching Services %U http://www.vldb.org/conf/2002/S16P03.pdf %X The J2EE platform provides a variety of options for making business data persistent using DBMS technology. However, the integration with existing backend database systems has proven to be of crucial importance for the scalability and performance of J2EE applications, because modern e-business systems are extremely data-intensive. As a result, the data access layer, and the link between the application server and the database server in particular, are very susceptible to turning into a system bottleneck. In this paper we use the ECperf benchmark as an example of a realistic application in order to illustrate the problems mentioned above and discuss how they could be approached and eliminated. In particular, we show how asynchronous, message-based processing could be exploited to reduce the load on the DBMS and improve system performance, scalability and reliability. Furthermore, we discuss the major issues related to the correct use of entity beans (the components provided by J2EE for modelling persistent data) and present a number of methods to optimize their performance utilizing caching mechanisms. We have evaluated the proposed techniques through measurements and have documented the performance gains that they provide.

• 
  The Analysis Phase in the Development of E-Commerce Software Systems. Kounev, Samuel; Nikolov, Kiril; in Proceedings of the Tools Eastern Europe ’99 Conference on Technology of Object Oriented Languages and Systems, Sofia-Blagoevgrad, Bulgaria, June 1-4, 1999 (1999).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{KoNi1999-TOOLS_EE-Analysis_Phase, author = {Kounev, Samuel and Nikolov, Kiril}, booktitle = {Proceedings of the Tools Eastern Europe '99 Conference on Technology of Object Oriented Languages and Systems, Sofia-Blagoevgrad, Bulgaria, June 1-4, 1999}, keywords = {descartes}, month = {06}, title = {{The Analysis Phase in the Development of E-Commerce Software Systems}}, year = 1999 }

  %0 Conference Paper %1 KoNi1999-TOOLS_EE-Analysis_Phase %A Kounev, Samuel %A Nikolov, Kiril %B Proceedings of the Tools Eastern Europe '99 Conference on Technology of Object Oriented Languages and Systems, Sofia-Blagoevgrad, Bulgaria, June 1-4, 1999 %D 1999 %T The Analysis Phase in the Development of E-Commerce Software Systems

• [ 2024 ]
• [ 2023 ]
• [ 2022 ]
• [ 2021 ]
• [ 2020 ]
• [ 2019 ]
• [ 2018 ]
• [ 2017 ]
• [ 2015 ]
• [ 2014 ]
• [ 2013 ]
• [ 2012 ]
• [ 2007 ]
• [ 2006 ]

• 
  Modelling and Optimizing a Graph-based Settlement Delineation and Analysis Workflow. Gruber, Lorenz; Herbst, Nikolas; Kounev, Samuel; Esch, Thomas; Nguyen, Thanh; in Softwaretechnik-Trends Band 44, Heft 4 (2024). Gesellschaft für Informatik e.V.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @incollection{incollection, author = {Gruber, Lorenz and Herbst, Nikolas and Kounev, Samuel and Esch, Thomas and Nguyen, Thanh}, booktitle = {Softwaretechnik-Trends Band 44, Heft 4}, keywords = {t_interdisciplinary}, publisher = {Gesellschaft für Informatik e.V.}, title = {Modelling and Optimizing a Graph-based Settlement Delineation and Analysis Workflow}, year = 2024 }

  %0 Book Section %1 incollection %A Gruber, Lorenz %A Herbst, Nikolas %A Kounev, Samuel %A Esch, Thomas %A Nguyen, Thanh %B Softwaretechnik-Trends Band 44, Heft 4 %D 2024 %I Gesellschaft für Informatik e.V. %T Modelling and Optimizing a Graph-based Settlement Delineation and Analysis Workflow %U https://dl.gi.de/handle/20.500.12116/45538
• 
  Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices. Borst, Vanessa; Dittus, Timo; Müller, Konstantin; Kounev, Samuel; in 47th German Conference on Artificial Intelligence - KI 2024 (2024).
  • [ BibTeX ]
  • [ EndNote ]
  • [ DOI ]
  • [ Download ]
  @conference{borst2024early, author = {Borst, Vanessa and Dittus, Timo and Müller, Konstantin and Kounev, Samuel}, booktitle = {47th German Conference on Artificial Intelligence - KI 2024}, keywords = {t_interdisciplinary}, title = {Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices}, year = 2024 }

  %0 Generic %1 borst2024early %A Borst, Vanessa %A Dittus, Timo %A Müller, Konstantin %A Kounev, Samuel %B 47th German Conference on Artificial Intelligence - KI 2024 %D 2024 %R https://doi.org/10.1007/978-3-031-70893-0_21 %T Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices
• 
  Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis. Bauer, André; Dittus, Timo; Straesser, Martin; Kamatar, Alok; Baughman, Matt; Beierlieb, Lukas; Hadry, Marius; Grillmeyer, Daniel; Lubas, Yannik; Kounev, Samuel; Foster, Ian; Chard, Kyle; in Companion of the 15th ACM/SPEC International Conference on Performance Engineering (2024). 72–76. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

  @inproceedings{10.1145/3629527.3651843, abstract = {As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.}, address = {New York, NY, USA}, author = {Bauer, André and Dittus, Timo and Straesser, Martin and Kamatar, Alok and Baughman, Matt and Beierlieb, Lukas and Hadry, Marius and Grillmeyer, Daniel and Lubas, Yannik and Kounev, Samuel and Foster, Ian and Chard, Kyle}, booktitle = {Companion of the 15th ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {72–76}, publisher = {Association for Computing Machinery}, series = {ICPE '24 Companion}, title = {Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis}, year = 2024 }

  %0 Conference Paper %1 10.1145/3629527.3651843 %A Bauer, André %A Dittus, Timo %A Straesser, Martin %A Kamatar, Alok %A Baughman, Matt %A Beierlieb, Lukas %A Hadry, Marius %A Grillmeyer, Daniel %A Lubas, Yannik %A Kounev, Samuel %A Foster, Ian %A Chard, Kyle %B Companion of the 15th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2024 %I Association for Computing Machinery %P 72–76 %T Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis %U https://doi.org/10.1145/3629527.3651843 %X As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

• 
  Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs. Bauer, André; Straesser, Martin; Leznik, Mark; Beierlieb, Lukas; Hadry, Marius; Hudson, Nathaniel; Chard, Kyle; Kounev, Samuel; Foster, Ian; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 95–99. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.

  @inproceedings{10.1145/3578245.3584693, abstract = {Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.}, address = {New York, NY, USA}, author = {Bauer, Andr\'{e} and Straesser, Martin and Leznik, Mark and Beierlieb, Lukas and Hadry, Marius and Hudson, Nathaniel and Chard, Kyle and Kounev, Samuel and Foster, Ian}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {distance_measures}, pages = {95–99}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs}, year = 2023 }

  %0 Conference Paper %1 10.1145/3578245.3584693 %A Bauer, André %A Straesser, Martin %A Leznik, Mark %A Beierlieb, Lukas %A Hadry, Marius %A Hudson, Nathaniel %A Chard, Kyle %A Kounev, Samuel %A Foster, Ian %B Companion of the 2023 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2023 %I Association for Computing Machinery %P 95–99 %T Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs %U https://doi.org/10.1145/3578245.3584693 %X Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.
• 
  Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example. Prantl, Thomas; Engel, Simon; Horn, Lukas; Kaiser, Dennis; Iffländer, Lukas; Bauer, André; Krupitzer, Christian; Kounev, Samuel; in Information Security Practice and Experience, W. Meng, Z. Yan, V. Piuri (eds.) (2023). 284–298. Springer Nature Singapore, Singapore.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority.

  @inproceedings{PrEnHoDeIfBaKrKo_ISPEC_De_Bello, abstract = {In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority.}, address = {Singapore}, author = {Prantl, Thomas and Engel, Simon and Horn, Lukas and Kaiser, Dennis and Iffl\"ander, Lukas and Bauer, Andr\'e and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Information Security Practice and Experience}, editor = {Meng, Weizhi and Yan, Zheng and Piuri, Vincenzo}, keywords = {descartes}, pages = {284–298}, publisher = {Springer Nature Singapore}, title = {Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example}, year = 2023 }

  %0 Conference Paper %1 PrEnHoDeIfBaKrKo_ISPEC_De_Bello %A Prantl, Thomas %A Engel, Simon %A Horn, Lukas %A Kaiser, Dennis %A Iffländer, Lukas %A Bauer, André %A Krupitzer, Christian %A Kounev, Samuel %B Information Security Practice and Experience %C Singapore %D 2023 %E Meng, Weizhi %E Yan, Zheng %E Piuri, Vincenzo %I Springer Nature Singapore %P 284--298 %T Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example %X In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority. %@ 978-981-99-7032-2

• 
  Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning. Bauer, André; Straesser, Martin; Beierlieb, Lukas; Meissner, Maximilian; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). ACM.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {machine_learning}, month = {04}, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }

  %0 Generic %1 bauer2022automated %A Bauer, André %A Straesser, Martin %A Beierlieb, Lukas %A Meissner, Maximilian %A Kounev, Samuel %B Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering %D 2022 %I ACM %T Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning
• 
  Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity. Leznik, Mark; Grohmann, Johannes; Kliche, Nina; Bauer, André; Seybold, Daniel; Eismann, Simon; Kounev, Samuel; Domaschka, Jörg; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 89–96. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.

  @inproceedings{leznik2022dissimilar, abstract = {Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.}, address = {New York, NY, USA}, author = {Leznik, Mark and Grohmann, Johannes and Kliche, Nina and Bauer, Andr\'{e} and Seybold, Daniel and Eismann, Simon and Kounev, Samuel and Domaschka, J\"{o}rg}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {analysis}, pages = {89–96}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity}, year = 2022 }

  %0 Conference Paper %1 leznik2022dissimilar %A Leznik, Mark %A Grohmann, Johannes %A Kliche, Nina %A Bauer, André %A Seybold, Daniel %A Eismann, Simon %A Kounev, Samuel %A Domaschka, Jörg %B Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering %C New York, NY, USA %D 2022 %I Association for Computing Machinery %P 89–96 %T Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity %U https://doi.org/10.1145/3489525.3511699 %X Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.
• 
  A data-driven Sensor Model for LIDAR Range Measurements used for Mobile Robot Navigation. Spiess, Florian; Strobel, Norbert; Kaupp, Tobias; Kounev, Samuel; in Proceedings of the IEEE IRC 2022 Conference (2022).
  • [ BibTeX ]
  • [ EndNote ]
  @conference{spiess2022datadriven, author = {Spiess, Florian and Strobel, Norbert and Kaupp, Tobias and Kounev, Samuel}, booktitle = {Proceedings of the IEEE IRC 2022 Conference}, keywords = {t_interdisciplinary}, title = {A data-driven Sensor Model for LIDAR Range Measurements used for Mobile Robot Navigation}, year = 2022 }

  %0 Generic %1 spiess2022datadriven %A Spiess, Florian %A Strobel, Norbert %A Kaupp, Tobias %A Kounev, Samuel %B Proceedings of the IEEE IRC 2022 Conference %D 2022 %T A data-driven Sensor Model for LIDAR Range Measurements used for Mobile Robot Navigation

• 
  Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT. Prantl, Thomas; Ten, Peter; Iffländer, Lukas; Herrnleben, Stefan; Dmitrenko, Alexandra; Kounev, Samuel; Krupitzer, Christian; in 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrTeIfHeDmKo2021-ICPE-GroupEncryptionCentralizedBenchmark, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Herrnleben, Stefan and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2021 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {Group\_Encyrption\_Scheme}, month = {04}, series = {ICPE'21}, title = {Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT}, year = 2021 }

  %0 Conference Paper %1 PrTeIfHeDmKo2021-ICPE-GroupEncryptionCentralizedBenchmark %A Prantl, Thomas %A Ten, Peter %A Iffländer, Lukas %A Herrnleben, Stefan %A Dmitrenko, Alexandra %A Kounev, Samuel %A Krupitzer, Christian %B 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) %D 2021 %T Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT
• 
  Performance Impact Analysis of Securing MQTT Using TLS. Prantl, Thomas; Iffländer, Lukas; Herrnleben, Stefan; Engel, Simon; Kounev, Samuel; Krupitzer, Christian; in 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrTeIfHeDmKo2021-ICPE-MQTT_TLS, author = {Prantl, Thomas and Iffländer, Lukas and Herrnleben, Stefan and Engel, Simon and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2021 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {IoT}, month = {04}, series = {ICPE'21}, title = {Performance Impact Analysis of Securing MQTT Using TLS}, year = 2021 }

  %0 Conference Paper %1 PrTeIfHeDmKo2021-ICPE-MQTT_TLS %A Prantl, Thomas %A Iffländer, Lukas %A Herrnleben, Stefan %A Engel, Simon %A Kounev, Samuel %A Krupitzer, Christian %B 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) %D 2021 %T Performance Impact Analysis of Securing MQTT Using TLS

• 
  A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation. Züfle, Marwin; Kounev, Samuel; in Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications (2020).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{ZuKo-AAIA2020-Framework, author = {Z{\"u}fle, Marwin and Kounev, Samuel}, booktitle = {Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications}, keywords = {online_monitoring_and_forecasting}, month = {09}, title = {A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation}, year = 2020 }

  %0 Conference Paper %1 ZuKo-AAIA2020-Framework %A Züfle, Marwin %A Kounev, Samuel %B Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications %D 2020 %T A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation
• 
  An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series. Bauer, André; Züfle, Marwin; Grohmann, Johannes; Schmitt, Norbert; Herbst, Nikolas; Kounev, Samuel; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2020). 48–55. Association for Computing Machinery (ACM), New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.

  @inproceedings{BaZuGrScHeKo-ICPE20-Seasonal-Forecast, abstract = {Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.}, address = {New York, NY, USA}, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Grohmann, Johannes and Schmitt, Norbert and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {PRISMA}, month = {04}, pages = {48–55}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {{An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series}}, year = 2020 }

  %0 Conference Paper %1 BaZuGrScHeKo-ICPE20-Seasonal-Forecast %A Bauer, André %A Züfle, Marwin %A Grohmann, Johannes %A Schmitt, Norbert %A Herbst, Nikolas %A Kounev, Samuel %B Proceedings of the ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2020 %I Association for Computing Machinery (ACM) %P 48--55 %R 10.1145/3358960.3379123 %T An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series %U https://doi.org/10.1145/3358960.3379123 %X Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy. %@ 9781450369916
• 
  Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field. Bauer, André; Züfle, Marwin; Herbst, Nikolas; Kounev, Samuel; Curtef, Valentin; in Proceedings of the 36th International Conference on Data Engineering (ICDE) (2020).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{BaZuHeKoCu-ICDE-Telescope, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 36th International Conference on Data Engineering (ICDE)}, keywords = {Prediction}, month = {04}, title = {Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field}, year = 2020 }

  %0 Conference Paper %1 BaZuHeKoCu-ICDE-Telescope %A Bauer, André %A Züfle, Marwin %A Herbst, Nikolas %A Kounev, Samuel %A Curtef, Valentin %B Proceedings of the 36th International Conference on Data Engineering (ICDE) %D 2020 %T Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field

• 
  Performance Oriented Dynamic Bypassing for Intrusion Detection Systems. Iffländer, Lukas; Stoll, Jonathan; Rawtani, Nishant; Lesch, Veronika; Lange, Klaus-Dieter; Kounev, Samuel; in Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 159–166. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

  @inproceedings{Ifflaender2019_PerformanceOrientedDynamicBypassingforIntrusionDetectionSystems, abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Stoll, Jonathan and Rawtani, Nishant and Lesch, Veronika and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering}, keywords = {Optimization}, pages = {159–166}, publisher = {ACM}, series = {ICPE '19}, title = {Performance Oriented Dynamic Bypassing for Intrusion Detection Systems}, year = 2019 }

  %0 Conference Paper %1 Ifflaender2019_PerformanceOrientedDynamicBypassingforIntrusionDetectionSystems %A Iffländer, Lukas %A Stoll, Jonathan %A Rawtani, Nishant %A Lesch, Veronika %A Lange, Klaus-Dieter %A Kounev, Samuel %B Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2019 %I ACM %P 159--166 %T Performance Oriented Dynamic Bypassing for Intrusion Detection Systems %U http://doi.acm.org/10.1145/3297663.3310313 %X Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

• 
  Run-time Prediction of Power Consumption for Component Deployments. von Kistowski, Jóakim; Deffner, Maximilian; Kounev, Samuel; in Proceedings of the 15th IEEE International Conference on Autonomic Computing (ICAC 2018) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  The Power consumption of servers in data centers depends greatly on the software running on each server and how it interacts with the hardware. Different deployments of distributed software components on heterogeneous servers can lead to significant differences in power consumption, depending on the server allocation and the current workload. As workloads and load intensity change, components may be re-deployed or exchanged in order to reduce the power consumption for the current load profile. The decision on which component to place on which server during run-time remains difficult as the power consumption that would result from such a placement remains unknown. Existing work on component deployment optimization at run-time focuses on maximizing performance or considers power in the context of static design time decisions. In this paper, we introduce a model to predict the power consumption of component placements at run-time based on the load and power profile collected for a running distributed application in a heterogeneous environment. In addition, we present a model that enables the use of our approach without dedicated power measurement devices, predicting power consumption based on load intensity and performance counters. We show that we can predict the power consumption of two different distributed web applications with a mean absolute percentage error of 2.21% and with an error of 1.04% when predicting a previously unobserved load level.

  @inproceedings{KiDeKo2018-ICAC-PowerPrediction, abstract = {The Power consumption of servers in data centers depends greatly on the software running on each server and how it interacts with the hardware. Different deployments of distributed software components on heterogeneous servers can lead to significant differences in power consumption, depending on the server allocation and the current workload. As workloads and load intensity change, components may be re-deployed or exchanged in order to reduce the power consumption for the current load profile. The decision on which component to place on which server during run-time remains difficult as the power consumption that would result from such a placement remains unknown. Existing work on component deployment optimization at run-time focuses on maximizing performance or considers power in the context of static design time decisions. In this paper, we introduce a model to predict the power consumption of component placements at run-time based on the load and power profile collected for a running distributed application in a heterogeneous environment. In addition, we present a model that enables the use of our approach without dedicated power measurement devices, predicting power consumption based on load intensity and performance counters. We show that we can predict the power consumption of two different distributed web applications with a mean absolute percentage error of 2.21% and with an error of 1.04% when predicting a previously unobserved load level.}, author = {von Kistowski, J{\'o}akim and Deffner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 15th IEEE International Conference on Autonomic Computing (ICAC 2018)}, keywords = {Power}, month = {09}, title = {{Run-time Prediction of Power Consumption for Component Deployments}}, year = 2018 }

  %0 Conference Paper %1 KiDeKo2018-ICAC-PowerPrediction %A von Kistowski, Jóakim %A Deffner, Maximilian %A Kounev, Samuel %B Proceedings of the 15th IEEE International Conference on Autonomic Computing (ICAC 2018) %D 2018 %T Run-time Prediction of Power Consumption for Component Deployments %X The Power consumption of servers in data centers depends greatly on the software running on each server and how it interacts with the hardware. Different deployments of distributed software components on heterogeneous servers can lead to significant differences in power consumption, depending on the server allocation and the current workload. As workloads and load intensity change, components may be re-deployed or exchanged in order to reduce the power consumption for the current load profile. The decision on which component to place on which server during run-time remains difficult as the power consumption that would result from such a placement remains unknown. Existing work on component deployment optimization at run-time focuses on maximizing performance or considers power in the context of static design time decisions. In this paper, we introduce a model to predict the power consumption of component placements at run-time based on the load and power profile collected for a running distributed application in a heterogeneous environment. In addition, we present a model that enables the use of our approach without dedicated power measurement devices, predicting power consumption based on load intensity and performance counters. We show that we can predict the power consumption of two different distributed web applications with a mean absolute percentage error of 2.21% and with an error of 1.04% when predicting a previously unobserved load level.

• 
  Emulating the Power Consumption Behavior of Server Workloads using CPU Performance Counters. Schmitt, Norbert; von Kistowski, Jóakim; Kounev, Samuel; in Proceedings of the 25th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (2017).

  (Short Paper) Acceptance Rate: 26/84 = 31%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  The accurate measurement of a server's power consumption when running realistic workloads enables characterization of its energy efficiency and helps to make better provisioning and workload placement decisions. Information on the energy efficiency of a server for a given target workload can greatly influence such decisions and thus the final energy efficiency of a cluster or data center. However, measuring energy efficiency and power consumption of server applications has become challenging as applications are often distributed or require work intensive configuration, setup, and specialized load drivers for reproducible testing. As a result, it may be not feasible to perform tests using the actual workload that is to be deployed. We introduce an approach to create small-scale workloads that emulate the power consumption-relevant behavior of an application by deliberately triggering specific power relevant performance counter events. These workloads can then be easily deployed on a target server for fast and efficient power characterization. We validate the proposed approach by approximating the power consumption behavior of different workloads at multiple load levels. We show that our approach is capable of producing small-scale workloads that reflect the power consumption behavior of their reference applications over multiple load levels with a minimum error of less than 1%.

  @inproceedings{ScKiKo2017-MASCOTS-Pet, abstract = {The accurate measurement of a server's power consumption when running realistic workloads enables characterization of its energy efficiency and helps to make better provisioning and workload placement decisions. Information on the energy efficiency of a server for a given target workload can greatly influence such decisions and thus the final energy efficiency of a cluster or data center. However, measuring energy efficiency and power consumption of server applications has become challenging as applications are often distributed or require work intensive configuration, setup, and specialized load drivers for reproducible testing. As a result, it may be not feasible to perform tests using the actual workload that is to be deployed. We introduce an approach to create small-scale workloads that emulate the power consumption-relevant behavior of an application by deliberately triggering specific power relevant performance counter events. These workloads can then be easily deployed on a target server for fast and efficient power characterization. We validate the proposed approach by approximating the power consumption behavior of different workloads at multiple load levels. We show that our approach is capable of producing small-scale workloads that reflect the power consumption behavior of their reference applications over multiple load levels with a minimum error of less than 1%.}, author = {Schmitt, Norbert and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 25th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems}, keywords = {Power}, month = {09}, note = {{(Short Paper) Acceptance Rate: 26/84 = 31\%}}, series = {MASCOTS '17}, title = {{Emulating the Power Consumption Behavior of Server Workloads using CPU Performance Counters}}, year = 2017 }

  %0 Conference Paper %1 ScKiKo2017-MASCOTS-Pet %A Schmitt, Norbert %A von Kistowski, Jóakim %A Kounev, Samuel %B Proceedings of the 25th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems %D 2017 %T Emulating the Power Consumption Behavior of Server Workloads using CPU Performance Counters %X The accurate measurement of a server's power consumption when running realistic workloads enables characterization of its energy efficiency and helps to make better provisioning and workload placement decisions. Information on the energy efficiency of a server for a given target workload can greatly influence such decisions and thus the final energy efficiency of a cluster or data center. However, measuring energy efficiency and power consumption of server applications has become challenging as applications are often distributed or require work intensive configuration, setup, and specialized load drivers for reproducible testing. As a result, it may be not feasible to perform tests using the actual workload that is to be deployed. We introduce an approach to create small-scale workloads that emulate the power consumption-relevant behavior of an application by deliberately triggering specific power relevant performance counter events. These workloads can then be easily deployed on a target server for fast and efficient power characterization. We validate the proposed approach by approximating the power consumption behavior of different workloads at multiple load levels. We show that our approach is capable of producing small-scale workloads that reflect the power consumption behavior of their reference applications over multiple load levels with a minimum error of less than 1%.
• 
  Self-Tuning Resource Demand Estimation. Grohmann, Johannes; Herbst, Nikolas; Spinner, Simon; Kounev, Samuel; in Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017) (2017). 21–26.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.

  @inproceedings{GrHeSpKo2017-ICAC-RDE, abstract = {The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.}, author = {Grohmann, Johannes and Herbst, Nikolas and Spinner, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017)}, keywords = {Self-adaptive-systems}, month = {07}, pages = {21–26}, title = {{Self-Tuning Resource Demand Estimation}}, year = 2017 }

  %0 Conference Paper %1 GrHeSpKo2017-ICAC-RDE %A Grohmann, Johannes %A Herbst, Nikolas %A Spinner, Simon %A Kounev, Samuel %B Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017) %D 2017 %P 21--26 %R 10.1109/ICAC.2017.19 %T Self-Tuning Resource Demand Estimation %U https://doi.org/10.1109/ICAC.2017.19 %X The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.

• 
  Univariate Interpolation-based Modeling of Power and Performance. von Kistowski, Jóakim; Kounev, Samuel; in Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2015) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Performance and power scale non-linearly with device utilization, making characterization and prediction of energy efficiency at a given load level a challenging issue. A common approach to address this problem is the creation of power or performance state tables for a pre-measured subset of all possible system states. Approaches to determine performance and power for a state not included in the measured subset use simple interpolation, such as nearest neighbor interpolation, or define state switching rules. This leads to a loss in accuracy, as unmeasured system states are not considered. In this paper, we compare different interpolation functions and automatically configure and select functions for a given domain or measurement set. We evaluate our approach by comparing interpolation of measurement data subsets against power and performance measurements on a commodity server. We show that for non-extrapolating models interpolation is significantly more accurate than regression, with our automatically configured interpolation function improving modeling accuracy up to 43.6%.}

  @inproceedings{KiKo2015-VALUETOOLS, abstract = {{Performance and power scale non-linearly with device utilization, making characterization and prediction of energy efficiency at a given load level a challenging issue. A common approach to address this problem is the creation of power or performance state tables for a pre-measured subset of all possible system states. Approaches to determine performance and power for a state not included in the measured subset use simple interpolation, such as nearest neighbor interpolation, or define state switching rules. This leads to a loss in accuracy, as unmeasured system states are not considered. In this paper, we compare different interpolation functions and automatically configure and select functions for a given domain or measurement set. We evaluate our approach by comparing interpolation of measurement data subsets against power and performance measurements on a commodity server. We show that for non-extrapolating models interpolation is significantly more accurate than regression, with our automatically configured interpolation function improving modeling accuracy up to 43.6\%.}}, author = {von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2015)}, keywords = {Power}, month = 12, title = {{Univariate Interpolation-based Modeling of Power and Performance}}, year = 2015 }

  %0 Conference Paper %1 KiKo2015-VALUETOOLS %A von Kistowski, Jóakim %A Kounev, Samuel %B Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2015) %D 2015 %T Univariate Interpolation-based Modeling of Power and Performance %X Performance and power scale non-linearly with device utilization, making characterization and prediction of energy efficiency at a given load level a challenging issue. A common approach to address this problem is the creation of power or performance state tables for a pre-measured subset of all possible system states. Approaches to determine performance and power for a state not included in the measured subset use simple interpolation, such as nearest neighbor interpolation, or define state switching rules. This leads to a loss in accuracy, as unmeasured system states are not considered. In this paper, we compare different interpolation functions and automatically configure and select functions for a given domain or measurement set. We evaluate our approach by comparing interpolation of measurement data subsets against power and performance measurements on a commodity server. We show that for non-extrapolating models interpolation is significantly more accurate than regression, with our automatically configured interpolation function improving modeling accuracy up to 43.6%.}
• 
  Challenges of Assessing the Hypercall Interface Robustness. Carvalho, Diogo; Antunes, Nuno; Vieira, Marco; Milenkoski, Aleksandar; Kounev, Samuel; in The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015) (2015). IEEE.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.

  @inproceedings{CaAnViMiKo2015-DSN-Challenges, abstract = {Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.}, author = {Carvalho, Diogo and Antunes, Nuno and Vieira, Marco and Milenkoski, Aleksandar and Kounev, Samuel}, booktitle = {The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015)}, keywords = {Reliability}, month = {06}, publisher = {{IEEE}}, title = {{Challenges of Assessing the Hypercall Interface Robustness}}, year = 2015 }

  %0 Conference Paper %1 CaAnViMiKo2015-DSN-Challenges %A Carvalho, Diogo %A Antunes, Nuno %A Vieira, Marco %A Milenkoski, Aleksandar %A Kounev, Samuel %B The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015) %D 2015 %I IEEE %T Challenges of Assessing the Hypercall Interface Robustness %U http://2015.dsn.org/program/ %X Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.

• 
  Modeling of I/O Performance Interference in Virtualized Environments with Queueing Petri Nets. Noorshams, Qais; Rostami, Kiana; Kounev, Samuel; Reussner, Ralf; in Proceedings of the IEEE 22nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (2014).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{NoRoKoRe2014-MASCOTS-IOQPNs, author = {Noorshams, Qais and Rostami, Kiana and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the IEEE 22nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems}, keywords = {Virtualization}, month = {{September}}, series = {MASCOTS 2014}, title = {{Modeling of I/O Performance Interference in Virtualized Environments with Queueing Petri Nets}}, year = 2014 }

  %0 Conference Paper %1 NoRoKoRe2014-MASCOTS-IOQPNs %A Noorshams, Qais %A Rostami, Kiana %A Kounev, Samuel %A Reussner, Ralf %B Proceedings of the IEEE 22nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems %D 2014 %T Modeling of I/O Performance Interference in Virtualized Environments with Queueing Petri Nets

• 
  Evaluating Approaches for Performance Prediction in Virtualized Environments. Brosig, Fabian; Gorsler, Fabian; Huber, Nikolaus; Kounev, Samuel; in Proceedings of the IEEE 21st International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013) (2013).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{BrGoHuKo2013-MASCOTS-EvaluationApproachesForPerformancePredictionInVirtualizedEnvironments, author = {Brosig, Fabian and Gorsler, Fabian and Huber, Nikolaus and Kounev, Samuel}, booktitle = {Proceedings of the IEEE 21st International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013)}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {08}, title = {{Evaluating Approaches for Performance Prediction in Virtualized Environments}}, year = 2013 }

  %0 Conference Paper %1 BrGoHuKo2013-MASCOTS-EvaluationApproachesForPerformancePredictionInVirtualizedEnvironments %A Brosig, Fabian %A Gorsler, Fabian %A Huber, Nikolaus %A Kounev, Samuel %B Proceedings of the IEEE 21st International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013) %D 2013 %T Evaluating Approaches for Performance Prediction in Virtualized Environments
• 
  Elasticity in Cloud Computing: What it is, and What it is Not. Herbst, Nikolas Roman; Kounev, Samuel; Reussner, Ralf; in Proceedings of the 10th International Conference on Autonomic Computing (ICAC 2013) (2013). USENIX.

  Acceptance Rate (Short Paper): 36.9%
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Originating from the field of physics and economics, the term elasticity is nowadays heavily used in the context of cloud computing. In this context, elasticity is commonly understood as the ability of a system to automatically provision and de-provision computing resources on demand as workloads change. However, elasticity still lacks a precise definition as well as representative metrics coupled with a benchmarking methodology to enable comparability of systems. Existing definitions of elasticity are largely inconsistent and unspecific leading to confusion in the use of the term and its differentiation from related terms such as scalability and efficiency; the proposed measurement methodologies do not provide means to quantify elasticity without mixing it with efficiency or scalability aspects. In this short paper, we propose a precise definition of elasticity and analyze its core properties and requirements explicitly distinguishing from related terms such as scalability, efficiency, and agility. Furthermore, we present a set of appropriate elasticity metrics and sketch a new elasticity tailored benchmarking methodology addressing the special requirements on workload design and calibration.

  @inproceedings{HeKoRe2013-ICAC-Elasticity, abstract = {{Originating from the field of physics and economics, the term elasticity is nowadays heavily used in the context of cloud computing. In this context, elasticity is commonly understood as the ability of a system to automatically provision and de-provision computing resources on demand as workloads change. However, elasticity still lacks a precise definition as well as representative metrics coupled with a benchmarking methodology to enable comparability of systems. Existing definitions of elasticity are largely inconsistent and unspecific leading to confusion in the use of the term and its differentiation from related terms such as scalability and efficiency; the proposed measurement methodologies do not provide means to quantify elasticity without mixing it with efficiency or scalability aspects. In this short paper, we propose a precise definition of elasticity and analyze its core properties and requirements explicitly distinguishing from related terms such as scalability, efficiency, and agility. Furthermore, we present a set of appropriate elasticity metrics and sketch a new elasticity tailored benchmarking methodology addressing the special requirements on workload design and calibration.}}, author = {Herbst, Nikolas Roman and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the 10th International Conference on Autonomic Computing (ICAC 2013)}, keywords = {Self-adaptive-systems}, month = {06}, note = {Acceptance Rate (Short Paper): 36.9\%}, publisher = {USENIX}, title = {{Elasticity in Cloud Computing: What it is, and What it is Not}}, year = 2013 }

  %0 Conference Paper %1 HeKoRe2013-ICAC-Elasticity %A Herbst, Nikolas Roman %A Kounev, Samuel %A Reussner, Ralf %B Proceedings of the 10th International Conference on Autonomic Computing (ICAC 2013) %D 2013 %I USENIX %T Elasticity in Cloud Computing: What it is, and What it is Not %U https://www.usenix.org/conference/icac13/elasticity-cloud-computing-what-it-and-what-it-not %X Originating from the field of physics and economics, the term elasticity is nowadays heavily used in the context of cloud computing. In this context, elasticity is commonly understood as the ability of a system to automatically provision and de-provision computing resources on demand as workloads change. However, elasticity still lacks a precise definition as well as representative metrics coupled with a benchmarking methodology to enable comparability of systems. Existing definitions of elasticity are largely inconsistent and unspecific leading to confusion in the use of the term and its differentiation from related terms such as scalability and efficiency; the proposed measurement methodologies do not provide means to quantify elasticity without mixing it with efficiency or scalability aspects. In this short paper, we propose a precise definition of elasticity and analyze its core properties and requirements explicitly distinguishing from related terms such as scalability, efficiency, and agility. Furthermore, we present a set of appropriate elasticity metrics and sketch a new elasticity tailored benchmarking methodology addressing the special requirements on workload design and calibration.
• 
  Soft Reservations: Uncertainty-Aware Resource Reservations in IaaS Environments. Mohammadi, Seyed Vahid; Kounev, Samuel; Juan-Verdejo, Adriän; Surajbali, Bholanathsingh; in Proceedings of the 3rd International Symposium on Business Modeling and Software Design (BMSD 2013) (2013).
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{MoKoJuBa2013-BMSD-SoftReservations, author = {Mohammadi, Seyed Vahid and Kounev, Samuel and Juan-Verdejo, Adri{\"a}n and Surajbali, Bholanathsingh}, booktitle = {Proceedings of the 3rd International Symposium on Business Modeling and Software Design (BMSD 2013)}, keywords = {Optimization}, title = {{Soft Reservations: Uncertainty-Aware Resource Reservations in IaaS Environments}}, year = 2013 }

  %0 Conference Paper %1 MoKoJuBa2013-BMSD-SoftReservations %A Mohammadi, Seyed Vahid %A Kounev, Samuel %A Juan-Verdejo, Adriän %A Surajbali, Bholanathsingh %B Proceedings of the 3rd International Symposium on Business Modeling and Software Design (BMSD 2013) %D 2013 %T Soft Reservations: Uncertainty-Aware Resource Reservations in IaaS Environments

• 
  Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0. Spinner, Simon; Kounev, Samuel; Meier, Philipp; in Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012), S. Haddad, L. Pomello (eds.) (2012). (Vol. 7347) 388–397. Springer-Verlag, Berlin, Heidelberg.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.

  @inproceedings{SpKoMe2012-PETRINETS-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.}, address = {Berlin, Heidelberg}, author = {Spinner, Simon and Kounev, Samuel and Meier, Philipp}, booktitle = {Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012)}, editor = {Haddad, Serge and Pomello, Lucia}, keywords = {QPN}, month = {06}, pages = {388–397}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0}}, volume = 7347, year = 2012 }

  %0 Conference Paper %1 SpKoMe2012-PETRINETS-QPME %A Spinner, Simon %A Kounev, Samuel %A Meier, Philipp %B Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012) %C Berlin, Heidelberg %D 2012 %E Haddad, Serge %E Pomello, Lucia %I Springer-Verlag %P 388--397 %T Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0 %U http://dx.doi.org/10.1007/978-3-642-31131-4_21 %V 7347 %X Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.
• 
  Architectural Concerns in Multi-Tenant SaaS Applications. Krebs, Rouven; Momm, Christof; Kounev, Samuel; in Proceedings of the 2nd International Conference on Cloud Computing and Services Science (CLOSER 2012) (2012). SciTePress.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{KrMoKo2012-closer-multitenant-saas, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, booktitle = {{Proceedings of the 2nd International Conference on Cloud Computing and Services Science (CLOSER 2012)}}, keywords = {Application_quality_of_service_management}, month = {04}, publisher = {SciTePress}, title = {{Architectural Concerns in Multi-Tenant SaaS Applications}}, year = 2012 }

  %0 Conference Paper %1 KrMoKo2012-closer-multitenant-saas %A Krebs, Rouven %A Momm, Christof %A Kounev, Samuel %B Proceedings of the 2nd International Conference on Cloud Computing and Services Science (CLOSER 2012) %D 2012 %I SciTePress %T Architectural Concerns in Multi-Tenant SaaS Applications

• 
  Towards a Common API for Publish/Subscribe. Pietzuch, Peter; Eyers, David; Kounev, Samuel; Shand, Brian; in Proceedings of the 2007 Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007), Toronto, Canada, June 20-22, 2007, H.-A. Jacobsen, G. Mühl, M. A. Jaeger (eds.) (2007). (Vol. 233) 152–157. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Over the last decade a wide range of publish/subscribe (pub/sub) systems have come out of the research community. However, there is little consensus on a common pub/sub API, which would facilitate innovation, encourage application building, and simplify the evaluation of existing prototypes. Industry pub/sub standards tend to be overly complex, technology-centric, and hard to extend, thus limiting their applicability in research systems. In this paper we propose a common API for pub/sub that is tailored towards research requirements. The API supports three levels of compliance (with optional extensions): the lowest level specifies abstract operations without prescribing an implementation or data model; medium compliance describes interactions using a light-weight XML-RPC mechanism; finally, the highest level of compliance enforces an XML-RPC data model, enabling systems to understand each other's event and subscription semantics. We show that, by following this flexible approach with emphasis on extensibility, our API can be supported by many prototype systems with little effort.

  @inproceedings{PiEyKoSh2007-DEBS-PubSubAPI, abstract = {Over the last decade a wide range of publish/subscribe (pub/sub) systems have come out of the research community. However, there is little consensus on a common pub/sub API, which would facilitate innovation, encourage application building, and simplify the evaluation of existing prototypes. Industry pub/sub standards tend to be overly complex, technology-centric, and hard to extend, thus limiting their applicability in research systems. In this paper we propose a common API for pub/sub that is tailored towards research requirements. The API supports three levels of compliance (with optional extensions): the lowest level specifies abstract operations without prescribing an implementation or data model; medium compliance describes interactions using a light-weight XML-RPC mechanism; finally, the highest level of compliance enforces an XML-RPC data model, enabling systems to understand each other's event and subscription semantics. We show that, by following this flexible approach with emphasis on extensibility, our API can be supported by many prototype systems with little effort.}, author = {Pietzuch, Peter and Eyers, David and Kounev, Samuel and Shand, Brian}, booktitle = {Proceedings of the 2007 Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007), Toronto, Canada, June 20-22, 2007}, editor = {Jacobsen, Hans-Arno and M{\"u}hl, Gero and Jaeger, Michael A.}, keywords = {descartes}, month = {06}, pages = {152–157}, publisher = {ACM, New York, NY, USA}, series = {ACM International Conference Proceeding Series}, title = {{Towards a Common API for Publish/Subscribe}}, volume = 233, year = 2007 }

  %0 Conference Paper %1 PiEyKoSh2007-DEBS-PubSubAPI %A Pietzuch, Peter %A Eyers, David %A Kounev, Samuel %A Shand, Brian %B Proceedings of the 2007 Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007), Toronto, Canada, June 20-22, 2007 %D 2007 %E Jacobsen, Hans-Arno %E Mühl, Gero %E Jaeger, Michael A. %I ACM, New York, NY, USA %P 152--157 %T Towards a Common API for Publish/Subscribe %U http://debs.msrg.utoronto.ca/ %V 233 %X Over the last decade a wide range of publish/subscribe (pub/sub) systems have come out of the research community. However, there is little consensus on a common pub/sub API, which would facilitate innovation, encourage application building, and simplify the evaluation of existing prototypes. Industry pub/sub standards tend to be overly complex, technology-centric, and hard to extend, thus limiting their applicability in research systems. In this paper we propose a common API for pub/sub that is tailored towards research requirements. The API supports three levels of compliance (with optional extensions): the lowest level specifies abstract operations without prescribing an implementation or data model; medium compliance describes interactions using a light-weight XML-RPC mechanism; finally, the highest level of compliance enforces an XML-RPC data model, enabling systems to understand each other's event and subscription semantics. We show that, by following this flexible approach with emphasis on extensibility, our API can be supported by many prototype systems with little effort.

• 
  QPME - Queueing Petri Net Modeling Environment. Kounev, Samuel; Dutz, Christofer; Buchmann, Alejandro; in Proceedings of the 3rd International Conference on Quantitative Evaluation of SysTems (QEST 2006), Riverside, California, USA, September 11-14, 2006 (2006). 115–116. IEEE Computer Society, Washington, DC, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. However, currently available tools for modeling and analysis using queueing Petri nets are very limited in terms of the scalability of the analysis algorithms they provide. Moreover, tools are available only on highly specialized platforms unaccessible to most potential users. In this paper, we present QPME - a Queueing Petri Net Modeling Environment that supports the modeling and analysis of systems using queueing Petri nets. QPME runs on a wide range of platforms and provides a powerful simulation engine that can be used to analyze models of realistically-sized systems.

  @inproceedings{KoDuBu2006-QEST-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. However, currently available tools for modeling and analysis using queueing Petri nets are very limited in terms of the scalability of the analysis algorithms they provide. Moreover, tools are available only on highly specialized platforms unaccessible to most potential users. In this paper, we present QPME - a Queueing Petri Net Modeling Environment that supports the modeling and analysis of systems using queueing Petri nets. QPME runs on a wide range of platforms and provides a powerful simulation engine that can be used to analyze models of realistically-sized systems.}, address = {Washington, DC, USA}, author = {Kounev, Samuel and Dutz, Christofer and Buchmann, Alejandro}, booktitle = {Proceedings of the 3rd International Conference on Quantitative Evaluation of SysTems (QEST 2006), Riverside, California, USA, September 11-14, 2006}, keywords = {QPN}, month = {09}, pages = {115–116}, publisher = {IEEE Computer Society}, title = {{QPME - Queueing Petri Net Modeling Environment}}, year = 2006 }

  %0 Conference Paper %1 KoDuBu2006-QEST-QPME %A Kounev, Samuel %A Dutz, Christofer %A Buchmann, Alejandro %B Proceedings of the 3rd International Conference on Quantitative Evaluation of SysTems (QEST 2006), Riverside, California, USA, September 11-14, 2006 %C Washington, DC, USA %D 2006 %I IEEE Computer Society %P 115--116 %T QPME - Queueing Petri Net Modeling Environment %X Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. However, currently available tools for modeling and analysis using queueing Petri nets are very limited in terms of the scalability of the analysis algorithms they provide. Moreover, tools are available only on highly specialized platforms unaccessible to most potential users. In this paper, we present QPME - a Queueing Petri Net Modeling Environment that supports the modeling and analysis of systems using queueing Petri nets. QPME runs on a wide range of platforms and provides a powerful simulation engine that can be used to analyze models of realistically-sized systems.

• [ 2025 ]
• [ 2024 ]
• [ 2022 ]
• [ 2021 ]
• [ 2020 ]
• [ 2019 ]
• [ 2018 ]
• [ 2017 ]
• [ 2016 ]
• [ 2015 ]
• [ 2014 ]
• [ 2013 ]
• [ 2010 ]
• [ 2009 ]
• [ 2007 ]
• [ 2006 ]

• 
  Microservice Applications and Their Workloads on GitHub. Lubas, Yannik; Straesser, Martin; Rohwer, Ivo; Kounev, Samuel; Bauer, André; in Companion of the 16th ACM/SPEC International Conference on Performance Engineering (2025). 199–205. Association for Computing Machinery, Toronto ON, Canada.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Many cloud applications employ a microservice architecture to reap its various benefits, such as independent scalability and development. However, microservices' distributed nature comes with its own challenges, which the literature tries to combat with various approaches in domains, such as autoscaling or service placement. Researchers, especially performance engineers, need representative microservice applications as a basis to develop and evaluate their approaches. While the literature proposes several reference and benchmark applications, the set of commonly used applications is rather limited. Further, the representativeness of reference applications for industry practices has been challenged, raising questions about the applicability of existing approaches. To investigate possible alternatives to and inspirations for microservice testbeds, we conduct an exploratory search of microservice applications and their workloads by mining GitHub repositories. Our work provides two datasets containing 553 applications and 8 workload dataset repositories, respectively. In addition, we provide a first analysis of the collected data that future work can build upon.

  @inproceedings{10.1145/3680256.3721321, abstract = {Many cloud applications employ a microservice architecture to reap its various benefits, such as independent scalability and development. However, microservices' distributed nature comes with its own challenges, which the literature tries to combat with various approaches in domains, such as autoscaling or service placement. Researchers, especially performance engineers, need representative microservice applications as a basis to develop and evaluate their approaches. While the literature proposes several reference and benchmark applications, the set of commonly used applications is rather limited. Further, the representativeness of reference applications for industry practices has been challenged, raising questions about the applicability of existing approaches. To investigate possible alternatives to and inspirations for microservice testbeds, we conduct an exploratory search of microservice applications and their workloads by mining GitHub repositories. Our work provides two datasets containing 553 applications and 8 workload dataset repositories, respectively. In addition, we provide a first analysis of the collected data that future work can build upon.}, address = {New York, NY, USA}, author = {Lubas, Yannik and Straesser, Martin and Rohwer, Ivo and Kounev, Samuel and Bauer, Andr\'{e}}, booktitle = {Companion of the 16th ACM/SPEC International Conference on Performance Engineering}, keywords = {t_workshop}, pages = {199–205}, publisher = {Association for Computing Machinery}, series = {ICPE '25}, title = {Microservice Applications and Their Workloads on GitHub}, year = 2025 }

  %0 Conference Paper %1 10.1145/3680256.3721321 %A Lubas, Yannik %A Straesser, Martin %A Rohwer, Ivo %A Kounev, Samuel %A Bauer, André %B Companion of the 16th ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2025 %I Association for Computing Machinery %P 199–205 %R 10.1145/3680256.3721321 %T Microservice Applications and Their Workloads on GitHub %U https://doi.org/10.1145/3680256.3721321 %X Many cloud applications employ a microservice architecture to reap its various benefits, such as independent scalability and development. However, microservices' distributed nature comes with its own challenges, which the literature tries to combat with various approaches in domains, such as autoscaling or service placement. Researchers, especially performance engineers, need representative microservice applications as a basis to develop and evaluate their approaches. While the literature proposes several reference and benchmark applications, the set of commonly used applications is rather limited. Further, the representativeness of reference applications for industry practices has been challenged, raising questions about the applicability of existing approaches. To investigate possible alternatives to and inspirations for microservice testbeds, we conduct an exploratory search of microservice applications and their workloads by mining GitHub repositories. Our work provides two datasets containing 553 applications and 8 workload dataset repositories, respectively. In addition, we provide a first analysis of the collected data that future work can build upon. %@ 9798400711305

• 
  An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times. Straesser, Martin; Erhard, Nicholas; Kounev, Samuel; (2024). Gesellschaft für Informatik e.V.

  In print.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @misc{StraesserSSP2024, author = {Straesser, Martin and Erhard, Nicholas and Kounev, Samuel}, booktitle = {Softwaretechnik-Trends Band 44, Heft 4}, keywords = {t_workshop}, note = {In print.}, publisher = {Gesellschaft für Informatik e.V.}, series = {Softwaretechnik-Trends}, title = {An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times}, year = 2024 }

  %0 Generic %1 StraesserSSP2024 %A Straesser, Martin %A Erhard, Nicholas %A Kounev, Samuel %B Softwaretechnik-Trends Band 44, Heft 4 %D 2024 %I Gesellschaft für Informatik e.V. %T An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times

• 
  Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency. Meissner, Maximilian; Kamthania, Supriya; Rawtani, Nishant; Bucek, James; Lange, Klaus-Dieter; Kounev, Samuel; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022). ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{meissner2022experience, address = {New York, NY, USA}, author = {Meissner, Maximilian and Kamthania, Supriya and Rawtani, Nishant and Bucek, James and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {t_workshop}, month = {04}, publisher = {ACM}, title = {Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency}, year = 2022 }

  %0 Conference Paper %1 meissner2022experience %A Meissner, Maximilian %A Kamthania, Supriya %A Rawtani, Nishant %A Bucek, James %A Lange, Klaus-Dieter %A Kounev, Samuel %B Companion of the 2022 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2022 %I ACM %T Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency
• 
  An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case. Prantl, Thomas; Engel, Simon; Bauer, Andre; Ben Yahya, Ala Eddine; Herrnleben, Stefan; Iffländer, Lukas; Dmitrienko, Alexandra; Kounev, Samuel; in 2022 IEEE 95th Vehicular Technology Conference (VTC2022) (2022). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrEnBaAlHeIfDmKo2022-IoT/CPS-DistributedGroupEncryption, author = {Prantl, Thomas and Engel, Simon and Bauer, Andre and Ben Yahya, Ala Eddine and Herrnleben, Stefan and Iffländer, Lukas and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {2022 IEEE 95th Vehicular Technology Conference (VTC2022)}, keywords = {t_workshop}, organization = {IEEE}, title = {An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case}, year = 2022 }

  %0 Conference Paper %1 PrEnBaAlHeIfDmKo2022-IoT/CPS-DistributedGroupEncryption %A Prantl, Thomas %A Engel, Simon %A Bauer, Andre %A Ben Yahya, Ala Eddine %A Herrnleben, Stefan %A Iffländer, Lukas %A Dmitrienko, Alexandra %A Kounev, Samuel %B 2022 IEEE 95th Vehicular Technology Conference (VTC2022) %D 2022 %T An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case

• 
  Utility-based Vehicle Routing Integrating User Preferences. Lesch, Veronika; Hadry, Marius; Kounev, Samuel; Krupitzer, Christian; in Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021 (2021). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{lesch2021utility, author = {Lesch, Veronika and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021}, keywords = {t_workshop}, month = {03}, publisher = {IEEE}, title = {Utility-based Vehicle Routing Integrating User Preferences}, year = 2021 }

  %0 Conference Paper %1 lesch2021utility %A Lesch, Veronika %A Hadry, Marius %A Kounev, Samuel %A Krupitzer, Christian %B Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021 %D 2021 %I IEEE %T Utility-based Vehicle Routing Integrating User Preferences
• 
  Performance Evaluation of a Post-Quantum Public-Key Cryptosystem. Prantl, Thomas; Prantl, Dominik; Beierlieb, Lukas; Iffländer, Lukas; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021). IEEE.
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem, author = {Prantl, Thomas and Prantl, Dominik and Beierlieb, Lukas and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {t_workshop}, organization = {IEEE}, title = {Performance Evaluation of a Post-Quantum Public-Key Cryptosystem}, year = 2021 }

  %0 Conference Paper %1 PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem %A Prantl, Thomas %A Prantl, Dominik %A Beierlieb, Lukas %A Iffländer, Lukas %A Dmitrienko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) %D 2021 %T Performance Evaluation of a Post-Quantum Public-Key Cryptosystem

• 
  A Concept for Crowd-sensed Prediction of Mobile Network Connectivity. Herrnleben, Stefan; Zeidler, Bernd; Züfle, Marwin; Krupitzer, Christian; Kounev, Samuel; in GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020 (2020).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{HeZeZuKrKo-FGMLN2020-Prediction-of-Mobile-Network-Connectivity, author = {Herrnleben, Stefan and Zeidler, Bernd and Z{\"u}fle, Marwin and Krupitzer, Christian and Kounev, Samuel}, booktitle = {GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020}, keywords = {t_workshop}, month = {02}, title = {{A Concept for Crowd-sensed Prediction of Mobile Network Connectivity}}, year = 2020 }

  %0 Conference Paper %1 HeZeZuKrKo-FGMLN2020-Prediction-of-Mobile-Network-Connectivity %A Herrnleben, Stefan %A Zeidler, Bernd %A Züfle, Marwin %A Krupitzer, Christian %A Kounev, Samuel %B GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020 %D 2020 %T A Concept for Crowd-sensed Prediction of Mobile Network Connectivity
• 
  SIMPL: Secure IoT Management Platform. Prantl, Thomas; Ben Yahya, Ala Eddine; Dmitrienko, Alexandra; Kounev, Samuel; Lipp, Fabian; Hock, David; Rathfelder, Christoph; Hofherr, Martin; in ITG Workshop on IT Security (ITSec) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL

  @inproceedings{PrYaDmKoLiHoRaHo2020-ITSec-SIMPL, abstract = {The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL}, author = {Prantl, Thomas and Ben Yahya, Ala Eddine and Dmitrienko, Alexandra and Kounev, Samuel and Lipp, Fabian and Hock, David and Rathfelder, Christoph and Hofherr, Martin}, booktitle = {ITG Workshop on IT Security (ITSec)}, keywords = {t_workshop}, title = {SIMPL: Secure IoT Management Platform}, year = 2020 }

  %0 Conference Paper %1 PrYaDmKoLiHoRaHo2020-ITSec-SIMPL %A Prantl, Thomas %A Ben Yahya, Ala Eddine %A Dmitrienko, Alexandra %A Kounev, Samuel %A Lipp, Fabian %A Hock, David %A Rathfelder, Christoph %A Hofherr, Martin %B ITG Workshop on IT Security (ITSec) %D 2020 %T SIMPL: Secure IoT Management Platform %X The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL

• 
  Towards Adaptive Car-to-Cloud Communication. Herrnleben, Stefan; Pfannemüller, Martin; Krupitzer, Christian; Kounev, Samuel; Segata, Michele; Fastnacht, Felix; Nigmann, Magnus; in Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (2019). Kyoto, Japan.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Recent developments in communication technology have led to cloud resources becoming ubiquitous. These resources enable many new applications by offering computational power for remote embedded devices. In combination with advances in the area of smart driving, this seems to be especially beneficial for applications such as remote maintenance of vehicles or integration with smart city services. As autonomous driving continues to gain traction, Car-to-Cloud communication can support transferring collected data to the cloud, e.g., for dynamic learning of new map information. Additionally, passengers can benefit from novel entertainment services. All these developments require a stable connection between a mobile vehicle and the cloud resources. In this vision paper, we survey Car-to-Cloud communication applications. Based on the analysis of the varying requirements for these applications, we formulate research questions and challenges. Further, we discuss how these challenges can be addressed by means of an adaptive Car-to-Cloud communication middleware. We conclude with an overview on our activities in this area and an outlook on our planned future work on adaptive communication.

  @inproceedings{HePfKrKoSeFaNi2019-PerAwareCity-carToCloudCommunication, abstract = {Recent developments in communication technology have led to cloud resources becoming ubiquitous. These resources enable many new applications by offering computational power for remote embedded devices. In combination with advances in the area of smart driving, this seems to be especially beneficial for applications such as remote maintenance of vehicles or integration with smart city services. As autonomous driving continues to gain traction, Car-to-Cloud communication can support transferring collected data to the cloud, e.g., for dynamic learning of new map information. Additionally, passengers can benefit from novel entertainment services. All these developments require a stable connection between a mobile vehicle and the cloud resources. In this vision paper, we survey Car-to-Cloud communication applications. Based on the analysis of the varying requirements for these applications, we formulate research questions and challenges. Further, we discuss how these challenges can be addressed by means of an adaptive Car-to-Cloud communication middleware. We conclude with an overview on our activities in this area and an outlook on our planned future work on adaptive communication.}, address = {Kyoto, Japan}, author = {Herrnleben, Stefan and Pfannem{\"u}ller, Martin and Krupitzer, Christian and Kounev, Samuel and Segata, Michele and Fastnacht, Felix and Nigmann, Magnus}, booktitle = {Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)}, keywords = {Intelligent_Transportation_Systems}, title = {Towards Adaptive {Car-to-Cloud} Communication}, year = 2019 }

  %0 Conference Paper %1 HePfKrKoSeFaNi2019-PerAwareCity-carToCloudCommunication %A Herrnleben, Stefan %A Pfannemüller, Martin %A Krupitzer, Christian %A Kounev, Samuel %A Segata, Michele %A Fastnacht, Felix %A Nigmann, Magnus %B Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) %C Kyoto, Japan %D 2019 %T Towards Adaptive Car-to-Cloud Communication %X Recent developments in communication technology have led to cloud resources becoming ubiquitous. These resources enable many new applications by offering computational power for remote embedded devices. In combination with advances in the area of smart driving, this seems to be especially beneficial for applications such as remote maintenance of vehicles or integration with smart city services. As autonomous driving continues to gain traction, Car-to-Cloud communication can support transferring collected data to the cloud, e.g., for dynamic learning of new map information. Additionally, passengers can benefit from novel entertainment services. All these developments require a stable connection between a mobile vehicle and the cloud resources. In this vision paper, we survey Car-to-Cloud communication applications. Based on the analysis of the varying requirements for these applications, we formulate research questions and challenges. Further, we discuss how these challenges can be addressed by means of an adaptive Car-to-Cloud communication middleware. We conclude with an overview on our activities in this area and an outlook on our planned future work on adaptive communication.
• 
  Systematic Search for Optimal Resource Configurations of Distributed Applications. Bauer, André; Eismann, Simon; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019). 120–125. IEEE Computer Society, Los Alamitos, CA, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.

  @inproceedings{BaEiGrHeKo-ICAC-BUNGEE, abstract = {With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.}, address = {Los Alamitos, CA, USA}, author = {Bauer, Andr{\'e} and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}, keywords = {Optimization}, month = {06}, pages = {120–125}, publisher = {IEEE Computer Society}, title = {{Systematic Search for Optimal Resource Configurations of Distributed Applications}}, year = 2019 }

  %0 Conference Paper %1 BaEiGrHeKo-ICAC-BUNGEE %A Bauer, André %A Eismann, Simon %A Grohmann, Johannes %A Herbst, Nikolas %A Kounev, Samuel %B 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) %C Los Alamitos, CA, USA %D 2019 %I IEEE Computer Society %P 120--125 %R 10.1109/FAS-W.2019.00040 %T Systematic Search for Optimal Resource Configurations of Distributed Applications %U https://doi.ieeecomputersociety.org/10.1109/FAS-W.2019.00040 %X With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.
• 
  On Learning Parametric Dependencies from Monitoring Data. Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in Proceedings of the 10th Symposium on Software Performance 2019 (SSP’19) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  A common approach to predict system performance are so-called architectural performance models. In these models, parametric dependencies describe the relation between the input parameters of a component and its performance properties and therefore significantly increase the model expressiveness. However, manually modeling parametric dependencies is often infeasible in practice. Existing automated extraction approaches require either application source code or dedicated performance tests, which are not always available. We therefore introduced one approach for identification and one for characterization of parametric dependencies, solely based on run-time monitoring data. In this paper, we propose our idea on combining both techniques in order to create a holistic approach for the identification and characterization of parametric dependencies. Furthermore, we discuss challenges we are currently facing and potential ideas on how to overcome them.

  @inproceedings{GrEiKo2019-SSP-LearningDependencies, abstract = {A common approach to predict system performance are so-called architectural performance models. In these models, parametric dependencies describe the relation between the input parameters of a component and its performance properties and therefore significantly increase the model expressiveness. However, manually modeling parametric dependencies is often infeasible in practice. Existing automated extraction approaches require either application source code or dedicated performance tests, which are not always available. We therefore introduced one approach for identification and one for characterization of parametric dependencies, solely based on run-time monitoring data. In this paper, we propose our idea on combining both techniques in order to create a holistic approach for the identification and characterization of parametric dependencies. Furthermore, we discuss challenges we are currently facing and potential ideas on how to overcome them.}, author = {Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)}, keywords = {t_workshop}, month = 11, title = {{On Learning Parametric Dependencies from Monitoring Data}}, year = 2019 }

  %0 Conference Paper %1 GrEiKo2019-SSP-LearningDependencies %A Grohmann, Johannes %A Eismann, Simon %A Kounev, Samuel %B Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19) %D 2019 %T On Learning Parametric Dependencies from Monitoring Data %U https://pi.informatik.uni-siegen.de/stt/39_4/01_Fachgruppenberichte/SSP2019/SSP2019_Grohmann.pdf %X A common approach to predict system performance are so-called architectural performance models. In these models, parametric dependencies describe the relation between the input parameters of a component and its performance properties and therefore significantly increase the model expressiveness. However, manually modeling parametric dependencies is often infeasible in practice. Existing automated extraction approaches require either application source code or dedicated performance tests, which are not always available. We therefore introduced one approach for identification and one for characterization of parametric dependencies, solely based on run-time monitoring data. In this paper, we propose our idea on combining both techniques in order to create a holistic approach for the identification and characterization of parametric dependencies. Furthermore, we discuss challenges we are currently facing and potential ideas on how to overcome them.
• 
  Utilizing Clustering to Optimize Resource Demand Estimation Approaches. Grohmann, Johannes; Eismann, Simon; Bauer, Andre; Zuefle, Marwin; Herbst, Nikolas; Kounev, Samuel; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019). 134–139. IEEE.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.

  @inproceedings{GrEiBaZuHeKo2019-ICAC-RDEClustering, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andre and Zuefle, Marwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}, keywords = {Optimization}, month = {06}, pages = {134–139}, publisher = {IEEE}, title = {{Utilizing Clustering to Optimize Resource Demand Estimation Approaches}}, year = 2019 }

  %0 Conference Paper %1 GrEiBaZuHeKo2019-ICAC-RDEClustering %A Grohmann, Johannes %A Eismann, Simon %A Bauer, Andre %A Zuefle, Marwin %A Herbst, Nikolas %A Kounev, Samuel %B 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) %D 2019 %I IEEE %P 134--139 %R 10.1109/FAS-W.2019.00043 %T Utilizing Clustering to Optimize Resource Demand Estimation Approaches %U https://doi.org/10.1109/FAS-W.2019.00043 %X Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.
• 
  Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior. Beierlieb, Lukas; Iffländer, Lukas; Kounev, Samuel; Milenkoski, Aleksandar; in Proceedings of the 10th Symposium on Software Performance 2019 (SSP’19) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.

  @inproceedings{Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior, abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.}, author = {Beierlieb, Lukas and Iffländer, Lukas and Kounev, Samuel and Milenkoski, Aleksandar}, booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)}, keywords = {t_workshop}, month = 11, title = {Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior}, year = 2019 }

  %0 Conference Paper %1 Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior %A Beierlieb, Lukas %A Iffländer, Lukas %A Kounev, Samuel %A Milenkoski, Aleksandar %B Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19) %D 2019 %T Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior %X With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.
• 
  Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces. Beierlieb, Lukas; Iffländer, Lukas; Milenkoski, Aleksandar; Goncalves, Charles F.; Antunes, Nuno; Kounev, Samuel; in 2019 IEEE International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2019). IEEE.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

  @inproceedings{Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces, abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.}, author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Goncalves, Charles F. and Antunes, Nuno and Kounev, Samuel}, booktitle = {2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})}, keywords = {t_workshop}, month = 11, organization = {{IEEE}}, title = {Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces}, year = 2019 }

  %0 Conference Paper %1 Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces %A Beierlieb, Lukas %A Iffländer, Lukas %A Milenkoski, Aleksandar %A Goncalves, Charles F. %A Antunes, Nuno %A Kounev, Samuel %B 2019 IEEE International Symposium on Software Reliability Engineering Workshops ({ISSREW}) %D 2019 %T Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces %X With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

• 
  Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking. Iffländer, Lukas; Geißler, Stefan; Walter, Jürgen; Beierlieb, Lukas; Kounev, Samuel; in Proceedings of the 9th Symposium on Software Performance 2018 (SSP’18) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore’s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

  @inproceedings{Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking, abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore’s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.}, author = {Iffländer, Lukas and Geißler, Stefan and Walter, Jürgen and Beierlieb, Lukas and Kounev, Samuel}, booktitle = {Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18)}, keywords = {t_workshop}, month = 11, title = {Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking}, year = 2018 }

  %0 Conference Paper %1 Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking %A Iffländer, Lukas %A Geißler, Stefan %A Walter, Jürgen %A Beierlieb, Lukas %A Kounev, Samuel %B Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18) %D 2018 %T Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking %X Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore’s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.
• 
  Black-box Learning of Parametric Dependencies for Performance Models. Ackermann, Vanessa; Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.

  @inproceedings{AcGrEiKo2018-MRT-DependencyModeling, abstract = {Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.}, author = {Ackermann, Vanessa and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018)}, keywords = {Optimization}, month = 10, series = {CEUR Workshop Proceedings}, title = {Black-box Learning of Parametric Dependencies for Performance Models}, year = 2018 }

  %0 Conference Paper %1 AcGrEiKo2018-MRT-DependencyModeling %A Ackermann, Vanessa %A Grohmann, Johannes %A Eismann, Simon %A Kounev, Samuel %B Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018) %D 2018 %T Black-box Learning of Parametric Dependencies for Performance Models %U http://ceur-ws.org/Vol-2245/mrt_paper_5.pdf %X Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.

• 
  Providing Model-Extraction-as-a-Service for Architectural Performance Models. Walter, Jürgen; Eismann, Simon; Reed, Nikolai; Kounev, Samuel; in Proceedings of the 2017 Symposium on Software Performance (SSP) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.

  @inproceedings{WaEiReKo2017-SPP-MEaaS, abstract = {Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.}, author = {Walter, J{\"u}rgen and Eismann, Simon and Reed, Nikolai and Kounev, Samuel}, booktitle = {Proceedings of the 2017 Symposium on Software Performance (SSP)}, keywords = {t_workshop}, month = 11, title = {{Providing Model-Extraction-as-a-Service for Architectural Performance Models}}, year = 2017 }

  %0 Conference Paper %1 WaEiReKo2017-SPP-MEaaS %A Walter, Jürgen %A Eismann, Simon %A Reed, Nikolai %A Kounev, Samuel %B Proceedings of the 2017 Symposium on Software Performance (SSP) %D 2017 %T Providing Model-Extraction-as-a-Service for Architectural Performance Models %X Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.
• 
  Mapping of Service Level Objectives to Performance Queries. Walter, Jürgen; Okanovic, Dusan; Kounev, Samuel; in Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) (2017). ACM.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.

  @inproceedings{WaOkKo2017-WOSPC-SLO2Queries, abstract = {The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.}, author = {Walter, J{\"u}rgen and Okanovic, Dusan and Kounev, Samuel}, booktitle = {{Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}}, keywords = {DQL}, month = {04}, publisher = {ACM}, title = {{Mapping of Service Level Objectives to Performance Queries}}, year = 2017 }

  %0 Conference Paper %1 WaOkKo2017-WOSPC-SLO2Queries %A Walter, Jürgen %A Okanovic, Dusan %A Kounev, Samuel %B Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) %D 2017 %I ACM %T Mapping of Service Level Objectives to Performance Queries %X The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.
• 
  An Expandable Extraction Framework for Architectural Performance Models. Walter, Jürgen; Stier, Christian; Koziolek, Heiko; Kounev, Samuel; in Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS’17) (2017). ACM.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.

  @inproceedings{WaStKoKo2017-QUDOS-PMXBuilder, abstract = {Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.}, author = {Walter, J{\"u}rgen and Stier, Christian and Koziolek, Heiko and Kounev, Samuel}, booktitle = {{Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17)}}, keywords = {t_workshop}, month = {04}, publisher = {ACM}, title = {{An Expandable Extraction Framework for Architectural Performance Models}}, year = 2017 }

  %0 Conference Paper %1 WaStKoKo2017-QUDOS-PMXBuilder %A Walter, Jürgen %A Stier, Christian %A Koziolek, Heiko %A Kounev, Samuel %B Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17) %D 2017 %I ACM %T An Expandable Extraction Framework for Architectural Performance Models %X Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.

• 
  Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation. Müller, Christoph; Rygielski, Piotr; Spinner, Simon; Kounev, Samuel; in Electronic Notes in Theoretical Computer Science (2016). 327 71–91. Elsevier.

  The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Abstract Due to the growing size of modern IT} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN} models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

  @article{MuRySpKo2016-PASM-QPN-LQN-Transformation, abstract = {Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.}, author = {M{\"u}ller, Christoph and Rygielski, Piotr and Spinner, Simon and Kounev, Samuel}, journal = {Electronic Notes in Theoretical Computer Science}, keywords = {QPN}, note = {The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016}, pages = {71–91}, publisher = {Elsevier}, title = {{Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}}, volume = 327, year = 2016 }

  %0 Journal Article %1 MuRySpKo2016-PASM-QPN-LQN-Transformation %A Müller, Christoph %A Rygielski, Piotr %A Spinner, Simon %A Kounev, Samuel %D 2016 %I Elsevier %J Electronic Notes in Theoretical Computer Science %P 71--91 %T Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation %U http://www.sciencedirect.com/science/article/pii/S157106611630069X %V 327 %X Abstract Due to the growing size of modern IT} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN} models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.
• 
  A Reference Architecture for Online Performance Model Extraction in Virtualized Environments. Spinner, Simon; Walter, Jürgen; Kounev, Samuel; in Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.

  @inproceedings{SpWaKo2015-WOSPC-ReferenceArchitecture, abstract = {Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.}, author = {Spinner, Simon and Walter, J{\"u}rgen and Kounev, Samuel}, booktitle = {{Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}}, keywords = {t_workshop}, month = {03}, title = {{A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}}, year = 2016 }

  %0 Conference Paper %1 SpWaKo2015-WOSPC-ReferenceArchitecture %A Spinner, Simon %A Walter, Jürgen %A Kounev, Samuel %B Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) %D 2016 %T A Reference Architecture for Online Performance Model Extraction in Virtualized Environments %X Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.
• 
  PAVO: A Framework for the Visualization of Performance Analyses Results. Walter, Jürgen; König, Maximilian; Eismann, Simon; Kounev, Samuel; in Proceedings of the 2016 Symposium on Software Performance (SSP) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).

  @inproceedings{WaKoEiKo2016-SPP-PAVO, abstract = {Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).}, author = {Walter, J{\"u}rgen and K{\"o}nig, Maximilian and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 2016 Symposium on Software Performance (SSP)}, keywords = {DQL}, month = 11, title = {{ PAVO: A Framework for the Visualization of Performance Analyses Results}}, year = 2016 }

  %0 Conference Paper %1 WaKoEiKo2016-SPP-PAVO %A Walter, Jürgen %A König, Maximilian %A Eismann, Simon %A Kounev, Samuel %B Proceedings of the 2016 Symposium on Software Performance (SSP) %D 2016 %T PAVO: A Framework for the Visualization of Performance Analyses Results %X Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).
• 
  Predicting Power Consumption in Virtualized Environments. von Kistowski, Joakim; Schreck, Marco; Kounev, Samuel; in Computer Performance Engineering: 13th European Workshop, EPEW 2016, Chios, Greece, October 5-7, 2016, Proceedings, D. Fiems, M. Paolieri, N. A. Platis (eds.) (2016). 79–93. Springer International Publishing, Cham.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  Energy efficiency and power consumption of data centers can be improved through conscientious placement of workloads on specific servers. Virtualization is commonly employed nowadays, as it allows for dynamic reallocation of work and abstraction from the concrete server hardware. The ability to predict the power consumption of workloads at different load levels is essential in this context. Prediction approaches can help to make better placement choices at run-time, as well as when purchasing new server equipment, by showing which servers are better suited for the execution of a given target workload. In existing work, power prediction for servers is limited to non-virtualized contexts or it does not take multiple load levels into account. Existing approaches also fail to leverage publicly available data on server efficiency and instead require experiments to be conducted on the target system. This makes these approaches unwieldy when making decision regarding systems that are not yet available to the decision maker. In this paper, we use the readily available data provided by the SPEC SERT to predict the power consumption of workloads for different load levels in virtualized environments. We evaluate our approach comparing predicted results against measurements of power consumption in multiple virtualized environment configurations on a target server that differs significantly from the reference system used for experimentation. We show that power consumption of CPU and storage loads can be reliably predicted with a prediction error of less than 15% across all tested virtualized environment configurations.

  @inproceedings{KiScKo-EPEW-VMPower, abstract = {Energy efficiency and power consumption of data centers can be improved through conscientious placement of workloads on specific servers. Virtualization is commonly employed nowadays, as it allows for dynamic reallocation of work and abstraction from the concrete server hardware. The ability to predict the power consumption of workloads at different load levels is essential in this context. Prediction approaches can help to make better placement choices at run-time, as well as when purchasing new server equipment, by showing which servers are better suited for the execution of a given target workload. In existing work, power prediction for servers is limited to non-virtualized contexts or it does not take multiple load levels into account. Existing approaches also fail to leverage publicly available data on server efficiency and instead require experiments to be conducted on the target system. This makes these approaches unwieldy when making decision regarding systems that are not yet available to the decision maker. In this paper, we use the readily available data provided by the SPEC SERT to predict the power consumption of workloads for different load levels in virtualized environments. We evaluate our approach comparing predicted results against measurements of power consumption in multiple virtualized environment configurations on a target server that differs significantly from the reference system used for experimentation. We show that power consumption of CPU and storage loads can be reliably predicted with a prediction error of less than 15% across all tested virtualized environment configurations.}, address = {Cham}, author = {von Kistowski, Joakim and Schreck, Marco and Kounev, Samuel}, booktitle = {Computer Performance Engineering: 13th European Workshop, EPEW 2016, Chios, Greece, October 5-7, 2016, Proceedings}, editor = {Fiems, Dieter and Paolieri, Marco and Platis, N. Agapios}, keywords = {Power}, month = {{October}}, pages = {79–93}, publisher = {Springer International Publishing}, title = {Predicting Power Consumption in Virtualized Environments}, year = 2016 }

  %0 Conference Paper %1 KiScKo-EPEW-VMPower %A von Kistowski, Joakim %A Schreck, Marco %A Kounev, Samuel %B Computer Performance Engineering: 13th European Workshop, EPEW 2016, Chios, Greece, October 5-7, 2016, Proceedings %C Cham %D 2016 %E Fiems, Dieter %E Paolieri, Marco %E Platis, N. Agapios %I Springer International Publishing %P 79--93 %T Predicting Power Consumption in Virtualized Environments %U http://dx.doi.org/10.1007/978-3-319-46433-6_6 %X Energy efficiency and power consumption of data centers can be improved through conscientious placement of workloads on specific servers. Virtualization is commonly employed nowadays, as it allows for dynamic reallocation of work and abstraction from the concrete server hardware. The ability to predict the power consumption of workloads at different load levels is essential in this context. Prediction approaches can help to make better placement choices at run-time, as well as when purchasing new server equipment, by showing which servers are better suited for the execution of a given target workload. In existing work, power prediction for servers is limited to non-virtualized contexts or it does not take multiple load levels into account. Existing approaches also fail to leverage publicly available data on server efficiency and instead require experiments to be conducted on the target system. This makes these approaches unwieldy when making decision regarding systems that are not yet available to the decision maker. In this paper, we use the readily available data provided by the SPEC SERT to predict the power consumption of workloads for different load levels in virtualized environments. We evaluate our approach comparing predicted results against measurements of power consumption in multiple virtualized environment configurations on a target server that differs significantly from the reference system used for experimentation. We show that power consumption of CPU and storage loads can be reliably predicted with a prediction error of less than 15% across all tested virtualized environment configurations.

• 
  Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques. Willnecker, Felix; Dlugi, Markus; Brunnert, Andreas; Spinner, Simon; Kounev, Samuel; Krcmar, Helmut; in Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015), M. Beltrán, W. Knottenbelt, J. Bradley (eds.) (2015). (Vol. 9272) 115–129. Springer.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.

  @inproceedings{WiDlBrSpKoKr2015-EPEW-LibredePMWT, abstract = {Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.}, author = {Willnecker, Felix and Dlugi, Markus and Brunnert, Andreas and Spinner, Simon and Kounev, Samuel and Krcmar, Helmut}, booktitle = {Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015)}, editor = {Beltr{\'a}n, Marta and Knottenbelt, William and Bradley, Jeremy}, keywords = {t_workshop}, month = {{August}}, pages = {115-129}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques}}, volume = 9272, year = 2015 }

  %0 Conference Paper %1 WiDlBrSpKoKr2015-EPEW-LibredePMWT %A Willnecker, Felix %A Dlugi, Markus %A Brunnert, Andreas %A Spinner, Simon %A Kounev, Samuel %A Krcmar, Helmut %B Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015) %D 2015 %E Beltrán, Marta %E Knottenbelt, William %E Bradley, Jeremy %I Springer %P 115-129 %T Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques %V 9272 %X Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.

• 
  Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems. Noorshams, Qais; Busch, Axel; Rentschler, Andreas; Bruhn, Dominik; Kounev, Samuel; Tuma, Petr; Reussner, Ralf; in 34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, DCPerf ’14 (2014). 88–93.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoBuReBrKoTuRe2014-DCPerf-IORegressionModels, author = {Noorshams, Qais and Busch, Axel and Rentschler, Andreas and Bruhn, Dominik and Kounev, Samuel and T{u}ma, Petr and Reussner, Ralf}, booktitle = {34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, DCPerf '14}, keywords = {t_workshop}, month = {07}, pages = {88-93}, title = {{Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems}}, year = 2014 }

  %0 Conference Paper %1 NoBuReBrKoTuRe2014-DCPerf-IORegressionModels %A Noorshams, Qais %A Busch, Axel %A Rentschler, Andreas %A Bruhn, Dominik %A Kounev, Samuel %A Tuma, Petr %A Reussner, Ralf %B 34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, DCPerf '14 %D 2014 %P 88-93 %T Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems %U http://dx.doi.org/10.1109/ICDCSW.2014.26
• 
  Data Center Network Throughput Analysis using Queueing Petri Nets. Rygielski, Piotr; Kounev, Samuel; in 34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, (DCPerf 2014) (2014). 100–105.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  In this paper, we address the problem of performance analysis in computer networks. We present a metamodel designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. In this paper, we present the transformation to Queueing Petri Nets (QPNs) and simulate them using SimQPN simulator. We show that despite the high level of abstraction of the QPNs, we are able to estimate the utilization of resources with good accuracy within a short time.

  @inproceedings{RyKo2014-DCPerf-DNI2QPN, abstract = {In this paper, we address the problem of performance analysis in computer networks. We present a metamodel designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. In this paper, we present the transformation to Queueing Petri Nets (QPNs) and simulate them using SimQPN simulator. We show that despite the high level of abstraction of the QPNs, we are able to estimate the utilization of resources with good accuracy within a short time.}, author = {Rygielski, Piotr and Kounev, Samuel}, booktitle = {34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, (DCPerf 2014)}, keywords = {DNI}, month = {06}, pages = {100–105}, title = {{Data Center Network Throughput Analysis using Queueing Petri Nets}}, year = 2014 }

  %0 Conference Paper %1 RyKo2014-DCPerf-DNI2QPN %A Rygielski, Piotr %A Kounev, Samuel %B 34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, (DCPerf 2014) %D 2014 %P 100--105 %T Data Center Network Throughput Analysis using Queueing Petri Nets %X In this paper, we address the problem of performance analysis in computer networks. We present a metamodel designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. In this paper, we present the transformation to Queueing Petri Nets (QPNs) and simulate them using SimQPN simulator. We show that despite the high level of abstraction of the QPNs, we are able to estimate the utilization of resources with good accuracy within a short time.
• 
  Modeling Variations in Load Intensity over Time. von Kistowski, Jóakim; Herbst, Nikolas Roman; Kounev, Samuel; in Proceedings of the 3rd International Workshop on Large-Scale Testing (LT 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 1–4. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Today's software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. Based on this observation, we identify the need for means allowing flexible definition of load profiles and address this by introducing two meta-models at different abstraction levels. At the lower abstraction level, the Descartes Load Intensity Meta-Model (DLIM) offers a structured and accessible way of describing the load intensity over time by editing and combining mathematical functions. The High-Level Descartes Load Intensity Meta-Model (HLDLIM) allows the description of load variations using few defined parameters that characterize the seasonal patterns, trends, bursts and noise parts. We demonstrate that both meta-models are capable of capturing real-world load profiles with acceptable accuracy through comparison with a real life trace.

  @inproceedings{KiHeKo2014-LT-DLIM, abstract = {{Today's software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. Based on this observation, we identify the need for means allowing flexible definition of load profiles and address this by introducing two meta-models at different abstraction levels. At the lower abstraction level, the Descartes Load Intensity Meta-Model (DLIM) offers a structured and accessible way of describing the load intensity over time by editing and combining mathematical functions. The High-Level Descartes Load Intensity Meta-Model (HLDLIM) allows the description of load variations using few defined parameters that characterize the seasonal patterns, trends, bursts and noise parts. We demonstrate that both meta-models are capable of capturing real-world load profiles with acceptable accuracy through comparison with a real life trace.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas Roman and Kounev, Samuel}, booktitle = {Proceedings of the 3rd International Workshop on Large-Scale Testing (LT 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {t_workshop}, month = {03}, pages = {1–4}, publisher = {ACM}, title = {{Modeling Variations in Load Intensity over Time}}, year = 2014 }

  %0 Conference Paper %1 KiHeKo2014-LT-DLIM %A von Kistowski, Jóakim %A Herbst, Nikolas Roman %A Kounev, Samuel %B Proceedings of the 3rd International Workshop on Large-Scale Testing (LT 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) %C New York, NY, USA %D 2014 %I ACM %P 1--4 %T Modeling Variations in Load Intensity over Time %U http://doi.acm.org/10.1145/2577036.2577037 %X Today's software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. Based on this observation, we identify the need for means allowing flexible definition of load profiles and address this by introducing two meta-models at different abstraction levels. At the lower abstraction level, the Descartes Load Intensity Meta-Model (DLIM) offers a structured and accessible way of describing the load intensity over time by editing and combining mathematical functions. The High-Level Descartes Load Intensity Meta-Model (HLDLIM) allows the description of load variations using few defined parameters that characterize the seasonal patterns, trends, bursts and noise parts. We demonstrate that both meta-models are capable of capturing real-world load profiles with acceptable accuracy through comparison with a real life trace.
• 
  Towards a Resource Elasticity Benchmark for Cloud Environments. Weber, Andreas; Herbst, Nikolas Roman; Groenda, Henning; Kounev, Samuel; in Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 5:1–5:8. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Auto-scaling features offered by today's cloud infrastructures provide increased flexibility especially for customers that experience high variations in the load intensity over time. However, auto-scaling features introduce new system quality attributes when considering their accuracy, timing, and boundaries. Therefore, distinguishing between different offerings has become a complex task, as it is not yet supported by reliable metrics and measurement approaches. In this paper, we discuss shortcomings of existing approaches for measuring and evaluating elastic behavior and propose a novel benchmark methodology specifically designed for evaluating the elasticity aspects of modern cloud platforms. The benchmark is based on open workloads with realistic load variation profiles that are calibrated to induce identical resource demand variations independent of the underlying hardware performance. Furthermore, we propose new metrics that capture the accuracy of resource allocations and de-allocations, as well as the timing aspects of an auto-scaling mechanism explicitly.

  @inproceedings{WeHeGrKo2014-HotTopicsWS-ElaBench, abstract = {{Auto-scaling features offered by today's cloud infrastructures provide increased flexibility especially for customers that experience high variations in the load intensity over time. However, auto-scaling features introduce new system quality attributes when considering their accuracy, timing, and boundaries. Therefore, distinguishing between different offerings has become a complex task, as it is not yet supported by reliable metrics and measurement approaches. In this paper, we discuss shortcomings of existing approaches for measuring and evaluating elastic behavior and propose a novel benchmark methodology specifically designed for evaluating the elasticity aspects of modern cloud platforms. The benchmark is based on open workloads with realistic load variation profiles that are calibrated to induce identical resource demand variations independent of the underlying hardware performance. Furthermore, we propose new metrics that capture the accuracy of resource allocations and de-allocations, as well as the timing aspects of an auto-scaling mechanism explicitly.}}, address = {New York, NY, USA}, author = {Weber, Andreas and Herbst, Nikolas Roman and Groenda, Henning and Kounev, Samuel}, booktitle = {Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {t_workshop}, month = {03}, pages = {5:1–5:8}, publisher = {ACM}, series = {HotTopiCS '14}, title = {{Towards a Resource Elasticity Benchmark for Cloud Environments}}, year = 2014 }

  %0 Conference Paper %1 WeHeGrKo2014-HotTopicsWS-ElaBench %A Weber, Andreas %A Herbst, Nikolas Roman %A Groenda, Henning %A Kounev, Samuel %B Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) %C New York, NY, USA %D 2014 %I ACM %P 5:1--5:8 %T Towards a Resource Elasticity Benchmark for Cloud Environments %U http://doi.acm.org/10.1145/2649563.2649571 %X Auto-scaling features offered by today's cloud infrastructures provide increased flexibility especially for customers that experience high variations in the load intensity over time. However, auto-scaling features introduce new system quality attributes when considering their accuracy, timing, and boundaries. Therefore, distinguishing between different offerings has become a complex task, as it is not yet supported by reliable metrics and measurement approaches. In this paper, we discuss shortcomings of existing approaches for measuring and evaluating elastic behavior and propose a novel benchmark methodology specifically designed for evaluating the elasticity aspects of modern cloud platforms. The benchmark is based on open workloads with realistic load variation profiles that are calibrated to induce identical resource demand variations independent of the underlying hardware performance. Furthermore, we propose new metrics that capture the accuracy of resource allocations and de-allocations, as well as the timing aspects of an auto-scaling mechanism explicitly.

• 
  Using Queuing Models for Large System Migration Scenarios -- An Industrial Case Study with IBM System z. Vaupel, Robert; Noorshams, Qais; Kounev, Samuel; Reussner, Ralf; in Computer Performance Engineering, Proceedings of the 10th European Performance Engineering Workshop (EPEW 2013), Venice, Italy, M. S. Balsamo, W. J. Knottenbelt, A. Marin (eds.) (2013). (Vol. 8168) 263–275. Springer Berlin Heidelberg.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @incollection{VaNoKoRe2013-EPEW-IndustrialCaseStudySystemz, author = {Vaupel, Robert and Noorshams, Qais and Kounev, Samuel and Reussner, Ralf}, booktitle = {Computer Performance Engineering, Proceedings of the 10th European Performance Engineering Workshop (EPEW 2013), Venice, Italy}, editor = {Balsamo, Maria Simonetta and Knottenbelt, William J. and Marin, Andrea}, keywords = {t_workshop}, month = {09}, pages = {263-275}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Using Queuing Models for Large System Migration Scenarios – An Industrial Case Study with IBM System z}}, volume = 8168, year = 2013 }

  %0 Book Section %1 VaNoKoRe2013-EPEW-IndustrialCaseStudySystemz %A Vaupel, Robert %A Noorshams, Qais %A Kounev, Samuel %A Reussner, Ralf %B Computer Performance Engineering, Proceedings of the 10th European Performance Engineering Workshop (EPEW 2013), Venice, Italy %D 2013 %E Balsamo, Maria Simonetta %E Knottenbelt, William J. %E Marin, Andrea %I Springer Berlin Heidelberg %P 263-275 %T Using Queuing Models for Large System Migration Scenarios -- An Industrial Case Study with IBM System z %U http://dx.doi.org/10.1007/978-3-642-40725-3_20 %V 8168
• 
  RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications. Kounev, Samuel; Rizou, Stamatia; Zschaler, Steffen; Alexakis, Spiros; Bures, Tomas; Jézéquel, Jean-Marc; Kourtesis, Dimitrios; Pantelopoulos, Stelios; in International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) (2013).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{Ko2013-HotTopiCS-Relate, author = {Kounev, Samuel and Rizou, Stamatia and Zschaler, Steffen and Alexakis, Spiros and Bures, Tomas and J{\'e}z{\'e}quel, Jean-Marc and Kourtesis, Dimitrios and Pantelopoulos, Stelios}, booktitle = {International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)}, keywords = {t_workshop}, month = {04}, title = {{RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications}}, year = 2013 }

  %0 Conference Paper %1 Ko2013-HotTopiCS-Relate %A Kounev, Samuel %A Rizou, Stamatia %A Zschaler, Steffen %A Alexakis, Spiros %A Bures, Tomas %A Jézéquel, Jean-Marc %A Kourtesis, Dimitrios %A Pantelopoulos, Stelios %B International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) %D 2013 %T RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications
• 
  Model-Based Throughput Prediction in Data Center Networks. Rygielski, Piotr; Kounev, Samuel; Zschaler, Steffen; in Proceedings of the 2nd IEEE International Workshop on Measurements and Networking (M\&N 2013) (2013). 167–172.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  In this paper, we address the problem of performance analysis in computer networks. We present a new meta-model designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. We evaluate the approach in a case study of a road traffic monitoring system. We compare the performance prediction results against the real system and a benchmark. The presented results show that our approach, despite of introducing many modeling abstractions, delivers predictions with errors less than 32% and correctly detects bottlenecks in the modeled network.

  @inproceedings{RyKoZs2013-ThroughputPrediction, abstract = {In this paper, we address the problem of performance analysis in computer networks. We present a new meta-model designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. We evaluate the approach in a case study of a road traffic monitoring system. We compare the performance prediction results against the real system and a benchmark. The presented results show that our approach, despite of introducing many modeling abstractions, delivers predictions with errors less than 32% and correctly detects bottlenecks in the modeled network.}, author = {Rygielski, Piotr and Kounev, Samuel and Zschaler, Steffen}, booktitle = {{Proceedings of the 2nd IEEE International Workshop on Measurements and Networking (M\&N 2013)}}, keywords = {DNI}, month = 10, pages = {167–172}, title = {{Model-Based Throughput Prediction in Data Center Networks}}, year = 2013 }

  %0 Conference Paper %1 RyKoZs2013-ThroughputPrediction %A Rygielski, Piotr %A Kounev, Samuel %A Zschaler, Steffen %B Proceedings of the 2nd IEEE International Workshop on Measurements and Networking (M\&N 2013) %D 2013 %P 167--172 %T Model-Based Throughput Prediction in Data Center Networks %X In this paper, we address the problem of performance analysis in computer networks. We present a new meta-model designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. We evaluate the approach in a case study of a road traffic monitoring system. We compare the performance prediction results against the real system and a benchmark. The presented results show that our approach, despite of introducing many modeling abstractions, delivers predictions with errors less than 32% and correctly detects bottlenecks in the modeled network.
• 
  Controlling the Palladio Bench using the Descartes Query Language. Gorsler, Fabian; Brosig, Fabian; Kounev, Samuel; in Proceedings of the Symposium on Software Performance: Joint Kieker/Palladio Days (KPDAYS 2013), S. Becker, W. Hasselbring, A. van Hoorn, R. Reussner (eds.) (2013). 109–118. CEUR-WS.org, Aachen, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  The Palladio Bench is a tool to model, simulate and analyze Palladio Component Model (PCM) instances. However, for the Palladio Bench, no single interface to automate experiments or Application Programming Interface (API) to trigger the simulation of PCM instances and to extract performance prediction results is available. The Descartes Query Language (DQL) is a novel approach of a declarative query language to integrate different performance modeling and prediction techniques behind a unifying interface. Users benefit from the abstraction of specific tools to prepare and trigger performance predictions, less effort to obtain performance metrics of interest, and means to automate performance predictions. In this paper, we describe the realization of a DQL Connector for PCM and demonstrate the applicability of our approach in a case study.

  @inproceedings{GoBrKo2013-ControllingPCM, abstract = {The Palladio Bench is a tool to model, simulate and analyze Palladio Component Model (PCM) instances. However, for the Palladio Bench, no single interface to automate experiments or Application Programming Interface (API) to trigger the simulation of PCM instances and to extract performance prediction results is available. The Descartes Query Language (DQL) is a novel approach of a declarative query language to integrate different performance modeling and prediction techniques behind a unifying interface. Users benefit from the abstraction of specific tools to prepare and trigger performance predictions, less effort to obtain performance metrics of interest, and means to automate performance predictions. In this paper, we describe the realization of a DQL Connector for PCM and demonstrate the applicability of our approach in a case study.}, address = {Aachen, Germany}, author = {Gorsler, Fabian and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the Symposium on Software Performance: Joint Kieker/Palladio Days (KPDAYS 2013)}, editor = {Becker, Steffen and Hasselbring, Wilhelm and van Hoorn, Andr{\'e} and Reussner, Ralf}, keywords = {DQL}, month = 11, number = 1083, pages = {109–118}, publisher = {CEUR-WS.org}, title = {Controlling the Palladio Bench using the Descartes Query Language}, year = 2013 }

  %0 Conference Paper %1 GoBrKo2013-ControllingPCM %A Gorsler, Fabian %A Brosig, Fabian %A Kounev, Samuel %B Proceedings of the Symposium on Software Performance: Joint Kieker/Palladio Days (KPDAYS 2013) %C Aachen, Germany %D 2013 %E Becker, Steffen %E Hasselbring, Wilhelm %E van Hoorn, André %E Reussner, Ralf %I CEUR-WS.org %N 1083 %P 109--118 %T Controlling the Palladio Bench using the Descartes Query Language %U http://ceur-ws.org/Vol-1083/ %X The Palladio Bench is a tool to model, simulate and analyze Palladio Component Model (PCM) instances. However, for the Palladio Bench, no single interface to automate experiments or Application Programming Interface (API) to trigger the simulation of PCM instances and to extract performance prediction results is available. The Descartes Query Language (DQL) is a novel approach of a declarative query language to integrate different performance modeling and prediction techniques behind a unifying interface. Users benefit from the abstraction of specific tools to prepare and trigger performance predictions, less effort to obtain performance metrics of interest, and means to automate performance predictions. In this paper, we describe the realization of a DQL Connector for PCM and demonstrate the applicability of our approach in a case study.

• 
  Predictive Modelling of Peer-to-Peer Event-driven Communication in Component-based Systems. Rathfelder, Christoph; Evans, David; Kounev, Samuel; in Proceedings of the 7th European Performance Engineering Workshop (EPEW 2010), A. Aldini, M. Bernardo, L. Bononi, V. Cortellessa (eds.) (2010). (Vol. 6342) 219–235. Springer-Verlag, Berlin, Heidelberg.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  The event-driven communication paradigm is used increasingly often to build loosely-coupled distributed systems in many industry domains including telecommunications, transportation, and supply chain management. However, the loose coupling of components in such systems makes it hard for developers to estimate their behaviour and performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-driven communication. In this paper, we present a case study of a real-life road traffic monitoring system that shows how event-driven communication can be modelled for performance prediction and capacity planning. Our approach is based on the Palladio Component Model (PCM) which we have extended to support event-driven communication. We evaluate the accuracy of our modelling approach in a number of different workload and configuration scenarios. The results demonstrate the practicality and effectiveness of the proposed approach.

  @inproceedings{rathfelder2010a, abstract = {The event-driven communication paradigm is used increasingly often to build loosely-coupled distributed systems in many industry domains including telecommunications, transportation, and supply chain management. However, the loose coupling of components in such systems makes it hard for developers to estimate their behaviour and performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-driven communication. In this paper, we present a case study of a real-life road traffic monitoring system that shows how event-driven communication can be modelled for performance prediction and capacity planning. Our approach is based on the Palladio Component Model (PCM) which we have extended to support event-driven communication. We evaluate the accuracy of our modelling approach in a number of different workload and configuration scenarios. The results demonstrate the practicality and effectiveness of the proposed approach.}, address = {Berlin, Heidelberg}, author = {Rathfelder, Christoph and Evans, David and Kounev, Samuel}, booktitle = {Proceedings of the 7th European Performance Engineering Workshop (EPEW 2010)}, editor = {Aldini, Alessandro and Bernardo, Marco and Bononi, Luciano and Cortellessa, Vittorio}, keywords = {t_workshop}, month = {09}, pages = {219–235}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{P}redictive {M}odelling of {P}eer-to-{P}eer {E}vent-driven {C}ommunication in {C}omponent-based {S}ystems}, volume = 6342, year = 2010 }

  %0 Conference Paper %1 rathfelder2010a %A Rathfelder, Christoph %A Evans, David %A Kounev, Samuel %B Proceedings of the 7th European Performance Engineering Workshop (EPEW 2010) %C Berlin, Heidelberg %D 2010 %E Aldini, Alessandro %E Bernardo, Marco %E Bononi, Luciano %E Cortellessa, Vittorio %I Springer-Verlag %P 219--235 %T Predictive Modelling of Peer-to-Peer Event-driven Communication in Component-based Systems %V 6342 %X The event-driven communication paradigm is used increasingly often to build loosely-coupled distributed systems in many industry domains including telecommunications, transportation, and supply chain management. However, the loose coupling of components in such systems makes it hard for developers to estimate their behaviour and performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-driven communication. In this paper, we present a case study of a real-life road traffic monitoring system that shows how event-driven communication can be modelled for performance prediction and capacity planning. Our approach is based on the Palladio Component Model (PCM) which we have extended to support event-driven communication. We evaluate the accuracy of our modelling approach in a number of different workload and configuration scenarios. The results demonstrate the practicality and effectiveness of the proposed approach.
• 
  Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe Kounev, Samuel; (V. Pankratius; S. Kounev, eds.) (2010). KIT Scientific Publishing, Karlsruhe, Germany.

  ISBN: 978-3-86644-508-6
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  @proceedings{PaKo2010-KIT-YoungInformatics, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, editor = {Pankratius, Victor and Kounev, Samuel}, keywords = {t_workshop}, month = {07}, note = {ISBN: 978-3-86644-508-6}, publisher = {KIT Scientific Publishing}, title = {{Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe}}, year = 2010 }

  %0 Conference Proceedings %1 PaKo2010-KIT-YoungInformatics %A Kounev, Samuel %C Karlsruhe, Germany %D 2010 %E Pankratius, Victor %E Kounev, Samuel %I KIT Scientific Publishing %T Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe %U http://uvka.ubka.uni-karlsruhe.de/shop/isbn/978-3-86644-508-6
• 
  Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap. Kounev, Samuel; in Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe (2010). KIT Scientific Publishing, Karlsruhe, Germany.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{Ko2010-KIT-Roadmap, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, booktitle = {{Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe}}, keywords = {t_workshop}, month = {07}, publisher = {KIT Scientific Publishing}, title = {{Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap}}, year = 2010 }

  %0 Conference Paper %1 Ko2010-KIT-Roadmap %A Kounev, Samuel %B Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe %C Karlsruhe, Germany %D 2010 %I KIT Scientific Publishing %T Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap %U http://uvka.ubka.uni-karlsruhe.de/shop/isbn/978-3-86644-508-6
• 
  Benchmarking Publish/Subscribe-based Messaging Systems. Sachs, Kai; Appel, Stefan; Kounev, Samuel; Buchmann, Alejandro; in Proc. of 2nd International Workshop on Benchmarking of Database Management Systems and Data-Oriented Web Technologies (BenchmarX’10)., M. Necasky, E. Pardede (eds.) (2010). (Vol. 6193) Springer.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{SaApKoBu2010-BenchmarX-PubSub, author = {Sachs, Kai and Appel, Stefan and Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proc. of 2nd International Workshop on Benchmarking of Database Management Systems and Data-Oriented Web Technologies (BenchmarX'10).}, editor = {Necasky, Martin and Pardede, Eric}, keywords = {t_workshop}, month = {04}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Benchmarking Publish/Subscribe-based Messaging Systems}}, volume = 6193, year = 2010 }

  %0 Conference Paper %1 SaApKoBu2010-BenchmarX-PubSub %A Sachs, Kai %A Appel, Stefan %A Kounev, Samuel %A Buchmann, Alejandro %B Proc. of 2nd International Workshop on Benchmarking of Database Management Systems and Data-Oriented Web Technologies (BenchmarX'10). %D 2010 %E Necasky, Martin %E Pardede, Eric %I Springer %T Benchmarking Publish/Subscribe-based Messaging Systems %V 6193

• 
  Automated Extraction of Palladio Component Models from Running Enterprise Java Applications. Brosig, Fabian; Kounev, Samuel; Krogmann, Klaus; in Proceedings of the 1st International Workshop on Run-time mOdels for Self-managing Systems and Applications (ROSSA 2009). In conjunction with Fourth International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2009), Pisa, Italy, October 19, 2009. (2009). ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Nowadays, software systems have to fulfill increasingly stringent requirements for performance and scalability. To ensure that a system meets its performance requirements during operation, the ability to predict its performance under different configurations and workloads is essential. Most performance analysis tools currently used in industry focus on monitoring the current system state. They provide low-level monitoring data without any performance prediction capabilities. For performance prediction, performance models are normally required. However, building predictive performance models manually requires a lot of time and effort. In this paper, we present a method for automated extraction of performance models of Java EE applications, based on monitoring data collected during operation. We extract instances of the Palladio Component Model (PCM) - a performance meta-model targeted at component-based systems. We evaluate the model extraction method in the context of a case study with a real-world enterprise application. Even though the extraction requires some manual intervention, the case study demonstrates that the existing gap between low-level monitoring data and high-level performance models can be closed.

  @inproceedings{BrKoKr2009-ROSSA-Extracting_PCM_JavaEE, abstract = {Nowadays, software systems have to fulfill increasingly stringent requirements for performance and scalability. To ensure that a system meets its performance requirements during operation, the ability to predict its performance under different configurations and workloads is essential. Most performance analysis tools currently used in industry focus on monitoring the current system state. They provide low-level monitoring data without any performance prediction capabilities. For performance prediction, performance models are normally required. However, building predictive performance models manually requires a lot of time and effort. In this paper, we present a method for automated extraction of performance models of Java EE applications, based on monitoring data collected during operation. We extract instances of the Palladio Component Model (PCM) - a performance meta-model targeted at component-based systems. We evaluate the model extraction method in the context of a case study with a real-world enterprise application. Even though the extraction requires some manual intervention, the case study demonstrates that the existing gap between low-level monitoring data and high-level performance models can be closed.}, author = {Brosig, Fabian and Kounev, Samuel and Krogmann, Klaus}, booktitle = {Proceedings of the 1st International Workshop on Run-time mOdels for Self-managing Systems and Applications (ROSSA 2009). In conjunction with Fourth International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2009), Pisa, Italy, October 19, 2009.}, keywords = {t_workshop}, month = 10, publisher = {ACM, New York, NY, USA}, title = {{Automated Extraction of Palladio Component Models from Running Enterprise Java Applications}}, year = 2009 }

  %0 Conference Paper %1 BrKoKr2009-ROSSA-Extracting_PCM_JavaEE %A Brosig, Fabian %A Kounev, Samuel %A Krogmann, Klaus %B Proceedings of the 1st International Workshop on Run-time mOdels for Self-managing Systems and Applications (ROSSA 2009). In conjunction with Fourth International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2009), Pisa, Italy, October 19, 2009. %D 2009 %I ACM, New York, NY, USA %T Automated Extraction of Palladio Component Models from Running Enterprise Java Applications %X Nowadays, software systems have to fulfill increasingly stringent requirements for performance and scalability. To ensure that a system meets its performance requirements during operation, the ability to predict its performance under different configurations and workloads is essential. Most performance analysis tools currently used in industry focus on monitoring the current system state. They provide low-level monitoring data without any performance prediction capabilities. For performance prediction, performance models are normally required. However, building predictive performance models manually requires a lot of time and effort. In this paper, we present a method for automated extraction of performance models of Java EE applications, based on monitoring data collected during operation. We extract instances of the Palladio Component Model (PCM) - a performance meta-model targeted at component-based systems. We evaluate the model extraction method in the context of a case study with a real-world enterprise application. Even though the extraction requires some manual intervention, the case study demonstrates that the existing gap between low-level monitoring data and high-level performance models can be closed.
• 
  Modeling Event-Driven Service-Oriented Systems using the Palladio Component Model. Rathfelder, Christoph; Kounev, Samuel; in Proceedings of the 1st International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS 2009) (2009). 33–38. ACM, New York, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  The use of event-based communication within a Service-Oriented Architecture promises several benefits including more loosely-coupled services and better scalability. However, the loose coupling of services makes it difficult for system developers to estimate the behavior and performance of systems composed of multiple services. Most existing performance prediction techniques for systems using event-based communication require specialized knowledge to build the necessary prediction models. Furthermore, general purpose design-oriented performance models for component-based systems provide limited support for modeling event-based communication. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication. We show how this extension can be exploited to model event-driven service-oriented systems with the aim to evaluate their performance and scalability.

  @inproceedings{rathfelder2009b, abstract = {The use of event-based communication within a Service-Oriented Architecture promises several benefits including more loosely-coupled services and better scalability. However, the loose coupling of services makes it difficult for system developers to estimate the behavior and performance of systems composed of multiple services. Most existing performance prediction techniques for systems using event-based communication require specialized knowledge to build the necessary prediction models. Furthermore, general purpose design-oriented performance models for component-based systems provide limited support for modeling event-based communication. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication. We show how this extension can be exploited to model event-driven service-oriented systems with the aim to evaluate their performance and scalability.}, address = {New York, USA}, author = {Rathfelder, Christoph and Kounev, Samuel}, booktitle = {Proceedings of the 1st International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS 2009)}, keywords = {t_workshop}, month = {08}, pages = {33–38}, publisher = {ACM}, title = {{M}odeling {E}vent-{D}riven {S}ervice-{O}riented {S}ystems using the {P}alladio {C}omponent {M}odel}, year = 2009 }

  %0 Conference Paper %1 rathfelder2009b %A Rathfelder, Christoph %A Kounev, Samuel %B Proceedings of the 1st International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS 2009) %C New York, USA %D 2009 %I ACM %P 33--38 %T Modeling Event-Driven Service-Oriented Systems using the Palladio Component Model %X The use of event-based communication within a Service-Oriented Architecture promises several benefits including more loosely-coupled services and better scalability. However, the loose coupling of services makes it difficult for system developers to estimate the behavior and performance of systems composed of multiple services. Most existing performance prediction techniques for systems using event-based communication require specialized knowledge to build the necessary prediction models. Furthermore, general purpose design-oriented performance models for component-based systems provide limited support for modeling event-based communication. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication. We show how this extension can be exploited to model event-driven service-oriented systems with the aim to evaluate their performance and scalability.

• 
  Workload Characterization of the SPECjms2007 Benchmark. Sachs, Kai; Kounev, Samuel; Bacon, Jean; Buchmann, Alejandro; in Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27--28, 2007, K. Wolter (ed.) (2007). (Vol. 4748) 228–244. Springer Verlag, Heidelberg, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. There is a strong interest in the end user and analyst communities for a standardized benchmark suite for evaluating the performance and scalability of MOM. In this paper, we present a workload characterization of the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. In addition to providing standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to customize the workload according to their requirements. The workload characterization presented in this paper serves two purposes i) to help users understand the internal components of the SPECjms2007 workload and the way they are scaled, ii) to show how the workload can be customized to exercise and evaluate selected aspects of MOM performance.We discuss how the various features supported by the benchmark can be exploited for in-depth performance analysis of MOM infrastructures.

  @inproceedings{SaKoBaBu2007-EPEW-WorkloadChar_SPECjms2007, abstract = {Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. There is a strong interest in the end user and analyst communities for a standardized benchmark suite for evaluating the performance and scalability of MOM. In this paper, we present a workload characterization of the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. In addition to providing standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to customize the workload according to their requirements. The workload characterization presented in this paper serves two purposes i) to help users understand the internal components of the SPECjms2007 workload and the way they are scaled, ii) to show how the workload can be customized to exercise and evaluate selected aspects of MOM performance.We discuss how the various features supported by the benchmark can be exploited for in-depth performance analysis of MOM infrastructures.}, address = {Heidelberg, Germany}, author = {Sachs, Kai and Kounev, Samuel and Bacon, Jean and Buchmann, Alejandro}, booktitle = {Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27–28, 2007}, editor = {Wolter, Katinka}, keywords = {t_workshop}, month = {09}, pages = {228–244}, publisher = {Springer Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Workload Characterization of the SPECjms2007 Benchmark}}, volume = 4748, year = 2007 }

  %0 Conference Paper %1 SaKoBaBu2007-EPEW-WorkloadChar_SPECjms2007 %A Sachs, Kai %A Kounev, Samuel %A Bacon, Jean %A Buchmann, Alejandro %B Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27--28, 2007 %C Heidelberg, Germany %D 2007 %E Wolter, Katinka %I Springer Verlag %P 228--244 %T Workload Characterization of the SPECjms2007 Benchmark %U http://www.springer.com/computer/programming/book/978-3-540-75210-3 %V 4748 %X Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. There is a strong interest in the end user and analyst communities for a standardized benchmark suite for evaluating the performance and scalability of MOM. In this paper, we present a workload characterization of the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. In addition to providing standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to customize the workload according to their requirements. The workload characterization presented in this paper serves two purposes i) to help users understand the internal components of the SPECjms2007 workload and the way they are scaled, ii) to show how the workload can be customized to exercise and evaluate selected aspects of MOM performance.We discuss how the various features supported by the benchmark can be exploited for in-depth performance analysis of MOM infrastructures.
• 
  Building Online Performance Models of Grid Middleware with Fine-Grained Load-Balancing: A Globus Toolkit Case Study. Nou, Ramon; Kounev, Samuel; Torres, Jordi; in Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27-28, 2007, K. Wolter (ed.) (2007). 125–140. Springer, Heidelberg, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  As Grid computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. To guarantee that QoS requirements are continuously satisfied, the Grid middleware must be capable of predicting the application performance on the fly when deciding how to distribute the workload among the available resources. One way to achieve this is by using online performance models that get generated and analyzed on the fly. In this paper, we present a novel case study with the Globus Toolkit in which we show how performance models can be generated dynamically and used to provide online performance prediction capabilities. We have augmented the Grid middleware with an online performance prediction component that can be called at any time during operation to predict the Grid performance for a given resource allocation and load-balancing strategy. We evaluate the quality of our performance prediction mechanism and present some experimental results that demonstrate its effectiveness and practicality. The framework we propose can be used to design intelligent QoS-aware resource allocation and admission control mechanisms.

  @inproceedings{NoKoTo2007-EPEW-GridOnlinePerfModels, abstract = {As Grid computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. To guarantee that QoS requirements are continuously satisfied, the Grid middleware must be capable of predicting the application performance on the fly when deciding how to distribute the workload among the available resources. One way to achieve this is by using online performance models that get generated and analyzed on the fly. In this paper, we present a novel case study with the Globus Toolkit in which we show how performance models can be generated dynamically and used to provide online performance prediction capabilities. We have augmented the Grid middleware with an online performance prediction component that can be called at any time during operation to predict the Grid performance for a given resource allocation and load-balancing strategy. We evaluate the quality of our performance prediction mechanism and present some experimental results that demonstrate its effectiveness and practicality. The framework we propose can be used to design intelligent QoS-aware resource allocation and admission control mechanisms.}, address = {Heidelberg, Germany}, author = {Nou, Ramon and Kounev, Samuel and Torres, Jordi}, booktitle = {Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27-28, 2007}, editor = {Wolter, Katinka}, keywords = {t_workshop}, month = {09}, number = 4748, pages = {125–140}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{Building Online Performance Models of Grid Middleware with Fine-Grained Load-Balancing: A Globus Toolkit Case Study}}, year = 2007 }

  %0 Conference Paper %1 NoKoTo2007-EPEW-GridOnlinePerfModels %A Nou, Ramon %A Kounev, Samuel %A Torres, Jordi %B Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27-28, 2007 %C Heidelberg, Germany %D 2007 %E Wolter, Katinka %I Springer %N 4748 %P 125--140 %T Building Online Performance Models of Grid Middleware with Fine-Grained Load-Balancing: A Globus Toolkit Case Study %X As Grid computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. To guarantee that QoS requirements are continuously satisfied, the Grid middleware must be capable of predicting the application performance on the fly when deciding how to distribute the workload among the available resources. One way to achieve this is by using online performance models that get generated and analyzed on the fly. In this paper, we present a novel case study with the Globus Toolkit in which we show how performance models can be generated dynamically and used to provide online performance prediction capabilities. We have augmented the Grid middleware with an online performance prediction component that can be called at any time during operation to predict the Grid performance for a given resource allocation and load-balancing strategy. We evaluate the quality of our performance prediction mechanism and present some experimental results that demonstrate its effectiveness and practicality. The framework we propose can be used to design intelligent QoS-aware resource allocation and admission control mechanisms.
• 
  Designing a Workload Scenario for Benchmarking Message-Oriented Middleware. Sachs, Kai; Kounev, Samuel; Carter, Marc; Buchmann, Alejandro; in Proceedings of the 2007 SPEC Benchmark Workshop, Austin, Texas, January 21, 2007 (2007). SPEC.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern information-driven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC's new SPECjms benchmark which will be the world's first industry-standard benchmark for MOM.

  @inproceedings{SaKoCaBu2007-SPEC_BW-BenchmarkingMOM, abstract = {Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern information-driven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC's new SPECjms benchmark which will be the world's first industry-standard benchmark for MOM.}, author = {Sachs, Kai and Kounev, Samuel and Carter, Marc and Buchmann, Alejandro}, booktitle = {Proceedings of the 2007 SPEC Benchmark Workshop, Austin, Texas, January 21, 2007}, keywords = {t_workshop}, month = {01}, publisher = {SPEC}, title = {{Designing a Workload Scenario for Benchmarking Message-Oriented Middleware}}, year = 2007 }

  %0 Conference Paper %1 SaKoCaBu2007-SPEC_BW-BenchmarkingMOM %A Sachs, Kai %A Kounev, Samuel %A Carter, Marc %A Buchmann, Alejandro %B Proceedings of the 2007 SPEC Benchmark Workshop, Austin, Texas, January 21, 2007 %D 2007 %I SPEC %T Designing a Workload Scenario for Benchmarking Message-Oriented Middleware %U http://www.spec.org/workshops/2007/austin/ %X Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern information-driven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC's new SPECjms benchmark which will be the world's first industry-standard benchmark for MOM.

• 
  J2EE Performance and Scalability - From Measuring to Predicting. Kounev, Samuel; in Proceedings of the 2006 SPEC Benchmark Workshop, Austin, Texas, USA (2006).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  J2EE applications are becoming increasingly ubiquitous and with their increasing adoption, performance and scalability issues are gaining in importance. For a J2EE application to perform well and be scalable, both the platform on which it is built and the application design must be efficient and scalable. Industry-standard benchmarks such as the SPECjAppServer set of benchmarks help to evaluate the performance and scalability of alternative platforms for J2EE applications, however, they cannot be used to evaluate the performance and scalability of concrete applications built on the selected platforms. In this paper, we present a systematic approach for evaluating and predicting the performance and scalability of J2EE applications based on modeling and simulation. The approach helps to identify and eliminate bottlenecks in the application design and ensure that systems are designed and sized to meet their quality of service requirements. We introduce our approach by showing how it can be applied to the SPECjAppServer2004 benchmark which is used as a representative J2EE application. A detailed model of a SPECjAppServer2004 deployment is built in a step-by-step fashion and then used to predict the behavior of the system under load. The approach is validated by comparing model predictions against measurements on the real system.

  @inproceedings{Ko2006-SPEC_BW-J2EEPerfScal, abstract = {J2EE applications are becoming increasingly ubiquitous and with their increasing adoption, performance and scalability issues are gaining in importance. For a J2EE application to perform well and be scalable, both the platform on which it is built and the application design must be efficient and scalable. Industry-standard benchmarks such as the SPECjAppServer set of benchmarks help to evaluate the performance and scalability of alternative platforms for J2EE applications, however, they cannot be used to evaluate the performance and scalability of concrete applications built on the selected platforms. In this paper, we present a systematic approach for evaluating and predicting the performance and scalability of J2EE applications based on modeling and simulation. The approach helps to identify and eliminate bottlenecks in the application design and ensure that systems are designed and sized to meet their quality of service requirements. We introduce our approach by showing how it can be applied to the SPECjAppServer2004 benchmark which is used as a representative J2EE application. A detailed model of a SPECjAppServer2004 deployment is built in a step-by-step fashion and then used to predict the behavior of the system under load. The approach is validated by comparing model predictions against measurements on the real system.}, author = {Kounev, Samuel}, booktitle = {Proceedings of the 2006 SPEC Benchmark Workshop, Austin, Texas, USA}, keywords = {t_workshop}, month = {01}, title = {{J2EE Performance and Scalability - From Measuring to Predicting}}, year = 2006 }

  %0 Conference Paper %1 Ko2006-SPEC_BW-J2EEPerfScal %A Kounev, Samuel %B Proceedings of the 2006 SPEC Benchmark Workshop, Austin, Texas, USA %D 2006 %T J2EE Performance and Scalability - From Measuring to Predicting %X J2EE applications are becoming increasingly ubiquitous and with their increasing adoption, performance and scalability issues are gaining in importance. For a J2EE application to perform well and be scalable, both the platform on which it is built and the application design must be efficient and scalable. Industry-standard benchmarks such as the SPECjAppServer set of benchmarks help to evaluate the performance and scalability of alternative platforms for J2EE applications, however, they cannot be used to evaluate the performance and scalability of concrete applications built on the selected platforms. In this paper, we present a systematic approach for evaluating and predicting the performance and scalability of J2EE applications based on modeling and simulation. The approach helps to identify and eliminate bottlenecks in the application design and ensure that systems are designed and sized to meet their quality of service requirements. We introduce our approach by showing how it can be applied to the SPECjAppServer2004 benchmark which is used as a representative J2EE application. A detailed model of a SPECjAppServer2004 deployment is built in a step-by-step fashion and then used to predict the behavior of the system under load. The approach is validated by comparing model predictions against measurements on the real system.

• [ 2023 ]
• [ 2022 ]
• [ 2021 ]
• [ 2020 ]
• [ 2018 ]
• [ 2017 ]
• [ 2016 ]
• [ 2013 ]
• [ 2010 ]

• 
  Challenges and Future Directions in Efficiency Benchmarking (Vision Paper). Meissner, Maximilian; Lange, Klaus-Dieter; Sharma, Sanjay; Arnold, Jeremy; Cragin, Aaron; Galizia, Patrick; Petrich, Mike; Zhang, Bin; Kounev, Samuel; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 51–55. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.

  @inproceedings{10.1145/3578245.3585034, abstract = {SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Sharma, Sanjay and Arnold, Jeremy and Cragin, Aaron and Galizia, Patrick and Petrich, Mike and Zhang, Bin and Kounev, Samuel}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {51–55}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Challenges and Future Directions in Efficiency Benchmarking (Vision Paper)}, year = 2023 }

  %0 Conference Paper %1 10.1145/3578245.3585034 %A Meissner, Maximilian %A Lange, Klaus-Dieter %A Sharma, Sanjay %A Arnold, Jeremy %A Cragin, Aaron %A Galizia, Patrick %A Petrich, Mike %A Zhang, Bin %A Kounev, Samuel %B Companion of the 2023 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2023 %I Association for Computing Machinery %P 51–55 %T Challenges and Future Directions in Efficiency Benchmarking (Vision Paper) %U https://doi.org/10.1145/3578245.3585034 %X SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.

• 
  FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles. Hadry, Marius; Lesch, Veronika; Kounev, Samuel; in Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022) (2022). ACM.
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  @inproceedings{hadry2022towards, author = {Hadry, Marius and Lesch, Veronika and Kounev, Samuel}, booktitle = {Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022)}, keywords = {cyber-physical_systems}, month = {04}, publisher = {ACM}, title = {FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles}, year = 2022 }

  %0 Conference Paper %1 hadry2022towards %A Hadry, Marius %A Lesch, Veronika %A Kounev, Samuel %B Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022) %D 2022 %I ACM %T FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles

• 
  Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads. Domaschka, Jörg; Eismann, Simon; Leznik, Mark; Grohmann, Johannes; Kounev, Samuel; Seybold, Daniel; in Companion of the ACM/SPEC International Conference on Performance Engineering (2021). (Vol. ICPE ’21) 175–178. Association for Computing Machinery, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies.

  @inproceedings{domaschka2021buzzy, abstract = {Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies.}, address = {New York, NY, USA}, author = {Domaschka, Jörg and Eismann, Simon and Leznik, Mark and Grohmann, Johannes and Kounev, Samuel and Seybold, Daniel}, booktitle = {Companion of the ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {175–178}, publisher = {Association for Computing Machinery}, title = {Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads}, volume = {ICPE '21}, year = 2021 }

  %0 Conference Paper %1 domaschka2021buzzy %A Domaschka, Jörg %A Eismann, Simon %A Leznik, Mark %A Grohmann, Johannes %A Kounev, Samuel %A Seybold, Daniel %B Companion of the ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2021 %I Association for Computing Machinery %P 175–178 %R 10.1145/3447545.3451175 %T Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads %U https://doi.org/10.1145/3447545.3451175 %V ICPE '21 %X Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies. %@ 9781450383318

• 
  Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite. Schmitt, Norbert; Bucek, James; Lange, Klaus-Dieter; Kounev, Samuel; in Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020) (2020). ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{ScLaBuKo-ICPE20-SPEC-CPU2017, address = {New York, NY, USA}, author = {Schmitt, Norbert and Bucek, James and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020)}, keywords = {Power}, month = {04}, publisher = {ACM}, title = {{Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite}}, year = 2020 }

  %0 Conference Paper %1 ScLaBuKo-ICPE20-SPEC-CPU2017 %A Schmitt, Norbert %A Bucek, James %A Lange, Klaus-Dieter %A Kounev, Samuel %B Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020) %C New York, NY, USA %D 2020 %I ACM %T Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite %U https://doi.org/10.1145/3375555.3383759

• 
  The Vision of Self-Aware Performance Models. Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in 2018 IEEE International Conference on Software Architecture Companion (ICSA-C) (2018). 60–63.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance models are necessary components of self-aware computing systems, as they allow such systems to reason about their own state and behavior. Research in this field has developed a multitude of approaches to create, maintain, and solve performance models. In this paper, we propose a meta-self-aware computing approach making the processes of model creation, maintenance and solution themselves self-aware. This enables the automated selection and adaption of software performance engineering approaches specifically tailored to the system under study.

  @inproceedings{GrEiKo-ICSA18-Vision, abstract = {Performance models are necessary components of self-aware computing systems, as they allow such systems to reason about their own state and behavior. Research in this field has developed a multitude of approaches to create, maintain, and solve performance models. In this paper, we propose a meta-self-aware computing approach making the processes of model creation, maintenance and solution themselves self-aware. This enables the automated selection and adaption of software performance engineering approaches specifically tailored to the system under study.}, author = {Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, booktitle = {2018 IEEE International Conference on Software Architecture Companion (ICSA-C)}, keywords = {Self-adaptive-systems}, month = {04}, pages = {60–63}, title = {{The Vision of Self-Aware Performance Models}}, year = 2018 }

  %0 Conference Paper %1 GrEiKo-ICSA18-Vision %A Grohmann, Johannes %A Eismann, Simon %A Kounev, Samuel %B 2018 IEEE International Conference on Software Architecture Companion (ICSA-C) %D 2018 %P 60--63 %R 10.1109/ICSA-C.2018.00024 %T The Vision of Self-Aware Performance Models %U https://doi.org/10.1109/ICSA-C.2018.00024 %X Performance models are necessary components of self-aware computing systems, as they allow such systems to reason about their own state and behavior. Research in this field has developed a multitude of approaches to create, maintain, and solve performance models. In this paper, we propose a meta-self-aware computing approach making the processes of model creation, maintenance and solution themselves self-aware. This enables the automated selection and adaption of software performance engineering approaches specifically tailored to the system under study.
• 
  Using Machine Learning for Recommending Service Demand Estimation Approaches. Grohmann, Johannes; Herbst, Nikolas; Spinner, Simon; Kounev, Samuel; in Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018) (2018). 473–480. SciTePress.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.

  @inproceedings{GrHeSpKo-Closer18-RecServDemEst, abstract = {Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.}, author = {Grohmann, Johannes and Herbst, Nikolas and Spinner, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018)}, keywords = {Optimization}, month = {03}, organization = {INSTICC}, pages = {473–480}, publisher = {SciTePress}, title = {Using Machine Learning for Recommending Service Demand Estimation Approaches}, year = 2018 }

  %0 Conference Paper %1 GrHeSpKo-Closer18-RecServDemEst %A Grohmann, Johannes %A Herbst, Nikolas %A Spinner, Simon %A Kounev, Samuel %B Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018) %D 2018 %I SciTePress %P 473--480 %R 10.5220/0006761104730480 %T Using Machine Learning for Recommending Service Demand Estimation Approaches %U https://doi.org/10.5220/0006761104730480 %X Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%. %@ 978-989-758-295-0
• 
  The Vision of Self-aware Reordering of Security Network Function Chains. Iffländer, Lukas; Walter, Jürgen; Eismann, Simon; Kounev, Samuel; in Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 1–4. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.

  @inproceedings{IfWaEiKo2018-ICPE-SSFC-Vision, abstract = {Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Walter, Jürgen and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering}, keywords = {Self-adaptive-systems}, pages = {1–4}, publisher = {ACM}, series = {ICPE '18}, title = {The Vision of Self-aware Reordering of Security Network Function Chains}, year = 2018 }

  %0 Conference Paper %1 IfWaEiKo2018-ICPE-SSFC-Vision %A Iffländer, Lukas %A Walter, Jürgen %A Eismann, Simon %A Kounev, Samuel %B Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2018 %I ACM %P 1--4 %T The Vision of Self-aware Reordering of Security Network Function Chains %U https://dl.acm.org/doi/10.1145/3185768.3186309 %X Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.

• 
  Towards a Scalability and Energy Efficiency Benchmark for VNF. Schmitt, Norbert; von Kistowski, Jóakim; Kounev, Samuel; in Proceedings of the 9th TPC Technology Conference on Performance Engineering and Benchmarking (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Network Function Virtualization (NFV) is the transfer of network functions from dedicated devices to high-volume commodity servers. It opens opportunities for flexibility and energy savings. Concrete insights on the flexibility of specific NFV environments require measurement methodologies and benchmarks. However, current benchmarks are not measuring the ability of a virtual network function (VNF) to scale either horizontally or vertically. We therefore envision a new benchmark that measures a VNF's ability to scale while evaluating its energy efficiency at the same time. Such a benchmark would enable the selection of a suitable VNF for changing demands, deployed at an existing or new resource landscape, while minimizing energy costs.

  @inproceedings{ScKiKo2017-TPCTC-BenchmarkVision, abstract = {Network Function Virtualization (NFV) is the transfer of network functions from dedicated devices to high-volume commodity servers. It opens opportunities for flexibility and energy savings. Concrete insights on the flexibility of specific NFV environments require measurement methodologies and benchmarks. However, current benchmarks are not measuring the ability of a virtual network function (VNF) to scale either horizontally or vertically. We therefore envision a new benchmark that measures a VNF's ability to scale while evaluating its energy efficiency at the same time. Such a benchmark would enable the selection of a suitable VNF for changing demands, deployed at an existing or new resource landscape, while minimizing energy costs.}, author = {Schmitt, Norbert and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 9th TPC Technology Conference on Performance Engineering and Benchmarking}, keywords = {Power}, month = {08}, series = {TPCTC '17}, title = {{Towards a Scalability and Energy Efficiency Benchmark for VNF}}, year = 2017 }

  %0 Conference Paper %1 ScKiKo2017-TPCTC-BenchmarkVision %A Schmitt, Norbert %A von Kistowski, Jóakim %A Kounev, Samuel %B Proceedings of the 9th TPC Technology Conference on Performance Engineering and Benchmarking %D 2017 %T Towards a Scalability and Energy Efficiency Benchmark for VNF %X Network Function Virtualization (NFV) is the transfer of network functions from dedicated devices to high-volume commodity servers. It opens opportunities for flexibility and energy savings. Concrete insights on the flexibility of specific NFV environments require measurement methodologies and benchmarks. However, current benchmarks are not measuring the ability of a virtual network function (VNF) to scale either horizontally or vertically. We therefore envision a new benchmark that measures a VNF's ability to scale while evaluating its energy efficiency at the same time. Such a benchmark would enable the selection of a suitable VNF for changing demands, deployed at an existing or new resource landscape, while minimizing energy costs.
• 
  Predicting Power Consumption of High-Memory-Bandwidth Workloads. Schmitt, Norbert; von Kistowski, Jóakim; Kounev, Samuel; in Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (2017). 353–356. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  High performance workloads with high bandwidth memory utilization are among the most power consuming software applications. When writing such applications, developers can directly influence power consumption of the final software through their choice of data size and traversal method, mostly due to caching characteristics. Explicit knowledge on how choices influence power consumption can thus lead to greater overall energy efficiency. In existing work, power prediction for memory accesses and high bandwidth applications requires either detailed measurement information on the system on which the software is executed or it is too generic, not taking significant aspects, such as caching and data size into account. In this paper, we propose a power model that bridges this gap by modeling power consumption based on concrete software properties, while considering hardware characteristics on a more abstract level, characterizing it primarily using publicly available data. The model is designed to enable developers to compare power consumption of implementation alternatives for high memory bandwidth software components. We validate our model by measuring modified versions of the high bandwidth benchmark stream. We show that our model can predict the relative change of power consumption due to implementation changes and the power consumption of a concrete system under test with an average error of 19 percent.

  @inproceedings{ScKiKo2017-ICPE-MemoryPower, abstract = {{High performance workloads with high bandwidth memory utilization are among the most power consuming software applications. When writing such applications, developers can directly influence power consumption of the final software through their choice of data size and traversal method, mostly due to caching characteristics. Explicit knowledge on how choices influence power consumption can thus lead to greater overall energy efficiency. In existing work, power prediction for memory accesses and high bandwidth applications requires either detailed measurement information on the system on which the software is executed or it is too generic, not taking significant aspects, such as caching and data size into account. In this paper, we propose a power model that bridges this gap by modeling power consumption based on concrete software properties, while considering hardware characteristics on a more abstract level, characterizing it primarily using publicly available data. The model is designed to enable developers to compare power consumption of implementation alternatives for high memory bandwidth software components. We validate our model by measuring modified versions of the high bandwidth benchmark stream. We show that our model can predict the relative change of power consumption due to implementation changes and the power consumption of a concrete system under test with an average error of 19 percent.}}, address = {New York, NY, USA}, author = {Schmitt, Norbert and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering}, keywords = {Power}, pages = {353–356}, publisher = {ACM}, series = {ICPE '17}, title = {{Predicting Power Consumption of High-Memory-Bandwidth Workloads}}, year = 2017 }

  %0 Conference Paper %1 ScKiKo2017-ICPE-MemoryPower %A Schmitt, Norbert %A von Kistowski, Jóakim %A Kounev, Samuel %B Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering %C New York, NY, USA %D 2017 %I ACM %P 353--356 %T Predicting Power Consumption of High-Memory-Bandwidth Workloads %U http://doi.acm.org/10.1145/3030207.3030241 %X High performance workloads with high bandwidth memory utilization are among the most power consuming software applications. When writing such applications, developers can directly influence power consumption of the final software through their choice of data size and traversal method, mostly due to caching characteristics. Explicit knowledge on how choices influence power consumption can thus lead to greater overall energy efficiency. In existing work, power prediction for memory accesses and high bandwidth applications requires either detailed measurement information on the system on which the software is executed or it is too generic, not taking significant aspects, such as caching and data size into account. In this paper, we propose a power model that bridges this gap by modeling power consumption based on concrete software properties, while considering hardware characteristics on a more abstract level, characterizing it primarily using publicly available data. The model is designed to enable developers to compare power consumption of implementation alternatives for high memory bandwidth software components. We validate our model by measuring modified versions of the high bandwidth benchmark stream. We show that our model can predict the relative change of power consumption due to implementation changes and the power consumption of a concrete system under test with an average error of 19 percent.

• 
  Asking ``What?’’, Automating the ``How?’’: The Vision of Declarative Performance Engineering. Walter, Jürgen; van Hoorn, Andre; Koziolek, Heiko; Okanovic, Dusan; Kounev, Samuel; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

  @inproceedings{WaHoKoOkKo2016-ICPE-DPE_Vision, abstract = {Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.}, author = {Walter, J{\"u}rgen and van Hoorn, Andre and Koziolek, Heiko and Okanovic, Dusan and Kounev, Samuel}, booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}}, keywords = {DQL}, month = {03}, title = {{Asking ``What?'', Automating the ``How?'': The Vision of Declarative Performance Engineering}}, year = 2016 }

  %0 Conference Paper %1 WaHoKoOkKo2016-ICPE-DPE_Vision %A Walter, Jürgen %A van Hoorn, Andre %A Koziolek, Heiko %A Okanovic, Dusan %A Kounev, Samuel %B Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) %D 2016 %T Asking ``What?'', Automating the ``How?'': The Vision of Declarative Performance Engineering %X Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

• 
  Towards Online Performance Model Extraction in Virtualized Environments. Spinner, Simon; Kounev, Samuel; Zhu, Xiaoyun; Uysal, Mustafa; in Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013), N. Bencomo, R. France, S. Götz, B. Rumpe (eds.) (2013). 89–95. CEUR-WS.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.

  @inproceedings{SpKoZhUy2013-mrt-TowardsModelExtraction, abstract = {Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.}, author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Uysal, Mustafa}, booktitle = {Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013)}, editor = {Bencomo, Nelly and France, Robert and G{\"o}tz, Sebastian and Rumpe, Bernhard}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {09}, pages = {89–95}, publisher = {CEUR-WS}, title = {{Towards Online Performance Model Extraction in Virtualized Environments}}, year = 2013 }

  %0 Conference Paper %1 SpKoZhUy2013-mrt-TowardsModelExtraction %A Spinner, Simon %A Kounev, Samuel %A Zhu, Xiaoyun %A Uysal, Mustafa %B Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013) %D 2013 %E Bencomo, Nelly %E France, Robert %E Götz, Sebastian %E Rumpe, Bernhard %I CEUR-WS %P 89--95 %T Towards Online Performance Model Extraction in Virtualized Environments %U http://ceur-ws.org/Vol-1079/mrt13_submission_4.pdf %X Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.
• 
  A Meta-Model for Performance Modeling of Dynamic Virtualized Network Infrastructures. Rygielski, Piotr; Zschaler, Steffen; Kounev, Samuel; in Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013) (2013). 327–330. ACM, New York, NY, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  In this work-in-progress paper, we present a new meta-model designed for the performance modeling of dynamic data center network infrastructures. Our approach models characteristic aspects of Cloud data centers which were not crucial in classical data centers. We present our meta-model and demonstrate its use for performance modeling and analysis through an example, including a transformation into OMNeT++ for performance simulation.

  @inproceedings{RyZsKo2013-DNI-meta-model, abstract = {In this work-in-progress paper, we present a new meta-model designed for the performance modeling of dynamic data center network infrastructures. Our approach models characteristic aspects of Cloud data centers which were not crucial in classical data centers. We present our meta-model and demonstrate its use for performance modeling and analysis through an example, including a transformation into OMNeT++ for performance simulation.}, address = {New York, NY, USA}, author = {Rygielski, Piotr and Zschaler, Steffen and Kounev, Samuel}, booktitle = {Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013)}, keywords = {DNI}, month = {04}, pages = {327–330}, publisher = {ACM}, title = {{A Meta-Model for Performance Modeling of Dynamic Virtualized Network Infrastructures}}, year = 2013 }

  %0 Conference Paper %1 RyZsKo2013-DNI-meta-model %A Rygielski, Piotr %A Zschaler, Steffen %A Kounev, Samuel %B Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013) %C New York, NY, USA %D 2013 %I ACM %P 327--330 %T A Meta-Model for Performance Modeling of Dynamic Virtualized Network Infrastructures %U http://icpe2013.ipd.kit.edu/ %X In this work-in-progress paper, we present a new meta-model designed for the performance modeling of dynamic data center network infrastructures. Our approach models characteristic aspects of Cloud data centers which were not crucial in classical data centers. We present our meta-model and demonstrate its use for performance modeling and analysis through an example, including a transformation into OMNeT++ for performance simulation.
• 
  A Generic Approach for Architecture-level Performance Modeling and Prediction of Virtualized Storage Systems. Noorshams, Qais; Rentschler, Andreas; Kounev, Samuel; Reussner, Ralf; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2013). 339–342. ACM, New York, NY, USA.
  • [ BibTeX ]
  • [ EndNote ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoReKoRe2013-ICPE-WIP, address = {New York, NY, USA}, author = {Noorshams, Qais and Rentschler, Andreas and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {t_wip}, pages = {339–342}, publisher = {ACM}, series = {ICPE '13}, title = {{A Generic Approach for Architecture-level Performance Modeling and Prediction of Virtualized Storage Systems}}, year = 2013 }

  %0 Conference Paper %1 NoReKoRe2013-ICPE-WIP %A Noorshams, Qais %A Rentschler, Andreas %A Kounev, Samuel %A Reussner, Ralf %B Proceedings of the ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2013 %I ACM %P 339--342 %T A Generic Approach for Architecture-level Performance Modeling and Prediction of Virtualized Storage Systems %U http://doi.acm.org/10.1145/2479871.2479921

• 
  Towards self-aware performance and resource management in modern service-oriented systems. Kounev, Samuel; Brosig, Fabian; Huber, Nikolaus; Reussner, Ralf; in Proceedings of the 7th IEEE International Conference on Services Computing (SCC 2010), July 5-10, Miami, Florida, USA (2010). IEEE Computer Society.
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  • [ Download ]
  Modern service-oriented systems have increasingly complex loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment (e.g., varying service workloads) and adapt the system configuration accordingly. In this paper, we describe a long-term vision and approach for designing systems with built-in self-aware performance and resource management capabilities. We advocate the use of architecture-level performance models extracted dynamically from the evolving system configuration and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and performance requirements are continuously satisfied.

  @inproceedings{KoBrHuRe2010-SCC-Towards, abstract = {Modern service-oriented systems have increasingly complex loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment (e.g., varying service workloads) and adapt the system configuration accordingly. In this paper, we describe a long-term vision and approach for designing systems with built-in self-aware performance and resource management capabilities. We advocate the use of architecture-level performance models extracted dynamically from the evolving system configuration and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and performance requirements are continuously satisfied.}, author = {Kounev, Samuel and Brosig, Fabian and Huber, Nikolaus and Reussner, Ralf}, booktitle = {Proceedings of the 7th IEEE International Conference on Services Computing (SCC 2010), July 5-10, Miami, Florida, USA}, keywords = {Self-adaptive-systems}, month = {07}, publisher = {IEEE Computer Society}, title = {{Towards self-aware performance and resource management in modern service-oriented systems}}, year = 2010 }

  %0 Conference Paper %1 KoBrHuRe2010-SCC-Towards %A Kounev, Samuel %A Brosig, Fabian %A Huber, Nikolaus %A Reussner, Ralf %B Proceedings of the 7th IEEE International Conference on Services Computing (SCC 2010), July 5-10, Miami, Florida, USA %D 2010 %I IEEE Computer Society %T Towards self-aware performance and resource management in modern service-oriented systems %X Modern service-oriented systems have increasingly complex loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment (e.g., varying service workloads) and adapt the system configuration accordingly. In this paper, we describe a long-term vision and approach for designing systems with built-in self-aware performance and resource management capabilities. We advocate the use of architecture-level performance models extracted dynamically from the evolving system configuration and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and performance requirements are continuously satisfied.